EP1966988A2 - Systemes de coordonnees de couleur et edition d'image - Google Patents

Systemes de coordonnees de couleur et edition d'image

Info

Publication number
EP1966988A2
EP1966988A2 EP06848818A EP06848818A EP1966988A2 EP 1966988 A2 EP1966988 A2 EP 1966988A2 EP 06848818 A EP06848818 A EP 06848818A EP 06848818 A EP06848818 A EP 06848818A EP 1966988 A2 EP1966988 A2 EP 1966988A2
Authority
EP
European Patent Office
Prior art keywords
color
value
coordinates
coordinate system
coordinate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP06848818A
Other languages
German (de)
English (en)
Other versions
EP1966988A4 (fr
Inventor
Sergey N. Bezryadin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
KWE International Inc
Original Assignee
KWE International Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US11/322,111 external-priority patent/US7495800B2/en
Priority claimed from US11/321,443 external-priority patent/US7489420B2/en
Priority claimed from US11/377,161 external-priority patent/US7486418B2/en
Priority claimed from US11/432,221 external-priority patent/US7589866B2/en
Priority claimed from US11/494,393 external-priority patent/US7639396B2/en
Application filed by KWE International Inc filed Critical KWE International Inc
Publication of EP1966988A2 publication Critical patent/EP1966988A2/fr
Publication of EP1966988A4 publication Critical patent/EP1966988A4/fr
Withdrawn legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/40—Picture signal circuits
    • H04N1/40012—Conversion of colour to monochrome
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/46—Colour picture communication systems
    • H04N1/56—Processing of colour picture signals
    • H04N1/60—Colour correction or control
    • H04N1/6027—Correction or control of colour gradation or colour contrast

Definitions

  • the present invention relates to digital representation, processing, storage and transmission of images, including editing of polychromatic and monochromatic color images.
  • a digital representation of an image can be stored in a storage device (e.g. a computer memory, a digital video recorder, or some other device). Such representation can be transmitted over a network, and can be used to display the image on a computer monitor, a television screen, a printer, or some other device.
  • the image can be edited using a suitable computer program.
  • Color is a sensation caused by electromagnetic radiation (light) entering a human eye.
  • the light causing the color sensation is called "color stimulus”.
  • Color depends on the radiant power and spectral composition of the color stimulus, but different stimuli can cause the same color sensation. Therefore, a large number of colors can be reproduced ("matched") by mixing just three "primary” color stimuli, e.g. a Red, a Blue and a Green.
  • the primary stimuli can be produced by three "primary" light beams which, when mixed and reflected from an ideal diffuse surface, produce a desired color.
  • the color can be represented by its coordinates, which specify the intensities of the primary light beams.
  • a color S is represented by coordinates R, G, B which define the intensities of the respective Red, Green and Blue primary light beams needed to match the color S.
  • lfP( ⁇ ) is the radiance (i.e. the energy per unit of time per unit wavelength) of a light source generating the color S, then the RGB coordinates can be computed as:
  • F(A) , g(X) , b (X) are "color matching functions" (CMF's).
  • CMF's color matching functions
  • the values F(A) , g(X) , b (X) are respectively the R, G and B values needed to match color produced by a monochromatic light of the wavelength ⁇ of a unit radiance.
  • the color matching functions are zero outside of the visible range of the ⁇ values, so the integration limits in (1) can be replaced with the limits of the visible range.
  • the integrals in (1) can be replaced with sums if the radiance P( ⁇ ) is specified as power at discrete wavelengths.
  • Fig. 1 illustrates the color matching functions for the 1931 CIE RGB color coordinate system for a 2° field.
  • the RGB system of Fig. 1 is called linear because, as shown by equations (1), the R, G, and B values are linear in P( ⁇ ). In a linear system, the intensities such as R, G, B are called "tristimulus values”.
  • the function F(A) can be negative, so the R coordinate can be negative. If R is negative, this means that when the color S is mixed with
  • New linear color coordinate systems can be obtained as non-degenerate linear transformations of other systems.
  • the 1931 CIE XYZ color coordinate system for a 2° field is obtained from the CIE RGB system of Fig. 1 using the following transformation:
  • This XYZ system does not correspond to real, physical primaries.
  • the color matching functions x( ⁇ ) , 7(A) , ⁇ z (A) for this XYZ system are shown in Fig. 2. These color matching functions are defined by the same matrix A RGB - XYZ '•
  • the tristimulus values X, Y, Z can be computed from the color matching functions in the usual way:
  • Non-linear color coordinate systems There are also non-linear color coordinate systems.
  • a nonlinear sRGB system standardized by International Electrotechnical Commission (IEC) as IEC 61966-2-1.
  • the sRGB coordinates can be converted to the XYZ coordinates (4) or the CIE RGB coordinates (1).
  • Another example is HSB (Hue, Saturation, Brightness).
  • the HSB system is based on sRGB. In the HSB system, the colors can be visualized as points of a vertical cylinder.
  • the Hue coordinate is an angle on the cylinder's horizontal circular cross section.
  • the angles between 0° and 120° correspond to mixtures of the Red and the Green; the angles between 120° and 240° correspond to mixtures of the Green and the Blue; the angles between 240° and 360° correspond to mixtures of the Red and the Blue.
  • the Saturation is maximal, which means that the White amount is 0 (this means that at least one of the R, G, and B coordinates is 0).
  • the Brightness is measured along the vertical axis of the cylinder, and is defined as max(R,G,B).
  • the sRGB system is convenient for rendering color on certain types of monitors which recognize the sRGB coordinates and automatically convert these coordinates into color.
  • the HSB system is convenient for some color editing operations including brightness adjustments.
  • Brightness can be thought of as a degree of intensity of a color stimulus. Brightness corresponds to our sensation of an object being "bright” or “dim". Brightness has been represented as the Y value of the XYZ system of Fig. 2, or as the maximum of the R, G and B coordinates of the sRGB coordinate system. Other representations also exist. Hue is the attribute of a color perception denoted by blue, green yellow, red, purple, and so on. See the Wyszecki & Styles book cited above, page 487. Saturation can be thought of as a measure of the white amount in the color.
  • Contrast can be thought of as the brightness difference between the brightest and the dimmest portions of the image or part of the image.
  • Sharpness relates to object boundary definition.
  • the image is sharp if the object boundaries are well defined.
  • the image is blurry if it is not sharp.
  • color coordinate systems which facilitate brightness editing and other types of image editing. For example, it may be desirable to highlight an image area, i.e. to increase its brightness, without making large changes to other image areas. Conversely, it may be desirable to shadow (i.e. to dim) a selected image area without making large changes in other image areas. It is also desirable to obtain color coordinate systems which facilitate transforming a polychromatic image into a monochromatic image (for example, in order to display a polychromatic image using a monochromatic printer or monitor, or to achieve a desired visual effect). It may also be desirable to change the hue or saturation, for example to change the hue of a monochromatic image.
  • Some embodiments of the present invention provide novel color coordinate systems and novel image editing techniques suitable for novel and/or conventional color coordinate systems.
  • the contrast is edited as follows. Let us suppose that a digital image consist of a number of pixels. Let us enumerate these pixels asp ⁇ ,p2,... (The invention is not limited to any pixel enumeration.) Let B(p t ) denote the brightness at the pixel p, (J-1 ,2, ). The contrast editing can be performed by changing the brightness B (pi) to the value
  • the invention is not limited to such embodiments.
  • the brightness can be edited according to the equation:
  • B ⁇ vg (pi) is a function of the B values of image portions in an image region R(p,) containing the portion p t .
  • Some embodiments use a color coordinate system in which the brightness B is one of the coordinates.
  • the brightness B can be changed without changing the chromaticity coordinates.
  • the chromaticity coordinates are defined as:
  • a non-linear color coordinate system is used in which the brightness B is defined by a square root of a quadratic polynomial in tristimulus values,
  • T ⁇ , T 2 , T 3 are tristimulus values.
  • B may or may not be one of the coordinates, but one of the coordinates is defined by B alone or in combination with the sign of one or more of T 1 , T 2 , T 3 .
  • the color coordinate system has the following coordinates:
  • T x I(T 1 + T 2 + T 3 ), T 2 Z(T 1 + T 2 + T 3 ), T 3 I(T 1 + T 2 + T 3 ).
  • the (B, S 2 , S 3 ) coordinate system facilitates color editing when it is desired not to change the chromaticity coordinates (so that no color shift would occur).
  • xyY which also allows changing only one coordinate Y without changing the chromaticity coordinates.
  • the xyY system differs from the (B, S 2 , S 3 ) system in that the Y coordinate is a linear coordinate and is a tristimulus value, while the B coordinate is a non-linear function of tristimulus values (and of the power distribution P( ⁇ )).
  • the Y coordinate is a linear coordinate and is a tristimulus value
  • the B coordinate is a non-linear function of tristimulus values (and of the power distribution P( ⁇ )).
  • the techniques described above can be used to adjust either the global contrast, i.e. when the region R(p t ) is the whole image, or the local contrast, when the region R(p t ) is only a part of the image.
  • the region R(p ⁇ ) contains 10% to 30% of the image in the local contrast adjustments, and other percentages are possible.
  • the inventor has observed that the same techniques can be used to change the image sharpness if the region R (p t ) is small, e.g. 1% of the image or less.
  • the region R(pi) is contained in a rectangle of at most 31 pixels by 31 pixels, with the center at pixel p t .
  • Even smaller outer rectangles can be used, e.g. 21 x 21 pixels, or 11 x 11 pixels, and other rectangles.
  • the image may contain thousands, millions, or some other number of pixels.
  • a dynamic range is the ratio of the maximum brightness B max to the minimum brightness B min .
  • the brightness can be defined in a number of ways some of which are discussed above and below.
  • For the global dynamic range the maximum and minimum brightness values are computed over the whole image (or a large portion of the image).
  • For a local dynamic range the maximum and minimum brightness values are computed over a small area.
  • "dynamic range" of an image is presumed global unless stated to the contrary. Suppose, for example, that an image has a (global) dynamic range and the image must be displayed on a monitor capable of a maximum dynamic range of only 300.
  • BQ and ⁇ are some constants.
  • B*(p ⁇ )/B*(p J ) [B(p ⁇ )/B(p ] )-]x(B ⁇ vg (p J )IB ⁇ vg (p 1 )) ⁇ (4C) If P 1 and p ⁇ are near each other, than B ⁇ vg (p,) and B ⁇ vg (p j ) are likely to be close to each other. Hence, the coefficient (B ⁇ vg (p J )IB ⁇ vg (p ⁇ )) ⁇ is small, so the maximum of the values B(p,)/B(p j ) over a small area will not change much. (The local dynamic range depends of course on how the "small area" is defined. In some embodiments, the area is defined so that the average brightness B ⁇ V g does not change much in the area, but other definitions are also possible. The invention is not limited to embodiments with small changes of the local dynamic range.)
  • BQ the minimum brightness.
  • BQ the maximum brightness and ⁇ to be negative.
  • the global dynamic range is increased if ⁇ is positive, and the global dynamic range is decreased if ⁇ is negative.
  • equation (4B) can be written as: where
  • Color editing (4B)-(4D) can be combined with other types of editing performed in separate steps or simultaneously.
  • highlighting of an image area can be combined with brightening the whole image (multiplying the brightness by a constant k> ⁇ ) via the following transformation:
  • the coordinates S 2 and S 3 can be set to a predefined value representing the desired color (e.g. white for a black-and-white monochromatic image), and the value B can be left unchanged for each pixel.
  • the brightness will be unchanged.
  • the B coordinate can be processed to change the image brightness or contrast or achieve some other brightness-related effect.
  • Another embodiment uses a color coordinate system BCH in which the B
  • B represents the brightness
  • C represents the chroma (saturation)
  • H represents the hue.
  • a color-to-monochromatic image conversion, or changing the color of a monochromatic image can be achieved by setting C and H to predefined values representing the desired color.
  • the B coordinate can be unchanged, or can be changed as needed for brightness-related processing.
  • the invention is not limited to the features and advantages described above.
  • the invention is not limited to editing an image to fit it into a dynamic range of a monitor, a printer, or some other display device.
  • the image editing may be performed to improve the image quality or for any other artistic, esthetic, or other purposes.
  • Other features are described below.
  • the invention is defined by the appended claims.
  • Figs. 1 and 2 are graphs of color matching functions for prior art color coordinate systems.
  • Fig. 3 illustrates some color coordinate systems according to some embodiments of the present invention.
  • FIGs. 4 and 5 are block diagrams illustrating color editing according to some embodiments of the present invention.
  • Fig. 6 illustrates an image edited according to some embodiments of the present invention.
  • Fig. 7 illustrates an image extension at the image boundaries for image editing according to some embodiments of the present invention.
  • Figs. 8, 9 are brightness graphs illustrating contrast editing according to some embodiments of the present invention.
  • Fig. 10 is a flowchart of an image editing method according to some embodiments of the present invention.
  • Fig. 11 is a block diagram illustrating color editing according to some embodiments of the present invention.
  • Figs. 12-14 are brightness graphs illustrating image editing according to some embodiments of the present invention.
  • Figs. 15, 16 illustrate image values transformed via image editing according to some embodiments of the present invention.
  • Figs. 17 and 18 are block diagrams illustrating color editing according to some embodiments of the present invention.
  • a linear color coordinate system DEF is defined as a linear transformation of the 1931 CIE XYZ color coordinate system of Fig. 2:
  • the DEF coordinate system corresponds to color matching functions d(X) , e ' (X) , /(A) which can be obtained from x(X) , y(X), z( ⁇ ) using the same matrix AXYZ-DEF ' ⁇
  • the color matching functions d(X) , e " ( ⁇ ) , /(A) form an orthonormal system in the function space Z 2 on [0, ⁇ ) (or on any interval containing the visible range of the ⁇ values if the color matching functions are zero outside of this range), that is:
  • K is a positive constant defined by the measurement units for the wavelength ⁇ and the radiance P( ⁇ ).
  • the DEF coordinate system can be thought of as a Cartesian coordinate system having mutually orthogonal axes D, E, F (Fig. 3), with the same measurement unit for each of these axes.
  • the dot product (11) does not depend on the color coordinate system as long as the color coordinate system is orthonormal in the sense of equations (9) or (10) and its CMF' s are linear combinations of x( ⁇ ) , y( ⁇ ) , z( ⁇ ) (and hence of d(X) , if( ⁇ ) , /(A) ). More particularly, let T ⁇ , T 2 , T 3 be tristimulus values in a color coordinate system whose CMF's T ⁇ , T 2 , T 3 belong to a linear span Span(d( ⁇ ) ,e ⁇ ( ⁇ ) ,/( ⁇ ) ) and satisfy the conditions (9) or (10). For the case of equations (9), this means that:
  • the brightness B of a color S can be represented as the length (the norm) of the vector S:
  • the Bef color coordinate system is defined as follows:
  • the Bef system is convenient for brightness editing because the brightness can be changed by changing the B coordinate and leaving e and /unchanged.
  • the brightness can be changed over the whole image or over a part of the image as desired.
  • the brightness transformation can be performed using computer equipment known in the art.
  • a computer processor 410 receives the color coordinates B, e, /(from a memory, a network, or some other source) and also receives the factor k (which can be defined by a user via some input device, e.g. a keyboard or a mouse).
  • the processor outputs the kxB, e, /coordinates.
  • this color editing is controlled by a user who watches the color on a monitor 420 before and after editing.
  • the processor 410 may have to convert the color from the Bef coordinate system to some other coordinate system suitable for the monitor 420, e.g. sRGB or some other system as defined by the monitor specification.
  • the color conversion can be performed by converting the color between Bef and DEF as specified by equations (14) and (15), and between DEF and the XYZ system of Fig. 2 by equations (5) and (6).
  • the color conversion between the XYZ coordinate system and the sRGB can be performed using known techniques.
  • Transformation (16) is a convenient way to perform expocorrection in a digital photographic camera (processor 410 can be part of the camera), or to obtain a result equivalent to expocorrection when editing a digital image.
  • Another color coordinate system that facilitates the brightness editing is the spherical coordinate system for the DEF space.
  • This coordinate system BCH (Brightness, Chroma, Hue) is defined as follows (see also Fig. 3):
  • C is the angle between the color S and the D axis
  • H is the angle between (i) the orthogonal projection S EF of the vector S on the EF plane and (ii) the E axis.
  • the H coordinate of the BCH system represents the angle between the projection S EF and the red color represented by the E axis, and thus the H coordinate is a good representation of hue.
  • Transformation from SCH to DEF can be performed as follows:
  • Transformation from DEF to BCH can be performed as follows.
  • the B coordinate can be computed as in (18).
  • the C and H computations depend on the range of these angles. Any suitable ranges can be chosen.
  • H tan 1 (f/e) +a where or depends on the range of H.
  • the angle His measured from the positive direction of the E axis, and is in the range from 0 to 2 ⁇ or - ⁇ to ⁇ . In some embodiments, one of the following computations is used:
  • H arctg(e,f) where arctg is computed as shown in pseudocode in Appendix 1 at the end of this section before the claims.
  • Transformation from BCH to Bef can be performed as follows:
  • the BCH system has the property that a change in the B coordinate without changing C and H corresponds to multiplying the tristimulus values D, E, F by some constant k (see equations (17)).
  • the brightness editing is therefore simple to perform.
  • the brightness editing is performed like in Fig. 4, with the C and H coordinates used instead of e and/
  • the ite/and 5CH systems are also suitable for contrast editing.
  • the contrast is enhanced by increasing the brightness difference between brighter and dimmer image portions.
  • B(pJ denote the brightness value (the B coordinate) at the pixel /? rock
  • B (pj denote the modified brightness to be computed to edit the contrast.
  • B av% (pJ be some average brightness value, for example, the mean of the brightness values in some region R(p,) containing the pixel p,:
  • N(pJ is the number of the pixels in the region R(P 1 ).
  • N(pJ is the number of the pixels in the region R(P 1 ).
  • the new coordinates B (p,), e * (p,), f * (pj for the contrast adjustment are computed from the original coordinates B(p,), e(p,), f(pj as follows:
  • the contrast increases because the brightness range is increased over the image. If ⁇ , the contrast decreases.
  • the contrast adjustment can be performed over pixels p, of a part of the image rather than the entire image. Also, different rvalues can be used for different pixels p t . Alternatively, the same rvalue can be used for all the pixels.
  • the region R(pJ is the entire image.
  • the value B avg (pj is constant, and can be computed only once for all the pixels p,.
  • different regions R(pJ are used for different pixels.
  • the region R(pJ is a square centered at the pixel p, and having a side of 5 pixels long (the length measured in pixels can be different along the x and y axes if a different pixel density is present at the two axes).
  • the region R(p t ) is a non-square rectangle centered at p t , or a circle or an ellipse centered at/?,.
  • the regions R(Pi) have the same size and the same geometry for different pixels p,, except possibly at the image boundary.
  • the regions R(p t ) corresponding to pixels p, at the image boundary have fewer pixels than the regions corresponding to inner pixels/* / .
  • the image is extended (or just the brightness values are extended) to allow the regions R(pt) to have the same size for all the pixels p t .
  • Different extensions are possible, and one extension is shown in Fig. 7 for the square regions R(p t ) of Fig. 6.
  • B(-l, O) B(1, 0)
  • B(m,0) B(m-2,0)
  • Fig. 7 shows the pixel's coordinates for each pixel and the brightness value for each extension pixel.
  • Figs. 8 and 9 are brightness graphs illustrating the brightness before and after editing near a color boundary for one example.
  • the image is assumed to be one-dimensional.
  • Each pixel/* has a single coordinate x.
  • the coordinate x is the coordinate of some predetermined point in the pixel, e.g. the left edge of the pixel (the x coordinates are discrete values).
  • B 0Vg is a weighted average:
  • each value w(p,pi) is completely defined by (x-Xj,y-yd where (x,y) are the coordinates of the pixel/? and (xi,y ⁇ ) are the coordinates of the pixel p t .
  • all the weights are positive.
  • the sum of the weights is 1. The sum of the weights can also differ from 1 to obtain a brightness shift. The brightness shift and the contrast change can thus be performed at the same time, with less computation than would be needed to perform the two operations separately.
  • the contrast editing techniques described above can be used to adjust either the global contrast, i.e. when the region R(p t ) is the whole image, or the local contrast, when the region R(p t ) is only a part of the image.
  • the region R(p t ) contains 10% to 30% of the image in the local contrast adjustments.
  • the same techniques can be used to change the image sharpness if the region R(Pi) is small, e.g. 1% of the image or less.
  • the region R(p t ) is contained in a rectangle of at most 31 pixels by 31 pixels, with the center at pixel p ⁇ .
  • Even smaller outer rectangles can be used, e.g. 21 x 21 pixels, or 11 x 11 pixels, and other rectangles.
  • the image may contain thousands, millions, or some other number of pixels.
  • a computer system receives an image editing command at step 1010.
  • the command can be issued by a user via a graphical user interface.
  • the command may specify contrast or sharpness editing. If the command is for contrast editing (step 1020), the computer system sets the size of region R(p t ) to a suitable value (e.g. 10% ⁇ 30% of the image), or defines the region R(p ⁇ ) to be a large specific region (e.g. the whole image). If the command is for sharpness editing (step 1030), the computer system sets the size of region R(Pi) to a small value (e.g. 1% of the image, or a rectangle of 11 pixels by 9 pixels), or defines the region R(p t ) to be a small specific region.
  • an editing algorithm is executed as described above with the region or regions R(p t ) as defined at step 1020 or 1030. Step 1040, or steps 1020-1040, can be repeated for different pixels p t .
  • the image processing system can be made simpler and/or smaller due to the same algorithm being used for the contrast and sharpness editing.
  • the contrast and sharpness editing can also be performed on monochromatic images. These images are formed with a single color of possibly varying intensity. In such images, the brightness B represents the color's intensity. In some embodiments, the brightness B is the only color coordinate. The same techniques can be employed for this case as described above except that the equations (27.2), (27.3), (29.2), (29.3) can be ignored.
  • the Bef and BCH systems are also suitable for changing an image's dynamic range according to equations (4A)-(4E).
  • the brightness is changed as in equation (4B) with the C and H coordinates remaining unchanged.
  • the Bef system can be used, with B edited according to (4B) and with the e and/coordinates unchanged.
  • B(p t ) denote the brightness value (the B coordinate) at the pixel /? stamp and B * (p,) denote the modified brightness.
  • B avg (p,) may be some average brightness value, for example, the mean of the brightness values in some region R(p ⁇ ) containing the pixel p, as in (26).
  • Bo is some pre-defined brightness level
  • BavgfP t is some average brightness value, e.g. the mean brightness over a region R(p,) containing the pixel p, as in (26), or a weighted average as described below in connection with equation (36); ⁇ is a pre-defined constant. If it is desired to increase the brightness at the pixels/?, at which B avg (p l ) ⁇ Bo, then ⁇ is positive (in some embodiments, ⁇ is in the interval (0,1)). Good results can be achieved for BQ being the maximum brightness and 0 ⁇ £ ⁇ l/2. If it is desired to reduce the brightness (e.g. to deepen the shadow) at the pixels /?, at which B avg (p,) ⁇ Bo, then ⁇ is negative (e.g. in the interval (-1 ,0)).
  • the region R(p,) is a square centered at the pixel p, and having a side of 5 pixels long (the length measured in pixels can be different along the x and y axes if a different pixel density is present at the two axes).
  • the region R(p,) is a non-square rectangle centered ZAp 1 , or a circle or an ellipse centered at/?,.
  • the regions R(p t ) have the same size and the same geometry for different pixels/?,, except possibly at the image boundary.
  • the region R(P 1 ) does not have to be symmetric or centered at/?,.
  • the regions R(pJ corresponding to pixels/?, at the image boundary have fewer pixels than the regions corresponding to inner pixels/?,.
  • the image is extended using any one of the methods discussed above in connection with contrast editing (see Fig. 7 for example).
  • the regions R(pJ are chosen such that the brightness does not change much in each region (e.g. such that the region does not cross a boundary at which the derivative of the brightness changes by more than a predefined value).
  • Fig. 12 illustrates the brightness values B and B* along an axis x assuming a one-dimensional image.
  • the brightness is shown as if changing continuously, i.e. the pixels are assumed to be small.
  • the value BQ is chosen to equal B max (the maximum brightness over the image), and ⁇ is about 1/2.
  • the minimum brightness value is the mean brightness in some neighborhood R(x) of the point x.
  • the original brightness B is near B mtn , i.e. about i? 0 /4.
  • the value B avg is also near Bo/4, so B* is about IB.
  • the difference (B 2 * -B ⁇ *) is about double the difference (B 1 -B x ) between the original brightness values. Therefore, the image details become more visible in the dimmest area.
  • the coefficient (Bo/B avg (pi)) ⁇ in (4B) is a function of B avg , so this coefficient changes smoothly over the image.
  • the visual effect is highlighting of the entire dimmer areas rather than just individual dim pixels (a generally dim area may include individual bright pixels).
  • the local dynamic range therefore does not change much.
  • the smoothness degree can be varied by varying the size of the region R(p t ) over which the average B avg is computed.
  • BQ is set to the minimum brightness value B mtn .
  • This type of editing is useful when the monitor displays the dimmest areas adequately but the brightest areas are out of the monitor's dynamic range.
  • the brightness is reduced in image areas in which B avg >Bo .
  • the brightness is reduced more in the brighter areas, less in the dimmer areas.
  • the regions R(Pi) are as in Fig. 12, but they could be chosen as in Fig. 13 or in some other manner.
  • Fig. 15 illustrates how the values BQ and ⁇ can be chosen in some embodiments.
  • an image editing system e.g. a computer
  • a maximum brightness B max and a minimum brightness B m j n can be, for example, the minimum and maximum brightness of the original image or image portion, or the minimum and maximum brightness to be edited, or the minimum and maximum brightness of a system which created the original image.
  • the image editing system is also provided with a target maximum brightness B* max and a target minimum brightness B* min . Substituting the values B max , B min , B* max , B* min into (4B), and assuming for simplicity that when B(p t ) is equal to B min or B max , we obtain:
  • Equation (30A) can be divided by (30B) to eliminate BQ and solve for ⁇ , and then the ⁇ solution can be substituted into (30A) or (30B) to find B 0 .
  • the solution is:
  • DR B max /B min
  • DR* B* mca /B* min
  • the logarithm can be to any base.
  • the image editing system is provided a starting dynamic range DR and a target dynamic range DR*. It is desired to obtain the values BQ and ⁇ so as to convert some predefined brightness value B ⁇ to a predefined brightness value B ⁇ *.
  • Substituting ⁇ 1 and .S 1 * into (4B), and assuming for simplicity that B avg (Pi) B(pi) when B(p t ) is equal to B ⁇ , we obtain:
  • the value BQ can be any value, not necessarily the minimum or maximum brightness.
  • the brightness values below BQ are modified as in Fig. 12 in some embodiments, and the brightness values above Bo as in Fig. 14.
  • BQ is chosen as the best brightness value for a given display device, i.e. the value at which (or near which) the display device provides the best image quality.
  • the transformation (4B) with ⁇ >0 tends to squeeze the brightness values closer to j? o . When ⁇ 0, the transformation (4B) tends to spread the brightness values wider above and/or below BQ.
  • Equation (4B) uses equations other than (4B).
  • equation (4A) where f(y) is a non-negative function strictly monotonic inj ⁇ .
  • the function is chosen to be strictly decreasing iny. Hence, as B ms increases, the modified brightness B* decreases for a given original brightness value B. In other embodiments, the function is chosen to be strictly increasing in y for each fixed x.
  • f(x,y) is strictly increasing in x for each fixed y.
  • f(x, Bo) X for all x for a predefined value BQ.
  • BQ average brightness
  • the transformation (35) can be applied to an entire image (e.g. a rectangular array of pixels) or a part of the image.
  • B avg is a weighted average:
  • each value w(p,pi) is completely defined by (x-x i: y-yi) where (x,y) are the coordinates of the pixel/; and (xi,y t ) are the coordinates of the pixel/?,.
  • all the weights are positive.
  • the sum of the weights is 1.
  • the sum of the weights can also differ from 1 to obtain a brightness shift. The brightness shift and the dynamic range change can thus be performed at the same time, with less computation than would be needed to perform the two operations separately.
  • the editing can also be performed on monochromatic images. These images are formed with a single color of possibly varying intensity.
  • the brightness B represents the color's intensity. In some embodiments, the brightness B is the only color coordinate.
  • the invention is not limited to the Befand BCH coordinate systems, or the brightness values given by equation (13).
  • the brightness B is defined in some other way such that multiplication of the brightness by a value k in some range of positive values corresponds to multiplication of all the tristimulus values by the same value.
  • the chromaticity coordinates remain unchanged.
  • the brightness can be defined by the equation (49) discussed below.
  • the Befand BCH systems are also convenient for performing hue and saturation transformations.
  • the brightness B is unchanged.
  • the hue transformation corresponds to rotating the color vector S (Fig. 3) around the D axis.
  • the chroma coordinate C is unchanged for the hue transformation.
  • the color vector rotation is to be performed by an angle ⁇ , i.e. the angle His changed to H+ ⁇ .
  • H computation places the result into the proper range from H min to H m ⁇ x , e.g. from - ⁇ to ⁇ or from 0 to 2 ⁇ . If the H value is outside the interval, then 2 ⁇ is added or subtracted until the H value in the interval is obtained. These additions and subtractions can be replaced with more efficient techniques, such as division modulo 2 ⁇ , as known in the art. In some embodiments, the ">" sign is replaced by ">” and " ⁇ " by " ⁇ ” (depending on whether the point H min or H m ⁇ x is in the allowable range of the H values).
  • a maximum saturation value C m ⁇ x (H) For a given hue H some embodiments specify a maximum saturation value C m ⁇ x (H). This can be done for one or more hue values H, possibly for all the hue values. If C exceeds C max (H), the corresponding color (B 1 QEF) may be invisible to a human being, or may be impossible to display on a given display device. Thus, for a given H, the value C max (H) may be specified as the maximum saturation visible to humans, or the maximum saturation reproducible on a display device, or as some value below the maximum visible or reproducible saturation. In some embodiments, if the hue is changed as in (37), then the saturation is changed so that C* does not exceed C max (H*). For example:
  • a saturation transformation can be specified as the change in the chroma angle C, e.g. multiplying C by some positive constant k. The hue and the brightness are unchanged. Since D is never negative, the C coordinate is at most ⁇ /2.
  • the maximum C value C max (H) can be a function of the H coordinate.
  • the saturation transformation can be performed as follows:
  • the desired saturation transformation by specifying the desired change in the e and/coordinates.
  • the saturation change corresponds to rotating the S vector in the plane perpendicular to the EF plane.
  • the angle H remains unchanged.
  • Equations (14), (15) imply that e 2 +f 2 ⁇ l (41) Therefore, the values e , f * may have to be clipped. In some embodiments, this is done as follows:
  • the Befana BCH systems are also convenient for converting a color image to a monochromatic image (i.e. a polychromatic image to a monochromatic image), or for changing the color (e.g. the hue) of a monochromatic image.
  • the brightness B can be unchanged.
  • H * H 0 where Co, Ho are some input values corresponding to the desired color.
  • Computer processor 410 (Fig. 18) receives the color coordinates B, C, Hand also receives the values C 0 , H 0 . The processor outputs the coordinates B, C 0 , H 0 for each pixel according to (42.2).
  • the transformation (42.1) or (42.2) can be combined with some brightness editing.
  • the output brightness B* can be computed as in (16) for some input value k, or as in (27.1) or (29A).
  • Other brightness editing types are also possible. See e.g. U.S. patent application no. 11/377,161, entitled "EDITING (INCLUDING CONTRAST AND SHARPNESS
  • the invention includes systems and methods for color image editing and display.
  • the Bef and BCH color coordinates can be transmitted in a data carrier such as a wireless or wired network link, a computer readable disk, or other types of computer readable media.
  • the invention includes computer instructions that program a computer system to perform the brightness editing and color coordinate system conversions.
  • Computer programs with such computer instructions can be transmitted in a data carrier such as a wireless or wired network link, a computer readable disk, or other types of computer readable media.
  • a data carrier such as a wireless or wired network link, a computer readable disk, or other types of computer readable media.
  • the Bef or BCH color coordinate system can be replaced with their linear transforms, e.g. the coordinates (B+e,e,f) or (2B,2e,2f) can be used instead of (B,e,f).
  • the angle H can be measured from the E axis or some other position.
  • the angle C can also be measured from some other position.
  • the invention is not limited to the order of the coordinates.
  • the invention is not limited to DEF, XYZ, or any other color coordinate system as the initial coordinate system.
  • the orthonormality conditions (9) or (10) are replaced with quasi-orthonormality conditions, i.e. the equations (9) or (10) hold only approximately.
  • CMF' s t ⁇ (X) , t 2 (X) , t 3 (X) will be called herein quasi-orthonormal with an error at most ⁇ if they satisfy the following conditions: roo_ _ poo_ _ r°°— — 1. each of / t ⁇ (X)t 2 (X)dX , I t ⁇ ( ⁇ )t 3 (X)d ⁇ , I t 2 (X)t 3 (X)dX is in the interval J O J 0 " 0
  • d ⁇ is in the interval [K- ⁇ ,K+ ⁇ for positive constants K and ⁇ .
  • the CMF's will be called quasi- orthonormal with an error at most ⁇ if they satisfy the following conditions:
  • each of is in the interval [- ⁇ , ⁇ ], and ⁇ ⁇ ⁇
  • each of ⁇ [T X (X)] 2 , ⁇ [T 2 (X)] 2 , ⁇ [T 3 (X)] 2 is in the interval [K- ⁇ ,K+ ⁇ ] ⁇ ⁇ ⁇ for positive constants K and ⁇ .
  • £" is 0.3K, or OAK, or some other value at most 0.3K, or some other value.
  • the invention is not limited to the orthonormal or quasi-orthonormal CMF' s or to any particular white color representation.
  • the following color coordinate system (S ⁇ ,S 2 ,S 3 ) is used for color editing:
  • S 2 XZS x
  • the XYZ tristimulus values in (43) can be replaced with linear RGB tristimulus values or with some other tristimulus values T 1 , T 2 , T 3 , e.g.:
  • S 2 T 2 ZSi If Ti can be negative, than the sign of T 1 can be provided in addition to the coordinates. Alternatively the sign of Ti can be incorporated into one of the Si, S 2 , and/or S 3 values, for example:
  • the brightness editing can still be performed by multiplying the Si coordinate by k, with the £ 2 and S 3 values being unchanged.
  • the BCH coordinate system can be constructed from linear systems other than DEF, including orthonormal and non-orthonormal, normalized and non-normalized systems.
  • a coordinate system is used with coordinates (S ⁇ ,S 2 ,S 3 ), where S 1 is defined as in (44) or (45), and the coordinates S 2 and S 3 are defined in a way similar to (20)-(23), e.g.:
  • the value B is the square root of some other quadratic polynomial of in Ti, T 2 , T 3 :
  • the coordinate system may use coordinates S 1 , S 2 , S 3 or their linear transform, wherein S 1 is defined by the value B, or Si is defined by B and the sign of a predefined function of one or more Of T 1 , T 2 , T 3 .
  • S 1 is defined by the value B
  • Si is defined by B
  • the sign of a predefined function of one or more Of T 1 , T 2 , T 3 may be used.
  • a n xTi+ai 2 xT 2 +ai3xT 3 cc 2i xT ⁇ + CC 22 XT 2 + (X 23 XT 3 a 3 (T ⁇ J 2 J 3 ) a 3X xT ⁇ + CC 32 XT 2 + Os 3 XT 3
  • CU 11 , ⁇ 12 , ⁇ 13 , a 2 ⁇ , a 22 , O 23 , ⁇ r 31 , a 32 , a 33 are predefined numbers such that the following matrix A is non-degenerate:
  • the coordinate Si is defined by the value B and by a sign of a predefined function of one or more Of T 1 , T 2 , T 3 .
  • the function can be one of Qr 1 (T ⁇ J 2 J 3 ), OC 2 (TiJ 2 J 3 ), Cc 3 (TiJ 2 J 3 ).
  • T 3 CC 3 (TiJ 2 J 3 ) are also tristimulus values.
  • the coordinate S 2 is defined by a value ⁇ (T ⁇ J 2 J 3 )IB, wherein wherein ⁇ , /J 2 , /% are predefined numbers at least one of which is not equal to zero, or the coordinate S 2 is defined by the value ⁇ (T ⁇ , T 2 J 3 )IB and the sign of a predefined function of one or more of T ⁇ , T 2 , T 3 .
  • the coordinate S 3 is defined by a value /(Ti, T 2 J 3 )IB, wherein ⁇ T ⁇ + ⁇ 2 ⁇ T 2 + ⁇ 3 ⁇ T 3 , wherein ⁇ , ⁇ i, Y 3 are predefined numbers at least one of which is not equal to zero, and ⁇ (T ⁇ , T 2 J 3 ) is not a multiple of ⁇ (T ⁇ , T 2 J 3 ), or the coordinate S 3 is defined by the value Y(T 1 JjJ 3 )IB and the sign of a predefined function of one or more Of T 1 , Tj, T 3 .
  • a value cos S 2 is defined by a value P(TuTjJi)IB, wherein wherein ⁇ , ft, are predefined numbers at least one of which is not equal to zero, or the value cos Sj is defined by the value ⁇ (T], TjJi)IB and the sign of a predefined function of one or more of T ⁇ , Tj, T 3 .
  • a value tan S 3 is defined by a value /(TiJ 2 J 3 )I ⁇ (Ti,T 2 ,T 3 ), wherein ri ⁇ T ⁇ +n ⁇ T 2 +n ⁇ T 3 , wherein ⁇ 2 , ⁇ 3 are predefined numbers at least one of which is not equal to zero, and S 1 , S 2 , S 3 are predefined numbers at least one of which is not equal to zero, or the value tan S 3 is defined by the value /(T], TjJ 3 )IS (Ti, TjJ 3 ) and the sign of a predefined function of one or more of T ⁇ , Tj, J 3 .
  • ⁇ (T ⁇ , TjJ 3 ) is one of Or 1 (T 1 , TjJ 3 ), U 2 (T 1 J 2 J 3 ), (X 3 (T 1 J 2 J 3 ); 7(T 1 J 2 J 3 ) is another one of CC 1 (T 1 J 2 J 3 ), (Xj(T x J 2 Ji), (Z 3 (T 1 JjJ 3 ); and S(T 1 , T 2 J 3 ) is the third one of U 1 (T 1 , T 2 Ji), CCj(T 1 J 2 Ji), Cc 3 (T 1 J 2 J 3 ).
  • U 1 (T 1 J 2 Ji), U 2 (T 1 J 2 Ji), Cc 3 (T 1 J 2 J 3 ) are tristimulus values corresponding to 70%-orthonormal color matching functions.
  • the tristimulus values T ⁇ , Z 2 , J 3 , or the corresponding color matching functions do not have to be normalized in any way.
  • the color matching functions are normalized to require that the integral of each color matching function be equal to 1.
  • P( ⁇ ) spectral power distribution function
  • the invention is not limited to the two-dimensional images (as in Fig. 6).
  • the invention is applicable to pixel (bitmap) images and to vector graphics. Colors can be specified separately for each pixel, or a color can be specified for an entire object or some other image portion.
  • the invention is not limited to a particular way to specify the brightness transformation or any other transformations.
  • Other embodiments and variations are within the scope of the invention, as defined by the appended claims.
  • ⁇ res tan "1 (b / a) ; if(a ⁇ 0)
  • Temp[offset++] pB [iRight--] ;
  • iRight 2 * nRadius +1;
  • pMeanB [offset] Brightness(plmage[offset]) ;
  • float kNorm 1. / (2 * nRadius Width + 1 ) / (2 * nRadiusHeight + 1 ) ;

Abstract

Selon l'invention, la couleur est éditée en utilisant une représentation de la couleur comprenant des valeurs numériques B (luminosité), e et f telles que B= , e=E/B, f=F/B, où DEF est un système de coordonnées de couleur linéaire. En variante, la couleur est représentée en utilisant des valeurs numériques B, C (chrominance) et H (teinte), où cos C=D/B et tan H=E/F. La luminosité peut être modifiée sans décalage chromatique en modifiant la coordonnée B et en laissant les autres coordonnées e et f ou C et H inchangées. L’invention concerne également d’autres techniques d’édition d’image et d’autres fonctionnalités.
EP06848818A 2005-12-28 2006-12-20 Systemes de coordonnees de couleur et edition d'image Withdrawn EP1966988A4 (fr)

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
US11/322,111 US7495800B2 (en) 2005-12-28 2005-12-28 Color coordinate systems including systems with a coordinate defined by a square root of a quadratic polynomial in tristimulus values and, possibly, by a sign of a function of one or more of tristimulus values
US11/321,443 US7489420B2 (en) 2005-12-28 2005-12-28 Color editing (including brightness editing) using color coordinate systems including systems with a coordinate defined by a square root of a quadratic polynomial in tristimulus values and, possibly, by a sign of a function of one or more of tristimulus values
US11/377,161 US7486418B2 (en) 2005-12-28 2006-03-16 Editing (including contrast and sharpness editing) of digital images
US11/377,591 US7486419B2 (en) 2005-12-28 2006-03-16 Editing (including saturation editing) of digital color images
US11/376,837 US7495801B2 (en) 2005-12-28 2006-03-16 Editing (including hue editing) of digital color images
US11/432,221 US7589866B2 (en) 2005-12-28 2006-05-10 Editing of digital images, including (but not limited to) color-to-monochromatic conversion and changing the color of a monochromatic image
US11/494,393 US7639396B2 (en) 2005-12-28 2006-07-26 Editing of digital images, including (but not limited to) highlighting and shadowing of image areas
PCT/US2006/062421 WO2007076408A2 (fr) 2005-12-28 2006-12-20 Systèmes de coordonnées de couleur et édition d’image

Publications (2)

Publication Number Publication Date
EP1966988A2 true EP1966988A2 (fr) 2008-09-10
EP1966988A4 EP1966988A4 (fr) 2009-07-29

Family

ID=38218816

Family Applications (1)

Application Number Title Priority Date Filing Date
EP06848818A Withdrawn EP1966988A4 (fr) 2005-12-28 2006-12-20 Systemes de coordonnees de couleur et edition d'image

Country Status (2)

Country Link
EP (1) EP1966988A4 (fr)
WO (1) WO2007076408A2 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102857766B (zh) * 2011-08-31 2014-10-15 北京泰邦天地科技有限公司 基于HGlCl颜色空间的颜色处理方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0635971A2 (fr) * 1993-07-19 1995-01-25 Xerox Corporation Appareil et méthode pour convertir les couleurs élargies en couleurs
US20030025835A1 (en) * 2001-08-06 2003-02-06 Oplus Technologies Ltd. Method for independently controlling hue or saturation of individual colors in a real time digital video image
US20040075852A1 (en) * 2002-10-22 2004-04-22 Canon Kabushiki Kaisha Data conversion between color coordinate systems
US6778185B1 (en) * 2002-02-06 2004-08-17 Hewlett-Packard Development Company, L.P. Method and apparatus for mapping colors into a color gamut
EP1708480A2 (fr) * 2005-03-21 2006-10-04 Microsoft Corporation Edition couleur interactive robuste

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH07101916B2 (ja) * 1991-05-14 1995-11-01 富士ゼロックス株式会社 カラー画像編集装置
IL98622A (en) * 1991-06-25 1996-10-31 Scitex Corp Ltd Method and device for using neural networks in figure work
KR100240070B1 (en) * 1997-07-10 2000-01-15 Samsung Electronics Co Ltd Apparatus and method for color calibration in image system
US7116443B2 (en) * 2002-05-02 2006-10-03 Xerox Corporation Neutral pixel detection using color space feature vectors wherein one color space coordinate represents lightness
US7764832B2 (en) * 2003-12-16 2010-07-27 Seiko Epson Corporation System and method for processing image data for color space conversion

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0635971A2 (fr) * 1993-07-19 1995-01-25 Xerox Corporation Appareil et méthode pour convertir les couleurs élargies en couleurs
US20030025835A1 (en) * 2001-08-06 2003-02-06 Oplus Technologies Ltd. Method for independently controlling hue or saturation of individual colors in a real time digital video image
US6778185B1 (en) * 2002-02-06 2004-08-17 Hewlett-Packard Development Company, L.P. Method and apparatus for mapping colors into a color gamut
US20040075852A1 (en) * 2002-10-22 2004-04-22 Canon Kabushiki Kaisha Data conversion between color coordinate systems
EP1708480A2 (fr) * 2005-03-21 2006-10-04 Microsoft Corporation Edition couleur interactive robuste

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Mark D. Fairchild: "Color appearance models", 1998, Addison Wesley Longman Inc., XP002670231, ISBN: 0-201-63464-3 pages 199-214, * pages 200, 211 * *
See also references of WO2007076408A2 *

Also Published As

Publication number Publication date
WO2007076408A3 (fr) 2008-04-03
WO2007076408A2 (fr) 2007-07-05
EP1966988A4 (fr) 2009-07-29

Similar Documents

Publication Publication Date Title
US7639396B2 (en) Editing of digital images, including (but not limited to) highlighting and shadowing of image areas
US7333237B2 (en) Color adjustment method, color adjustment apparatus, color conversion definition editing apparatus, image processing apparatus, program, and storage medium
EP1909486B1 (fr) Procédé et appareil pour réaliser la mise en correspondance de la gamme de couleurs entre dispositifs hétérogènes
EP1100256B1 (fr) Procédé et appareil de conversion de gamme ¬
US7486419B2 (en) Editing (including saturation editing) of digital color images
US20090040564A1 (en) Vision-Based Color and Neutral-Tone Management
JPH11341296A (ja) 色域変換方法及び色域変換装置
US8116562B2 (en) Image processing including, but not limited to, logarithimic coding in color coordinate systems including systems with a coordinate defined by a square root of a quadratic polynomial in tristimulus values and, possibly, by a sign of a function of one or more of tristimulus values
US6744534B1 (en) Robust colorimetry transform
JPH1084487A (ja) 画像処理装置および方法
US8416453B2 (en) Selecting blended regions of color space
US20070171441A1 (en) Color and darkness management system
US20070171442A1 (en) Color and neutral tone management system
US7589866B2 (en) Editing of digital images, including (but not limited to) color-to-monochromatic conversion and changing the color of a monochromatic image
US20020021833A1 (en) Image processing apparatus and method
US7639398B2 (en) Color coordinate systems and image editing
WO2007076408A2 (fr) Systèmes de coordonnées de couleur et édition d’image
US7486418B2 (en) Editing (including contrast and sharpness editing) of digital images
US8488192B2 (en) Method for mapping an out-of-gamut color using a reference gamut mapping
US7495800B2 (en) Color coordinate systems including systems with a coordinate defined by a square root of a quadratic polynomial in tristimulus values and, possibly, by a sign of a function of one or more of tristimulus values
US20160247049A1 (en) Monochrome Profile Generation Mechanism
US7605959B2 (en) System and method of color image transformation
Thomas et al. Cross-media color reproduction and display characterization
Lukac Joint White Balancing and Color Enhancement
US8705115B1 (en) Generation of inverse transfer function for image rendering device

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20080704

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK RS

RIN1 Information on inventor provided before grant (corrected)

Inventor name: BEZRYADIN, SERGEY N.

A4 Supplementary search report drawn up and despatched

Effective date: 20090626

17Q First examination report despatched

Effective date: 20091009

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20120918