EP1716542B1 - Procede et dispositif pour determiner par ordinateur une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numerique apres modification de la couleur de cheveux - Google Patents

Procede et dispositif pour determiner par ordinateur une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numerique apres modification de la couleur de cheveux Download PDF

Info

Publication number
EP1716542B1
EP1716542B1 EP05706768A EP05706768A EP1716542B1 EP 1716542 B1 EP1716542 B1 EP 1716542B1 EP 05706768 A EP05706768 A EP 05706768A EP 05706768 A EP05706768 A EP 05706768A EP 1716542 B1 EP1716542 B1 EP 1716542B1
Authority
EP
European Patent Office
Prior art keywords
hair
color
colour
hair colour
coding information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP05706768A
Other languages
German (de)
English (en)
Other versions
EP1716542A1 (fr
Inventor
Gerrit Fraune
Markus Heinemann
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Henkel AG and Co KGaA
Original Assignee
Henkel AG and Co KGaA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henkel AG and Co KGaA filed Critical Henkel AG and Co KGaA
Publication of EP1716542A1 publication Critical patent/EP1716542A1/fr
Application granted granted Critical
Publication of EP1716542B1 publication Critical patent/EP1716542B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A45—HAND OR TRAVELLING ARTICLES
    • A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
    • A45D44/00—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
    • A45D44/005—Other cosmetic or toiletry articles, e.g. for hairdressers' rooms for selecting or displaying personal cosmetic colours or hairstyle
    • A—HUMAN NECESSITIES
    • A45—HAND OR TRAVELLING ARTICLES
    • A45D—HAIRDRESSING OR SHAVING EQUIPMENT; EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
    • A45D19/00—Devices for washing the hair or the scalp; Similar devices for colouring the hair
    • A45D19/0041—Processes for treating the hair of the scalp
    • A45D19/0066—Coloring or bleaching

Definitions

  • Digital image processing is an important area in the field of hair color counseling, because in this way a person can be brought to the effects of a possible hair coloring on the hair color of the person very quickly and vividly. This is done, for example, by taking a digital image of a person including the hair area of the person, and determining the hair color of the person. Subsequently, a color transformation of the color information associated with the pixels of the hair region to the target hair color according to the desired hair coloring takes place. The image of the person with the hair area, which is now assigned the color-transformed color information, is presented to the person who now gets an idea of the coloring result, without the hair coloring actually having to be performed on the hair.
  • color fastness is very important, in other words the reliability that the result hair color represented to the person corresponds as exactly as possible to the actual result of a hair coloration that would be made on the hair of the person.
  • RGB color space ot R, G reen, B lauryl F arbraum
  • CMY color space C yan, M Agenta, Y ellow color space
  • HLS color space H: color angle in the color wheel
  • L brightness of the color
  • S saturation of the color
  • [2] a method and an apparatus for processing a digitized image stored in a first computer are described.
  • data display unit parameters of a data display unit connected to a second computer are manually set by a user of the data display unit and transmitted from the second computer to the first computer via a telecommunication network.
  • the data display unit parameters describe the data display unit.
  • the first coding information in particular the color information, is changed to second coding information depending on the data display unit parameters in the HLS color space, and the second coding information is transmitted to the first computer.
  • a transformation of coding information from the RGB color space into the HLS color space and vice versa from the HLS color space into the RGB color space is described.
  • a hair color guidance system in which by means of a computer a source hair color of a person in a captured image is virtually mapped to a target hair color. On a screen, a user is presented with the captured image of the person with the original hair color and / or a simulated image of the person with the target hair color.
  • a method for a hair color consultation is described in which individual pictures of a customer are continuously recorded by means of a video camera and are transmitted in real time to the computer. For each frame, a hair area of the customer is identified, the hair color in this hair area according to specifications changed and the changed frames are displayed in real time to a user on a screen.
  • [5] describes a color matching method wherein it is disclosed in [5] to form plural color categories in an HLS color space depending on different Hue values, different luminance values, and different saturation values.
  • a method of changing colors in a digital image is known.
  • the color representation of the color region to be changed with the source color and the target color is mapped into an HLS color space representation. Further, the difference of the luminance values of the source color and the target color is determined.
  • the color change is performed for every pixel, and it is checked for the hue values of the source color whether they are within a range of variation of the hue values of the target color. If this is not the case, the original color is not changed. If this is the case, however, the respective hue value of the pixel of the original color is replaced by the hue value of the target color. Further, in this case, the luminance value of the pixel of the source color is increased by the difference of the luminance values of the source color and the target color. Subsequently, the pixels are transformed into the RGB color space representation.
  • the invention is based on the problem of specifying a method and a device for determining a target hair color for an original hair color of a hair area in a digital image after hair color change, which thus requires less computer-related resources compared to the prior art without the color fastness is reduced.
  • the problem is solved by the method and the device for determining a target hair color for an original hair color of a hair area in a digital image after hair color change with the features according to the independent claims.
  • hair colors are associated with a plurality of predetermined hair color classes.
  • the hair region has in the digital image to pixels to which coding information, encoded in accordance with the HLS color space (H: hue angle in the so-called color circle, L: brightness of the color, S: saturation of the color) is assigned, each pixel in the hair area a Hair color class in the HLS color space is assigned in each case at least, preferably exactly one, a predetermined luminance value according to the hair color class. It is checked if the source hair color and the target hair color are assigned to the same hair color class.
  • HLS color space hue angle in the so-called color circle
  • L brightness of the color
  • S saturation of the color
  • the original hair color coding information associated with the original hair color is transformed to the target hair color coding information associated with the target hair color in the HLS color space, wherein the at least one, preferably exactly one, luminance value (brightness value) of the hair color class of the original hair color is replaced by the at least one luminance value of the hair color class of the target hair color.
  • a device for determining a target hair color for an original hair color of a hair area in a digital image after hair color change has a processor unit which is set up such that the Process steps described above are performed or feasible.
  • the invention can be seen to recognize that the luminance values of hair of a hair color class are substantially the same, in other words, constant. According to the invention, this finding has been used graphically to describe the hair color classes in discrete planes in the HLS color space, also referred to hereafter as hair color class levels, perpendicular to the luminance axis (L axis) of the luminance (L values) Subdivide saturation (s-values) and hue values (h-values) of spanned HLS color space.
  • the entire HLS color space in the context of digital image processing for the description of hair colors is clearly interpretable as a sphere, in which the hair color class levels are introduced, which are aligned normal to the L axis.
  • the invention is regarded as a cylinder whose longitudinal axis is parallel to the L-axis or is formed by it.
  • the replacement of the luminance value according to the invention thus clearly corresponds to a discrete shift of the L values along the L-axis from a hair color class level to an immediately adjacent or even more distant hair color class level.
  • the replacement of the luminance value would clearly correspond to a shift of the L values along a great circle arranged perpendicular to the color circle.
  • the invention makes it possible to carry out the hair color change in the context of digital image processing with considerably reduced computing resources.
  • the replacement of the luminance value or the luminance values takes place only in a predetermined range of hair color class levels which are adjacent to the hair color class level assigned to the original hair color, preferably only in the range of the hair color class levels corresponding to FIG Tolerance range of the respective hair coloring.
  • a reference source hair color coding information and at least one reference target hair color coding information which is generated by applying the hair color change to hair of the reference original hair color are stored wherein the reference source hair color coding information and the reference target hair color coding information are coded according to the HLS color space.
  • the storage is preferably done in a reference table.
  • the original hair color coding information is, according to this embodiment, transformed to the destination hair color coding information in the HLS color space using the reference original hair color coding information corresponding to the original hair color coding information and the corresponding reference destination hair color coding information as target hair color coding information, preferably thus using the reference table in which the above-mentioned information is stored.
  • a tolerance range i. If a scope of application is provided across several hair color classes, the scope of application is also stored in the reference table. In this case, it is advantageously provided that the target hair color does not need to be determined and stored for each hair color of origin in the field of application, but it is sufficient to determine the corresponding target hair color only for the primary hair color of origin to save.
  • the other source hair colors of the application can be changed by simply "jumping" between the hair color class levels, i. by appropriately replacing the luminance values of the source hair colors.
  • hair coloring with a hair coloring agent
  • hair lightening with a hair lightening agent
  • the original hair color coding information associated with the original hair color is related to the target hair color coding information associated with the target hair color transformed, wherein the at least one luminance value remains substantially unchanged.
  • this procedure corresponds to a transformation of the original hair color coding information to an intermediate value, namely to the first intermediate hair color coding information.
  • the shift then takes place between the hair color class levels, i. replacing the L value of the first intermediate hair color coding information with the corresponding L value of the second intermediate hair color coding information, which is preferably stored in the reference table.
  • the first intermediate hair color coding information and the second intermediate hair color coding information both have the same predetermined saturation value in the HLS color space.
  • the first intermediate hair color coding information and the second intermediate hair color coding information both have the predetermined saturation value zero in the HLS color space.
  • Fig.2 shows a sketch of an HLS color space 200 (H (Hue): color angle in the color wheel, L (luminance): brightness of the color, S: saturation of the color) in a coordinate system.
  • H Hue
  • L luminance
  • S saturation of the color
  • the value ranges of the HLS color space 200 are as follows:
  • the luminance of a respective pixel describes the "brightness" of a color and is plotted along a luminance axis 202 in the HLS color space 200.
  • the luminance values are normalized according to the HLS color space 200 and the HLS color model, respectively, to a value range between the value 0 and the value 1 ( 0 ⁇ L ⁇ 1).
  • the saturation S plotted in the HLS color space 200 along a saturation axis 203 indicates the saturation of the respective color.
  • the saturation S is plotted in a respective plane given by the luminance.
  • the saturation values are normalized in the HLS color space 200 and the HLS color model to a value range between the value 0 and the value 1 (0 ⁇ S ⁇ 1).
  • a color in the HLS color space 200 is defined by the triple defined by the three variables described above, and thus represents a vector in the HLS color space 200 within the HLS color space 200.
  • a digital image is recorded by means of a camera and supplied to a computer for digital image processing.
  • the camera can be an analog camera or a digital camera.
  • an analog camera it is provided to convert the analog signals by means of an analog / digital converter into a digital signal, for example by means of sampling, and to further process the digital signal generated thereby in the context of digital image processing.
  • the digital image 300 includes, for example, as in FIG Figure 3 illustrated, at least one hair loop, generally a hair region 301, ie a region in which elements are contained in a hair color provided.
  • the digital image to be processed 300 has pixels in the hair area 301 to which encoding information coded according to the HLS color model described above is assigned.
  • the hair area 301 in an alternative embodiment of the invention is subdivided into sub-hair areas, for example the main hair area 302 (hereinafter also referred to as "normal" hair area), a shadow hair area 303 and a gloss hair area 304 (hereinafter also referred to as highlight hair area).
  • the coding information after recording by means of the digital camera or by means of the analog camera and in the case of an analog camera after the analog / digital conversion is coded according to the RGB color space.
  • the color information coded according to the RGB color space, is converted, i. transformed into the HLS color space 200, for example according to the transformation described in [2].
  • Figure 4 shows a plurality of digital images 401, 402, 403, 404, wherein in each digital image 401, 402, 403, 404 in a respective hair area 405, 406, 407, 408 hair in different natural hair color and thus provided with different coding information which the pixels in the hair region 405, 406, 407, 408 are included.
  • the arithmetic average of the individual components of the HLS vectors (HLS-triples) for each component of the HLS vector (ie for the color angle value, the luminance value and the saturation value) over all pixels of the respective hair region 405, 406, 407, 408.
  • the resulting arithmetic average gives the position of the considered natural hair color of the respective hair region 405, 406, 407, 408 in the HLS color model, i. in the HLS color space 200.
  • the blond tones have nearly identical H values (color angle values). Furthermore, the H values of the brown tones are similar.
  • the L values indicate the different brightnesses.
  • the saturation S is substantially the same for all natural hair colors.
  • Fig.1 shown in a discretization of the hair colors described above in hair color classes, which each discrete, ie several along the L-axis 202 at a distance Hair color class levels 100, 101, 102, 103 arranged in the HLS color space 200 are assigned to one another, wherein the distance of two adjacent hair color class levels 100, 101, 102, 103 corresponds to the respective difference of their respective L values.
  • the hair color class levels 100, 101, 102, 103 of the hair color classes described above are substantially plane-parallel to the plane spanned by the color angle H 201 in the HLS color space 200. In other words, this means that each hair color class level 101, 101, 102 , 103 substantially perpendicular, ie is normal to the luminance axis 202 in the HLS color space 200.
  • a hair color class cylinder is formed with the color angle H 201 and the saturation S 203 as the base area, the height of the hair color class cylinder being determined by the respective brightness, i. the luminance value of the hair color class.
  • hair color class levels 100, 101, 102, 103 corresponding to the number of provided hair color classes, which are arranged at a corresponding distance along the luminance axis 202 in the HLS color space 200 from each other according to the different luminance values.
  • the HLS color space 200 is broken down into discrete levels with respect to hair color classes, with each Hair color class level 100, 101, 102, 103 is clearly associated with a hair color class of natural hair colors.
  • Fig.1 For simplifying the illustration of the invention, only four hair color class levels 100, 101, 102, 103 are shown, namely a first hair color class level 100 for the hair color class "Black”, a second hair color class level 101 for the hair color class "Dark Brown”, a third hair color class. Level 102 for hair color class "Medium brown” and a fourth hair color class level 103 for hair color class "Light brown”.
  • the other hair color class levels of the other hair color classes are also provided, but in Fig.1 not shown.
  • Brighteners are different from bleaches which, while lightening the hair, add color to brightened hair. Brighteners, on the other hand, only whiten the hair and thus achieve a lightening of the original hair color and in this way a hair color change in the sense of this description.
  • a brightener is often used as a preparation before actual hair coloring or hair tinting if the source hair is for the desired hue, i. for the use of the desired hair dye or hair tinting agent is too dark, in other words, if the original hair color to achieve a desired target hair color using a corresponding desired hair dye or hair tinting agent is too dark.
  • Brighteners are classified according to their brightening power.
  • a brightener which is intended for Change the brightening of the original hair color by one to two hair color shades, ie one to two
  • Hair color classes for example, the original hair color “middle brown” on the hair color “light brown” or a source hair color “dark blond” on the hair color “middle blond”, etc. brightened.
  • a hair color located in a hair color class plane is shifted by one or two hair color class planes along the luminance axis 202.
  • Opacity expressed as a percentage, describes the ability to use the hair colorant, the original hue, i. replace the original hair color with the target hair color of the hair colorant.
  • the margin i. the range in which the 100% replacement of the source hair color by the target hair color of the hair colorant is even possible is usually limited in human hair.
  • the individual hair color shades i. To assign specific original hair colors to the individual hair colors of a hair coloration series, i. a hair color for which the respective hair color shade, in other words the respective hair colorant, is provided or applicable.
  • a color result of a hair colorant according to a shade of a hair coloration series for the hair color class "dark blond” thus usually applies only to the hair color class "middle blond", the hair color class "Dark blonde” or the hair color class "light brown”, in other words for medium blond, dark blond, or light brown hair.
  • the source hair colors valid for each hair color nuance are always contiguous within the HLS color model, i. they are immediately adjacent to each other arranged hair color class levels in the HLS color space 200 and form the so-called application of the hair colorant.
  • hair color range such as medium blond, medium brown and black.
  • a hair colorant has an application tolerance range of one to two, usually a maximum of three hair color shades (expressed in hair color classes) for which the achievement of the target hair color can be achieved with the use of the respective hair colorant ,
  • the reason for this is usually in the preparation of the hair colorant, especially in the preparation of a hair dye or a hair tint, and the determination of the source hair colors.
  • a hair colorant is developed for a particular source of hair color.
  • the indicated color result ie the indicated hair colorant target hair color
  • it is usually checked on which original hair colors in the lighter and darker range of hair color classes the desired color result, ie the achievement of the target hair color while maintaining a low, predetermined tolerance in terms of color nor the desired color result of hair coloring.
  • the hair colors which satisfy this criterion within the given tolerance for a hair colorant in each case form the field of application of the hair colorant.
  • each pixel of the digitized image in the hair area 301 is assigned coding information according to the HLS color space 200.
  • a hair colouration provides the same color result for different hair colors of origin, i. if different hair colors originating form the scope of the respective Haar-Coloration
  • the coordinates of the original hair color triples in the formed hair color cylinder are approximately vertically superimposed on each adjacent discrete hair color class levels 100, 101, 102, 103. Die Indication of the respective field of application of the hair colorant is optionally also stored in the reference table.
  • the source hair color, the target hair color and the application range are determined, described and stored in a reference table in the memory of the computer for each considered hair colorant within the HLS color space 200, as is clearly shown in FIG Figure 5 in plan view 500 of a hair color class level n of the HLS color space 200 for a first hair colorant 501, a second hair colorant 502, a third hair colorant 503 and a fourth hair colorant 504 is described.
  • Figure 6 shows in a second plan view 600 for a fifth hair coloring agent 601, a sixth hair coloring agent 602, a seventh hair coloring agent 603, and an eighth hair coloring agent 604 the original hair colors in a hair color class level n + 1, which in the hair color cylinder compared with the one in Figure 5 represented hair color class level n by a discretization L value up along the L-axis 202 shifted hair color class level.
  • Figure 7 In a top view 700, the hair colorants achievable by means of the respective hair colorations using a brightener by shifting within the respective hair color class levels for the hair colorants in the respective application area, in other words a combination of the hair color class level n and the hair color class level n + 1 from the Figure 5 respectively.
  • Figure 6
  • the coding information corresponding to the pixels of the hair region 301 in the digital image 800 (see flowchart 800 in FIG Figure 8 ) , the entire color removed.
  • each individual HLS triple is analyzed and, as described above, for example, according to the method described in [2], converted into the HLS color space 200.
  • a digital image which, at least in the hair area, consists exclusively of gray values.
  • the mean i. the arithmetic average of all gray values of this digital image, preferably the arithmetic average of all gray values of the hair area, determines the brightness of the source hair color and thus the hair color class level in the color cylinder corresponding to the hair color present in the hair area 301 of the hair colorant digital image 300, and thus the hair color class level that is valid for the digital image.
  • the gray value image thus only contains contrasts and brightnesses and thus shows the structure of the original digital image 300.
  • the hair color class level resulting from the gray value deviates from the hair color class level for which the hair colorant was respectively developed as described above, and the hair color class level is in the application range of hair coloring, then the L value of the HLS triple of the respective coding information of the pixels of the hair region 301 is changed so far, ie increased or possibly decreased, that the changed average gray value of that hair color class level corresponds to each hair colorant has been developed (step 803).
  • the corresponding calculation also takes place at this point. According to the degree of lightening, the luminance values are increased so that the average gray value corresponds to the hair color class level resulting from the use of the brightener.
  • each HLS triple now representing an average gray value is assigned the respective H value, L value, S value of the hair coloration.
  • the HLS triples for displaying the digital image for a user on a screen in a conventional manner are converted back into the RGB color space, for example according to the method described in [2], since conventional display units for displaying colors the RGB Use color space.
  • a digital image ie, a color image, which reproduces the color result of a hair coloration with 100% opacity on a given original hair, ie, a digital image with hair of a given original hair color.
  • hair colorations with less hiding power than 100% may be considered.
  • a hair colouration with an opacity of less than 100% does not completely replace the original hair color, but lets the original hair color shine through.
  • the opacity is primarily considered as the degree of coverage of naturally grayed hair.
  • a hair colouration with 70% gray coverage means that, statistically speaking, out of 100 individual hairs, 70 will accept hair coloring and 30 hair will not.
  • the original hair color contributes 30% to the result after applying hair colouration with 70% opacity. It thus creates a mixed color of the color of hair coloring and the original hair color.
  • a hair colorant with less than 100% opacity is represented in the HLS Color Model 200 described above just like a hair colorant with 100% opacity.
  • the target hair color of a hair color class level of the field of application will take into account the original color values of the source hair color.
  • the same result colors in the HLS color space 200 are vertically stacked at different hair color class levels.
  • the colors on the vertical differ only in their L value.
  • the L value does not correspond to the L value assigned to the respective hair color class level, but results from the above-described ratio of the hiding power of the hair colorant to the respective original hair color.
  • a reference table is also formed in which the resulting hair colors or the color values determined are applied to the individual original hair colors for each hair colouration used.
  • the pixels i. the coding information associated with the pixels of a hair region 301 of the original digital image 901 is transformed from the RGB color space into the HLS color space in a manner known per se, for example according to the method described in [2], and the associated HLS values are stored ( Step 902).
  • the saturation value of the coding information of all the pixels of the hair region 301 is set to the value 0 (step 903), and the L value is appropriately increased, alternatively lowered, to reach the target hair color and, in particular, the target luminance value according to the hair coloring used (step 904).
  • the hair color class plane of the color cylinder is determined and the target hair color is selected. Depending on the L value of the target hair color, the L value of the gray image is adjusted.
  • the new HLS values for the respective coding information of the pixels of the hair region 301 are determined (step 905), according to this embodiment of the invention taking into account the percentage of opacity and thus the stored HLS values of the source hair color.
  • step 905 the newly calculated HLS values in step 905, and in particular the newly calculated H values, are added to the respective HLS triple (step 906), so that a colored image 907 is generated.
  • a first data array P 1001 is applied, wherein the first data array P 1001 is dependent on the dimensions of the image, i. the number of pixels along a longitudinal direction of the image (Y direction) and one of the number of pixels along the width direction of the image (X direction) and the number of color depths provided for color reproduction, i. depends on the color depth provided bits depends.
  • the length or the width of the image is thus the number of pixels in a digital image in the X-direction or the Y-direction, generally along two mutually perpendicular directions within the digital image.
  • the color depth is the number of possible values for the components red, green and blue. In the case of 24-bit color depth, 8 bits are used for each of the three colors. Each of the three colors can therefore assume 256 different states. The maximum number of colors within such a picture is 16.7 million.
  • the RGB color format is also called a true color color format.
  • the digital image is decomposed and the values of each pixel are stored in the first data array P 1001.
  • a second data array Q 1003 is applied, which is for recording, i. for storing the values of the coding information of the pixels in the HLS color space.
  • the dimensions of the second data array Q 1003 are the same as those of the first data array P 1001 as described above.
  • a data array R of dimension X * Y is applied for buffering the H-values as described above in connection with the calculation of target H-values for a hair colouration with an opacity of less than 100%.
  • 10A 1 shows the first data array P 1001 as well as the method step of the transformation of each pixel triple from the RGB color space, ie of each triple in the first data array P 1001 to an associated value in the second data array Q 1003, ie vividly the Transformation of the pixels from the RGB color space into the HLS color space 200 (step 1002).
  • the converted, i. the transformed coding information is stored in the second data array Q 1003.
  • the associated saturation values are set to the value 0.
  • the associated H values are stored in the additional data array R, i. saved. Subsequently, an arithmetic average of the luminance values of the second data array Q 1003, in particular the luminance values of the encoding information associated with the pixels of the hair region 301, is calculated.
  • Step 1004 After calculating the target hair colors in the HLS color space 200 and restoring the final target hair color coding information into the second data array Q 1003, the coding information stored in the second data array Q 1003 is transformed back into the RGB color space (Step 1004), as in 10B is shown.
  • the re-transformed values of the coding information are stored in the first data array P 1001.
  • the coding information of the pixels of the hair region which in turn is present in the RGB color space, is fed to the monitor, which is coupled to the computer and can represent data in accordance with the RGB color space, and is displayed by it to a user.
  • the correct values for color, luminance and saturation are taken from the reference table of the particular hair colouration used, which was previously stored in the computer.
  • RGB color space the values of each color can take on values between 0 and 255.
  • a value of 0 means black, a value of 255 means full color.
  • a number triplet RGB (255, 255, 255) represents white.
  • triples in the range from RGB (0, 0, 0) to RGB (65, 65, 65) represent shadow hair areas and triples with values in the range of RGB (200, 200, 200) represent the highlight hair areas. ie the glossy sub-hair areas of the hair area.
  • the above-described sub-hair areas are treated separately according to another embodiment of the invention because they show less the inherent color of the hair but in the highlight hair areas the color of the illuminating light source and in the shadows Hairballs are the color of the environment in which the image was taken.
  • the color change with the method described above is performed for the data triples with RGB color values RGB (66, 66, 66) to RGB (199, 199, 199).
  • Shadow hair areas i. the pixels with RGB color values from RGB (0, 0, 0) to RGB (65, 65, 65) as well as highlight hair areas, i. the ranges of RGB color values from RGB (200, 200, 200) to RGB (255, 255, 255) are each 50% with the target hair color, i. the new color, charged. This procedure is equivalent to hair colouration with less than 100% opacity.
  • Figure 11 11 shows a schematic illustration of the hair color change in a diagram 1100, wherein a first region 1101, ie the shadow hair region, the "normal hair region" as the second region 1102 and the highlight hair region as the third region 1103 are shown with the corresponding different weighting of the hair color change respective RGB values in the context of the color transformation as described above.
  • the system for displaying the effect of a hair colouration on a portrait photograph of a person for example on the portrait photo of the user, can be realized.
  • a digital photo is processed with a suitable computer program, as described above, in such a way that the hair area 301 can be changed digitally; all other areas, for example face, body and background, remain unchangeable.
  • the hair area is, as described above, decolored, and provided with the color of a hair colouration.
  • the color values of a manufacturer's hair coloring can ideally be stored in a database. So they are quickly available.
  • Each manufacturer of a hair colouration provides its customers so-called hair color cards.
  • the hair color maps clearly depict a matrix on whose axes, on the one hand, the original hair colors or the natural hair colors and, on the other hand, the hair colorations of a coloration series, ie the individual colors of the products of a coloration series.
  • the cells of the matrix hair loops, or similar hair pictures are listed, as in Figure 3 shown showing the resulting color result.
  • the templates for these hair images can be quickly created by means of a database with the stored values and a template of the hair loop to be imaged, ie a reference hair loop for all the colors of a series of hair colorations. With an average range of four original hair colors and about 200 products, 800 images can be generated automatically.

Landscapes

  • Health & Medical Sciences (AREA)
  • Dermatology (AREA)
  • General Health & Medical Sciences (AREA)
  • Image Processing (AREA)
  • Cosmetics (AREA)
  • Color Image Communication Systems (AREA)
  • Image Analysis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Claims (9)

  1. Procédé pour la détermination assistée par ordinateur d'une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numérique suite à une modification de la couleur de cheveux, des couleurs de cheveux étant attribuées à une multitude de classes de couleurs de cheveux prédéfinies,
    • dans lequel la zone capillaire dans l'image numérique présente des pixels auxquels est attribuée une information de codage codée conformément à l'espace chromatique HLS, respectivement au moins une valeur de luminosité prédéfinie conformément à la classe de couleurs de cheveux étant respectivement attribuée à chaque pixel dans la zone capillaire d'une classe de couleurs de cheveux dans l'espace chromatique HLS,
    • dans lequel on évalue le fait de savoir si la couleur de cheveux d'origine et la couleur de cheveux cible sont attribuées à la même classe de couleurs de cheveux,
    • dans lequel, pour le cas où la couleur de cheveux d'origine et la couleur de cheveux cible sont attribuées à des classes de couleurs de cheveux différentes, l'information de codage de la couleur de cheveux d'origine attribuée à la couleur de cheveux d'origine est transformée dans l'espace chromatique HLS en l'information de codage de la couleur de cheveux cible attribuée à la couleur de cheveux cible, ladite au moins une valeur de luminosité de la classe de couleurs de cheveux de la couleur de cheveux d'origine étant remplacée par ladite au moins une valeur de luminosité de la classe de couleurs de cheveux de la couleur de cheveux cible.
  2. Procédé selon la revendication 1,
    • dans lequel, pour au moins une modification de la couleur de cheveux, une information de codage de la couleur de cheveux d'origine de référence et au moins une information de codage de la couleur de cheveux cible de référence, que l'on obtient par l'application de la modification de la couleur de cheveux sur des cheveux de la couleur de cheveux d'origine de référence sont mémorisées, l'information de codage de la couleur de cheveux d'origine de référence et l'information de codage de la couleur de cheveux cible de référence étant codées conformément à l'espace chromatique HLS,
    • dans lequel l'information de codage de la couleur de cheveux d'origine est transformée en l'information de codage de la couleur de cheveux cible dans l'espace chromatique HLS en utilisant l'information de codage de la couleur de cheveux d'origine de référence correspondant à l'information de codage de la couleur de cheveux d'origine et l'information de codage de la couleur de cheveux cible de référence correspondante comme information de codage de la couleur de cheveux cible.
  3. Procédé selon la revendication 1 ou 2,
    dans lequel, à titre de modification de la couleur de cheveux, on met en oeuvre une coloration des cheveux et/ou un éclaircissage des cheveux.
  4. Procédé selon l'une quelconque des revendications 1 à 3,
    dans lequel, pour le cas où la couleur de cheveux d'origine et la couleur de cheveux cible sont attribuées à la même classe de couleurs de cheveux, l'information de codage de la couleur de cheveux d'origine attribuée à la couleur de cheveux d'origine est transformée en l'information de codage de la couleur de cheveux cible attribuée à la couleur de cheveux cible, ladite au moins une valeur de luminosité restant essentiellement inchangée.
  5. Procédé selon l'une quelconque des revendications 1 à 4,
    dans lequel les couleurs des cheveux sont attribuées au moins à une partie des classes de couleurs de cheveux suivantes :
    • blond extra clair ;
    • blond clair ;
    • blond moyen ;
    • blond foncé
    • brun clair ;
    • brun moyen ;
    • brun foncé ;
    • brun noir ;
    • noir.
  6. Procédé selon la revendication 5,
    • dans lequel la transformation de l'information de codage de la couleur de cheveux d'origine en l'information de codage de la couleur de cheveux cible attribuée à la couleur de cheveux cible dans l'espace chromatique HLS présente les étapes suivantes :
    • l'information de codage de la couleur de cheveux d'origine est transformée sur base d'une première information de codage de la couleur de cheveux intermédiaire dans l'espace chromatique HLS, ladite au moins une valeur de luminosité restant inchangée ;
    • ladite au moins une valeur de luminosité de la classe de couleurs de cheveux de la couleur de cheveux d'origine est remplacée par la valeur de luminosité de la classe de couleurs de cheveux de la couleur de cheveux cible, de sorte que l'on génère une deuxième information de codage de la couleur de cheveux intermédiaire dans l'espace chromatique HLS ;
    • la deuxième information de codage de la couleur de cheveux intermédiaire est transformée en l'information de codage de la couleur de cheveux cible dans l'espace chromatique HLS.
  7. Procédé selon la revendication 6,
    dans lequel à la fois la première information de codage de la couleur de cheveux intermédiaire et la deuxième information de codage de la couleur de cheveux intermédiaire présentent la même valeur de saturation prédéfinie dans l'espace chromatique HLS.
  8. Procédé selon la revendication 7,
    dans lequel à la fois la première information de codage de la couleur de cheveux intermédiaire et la deuxième information de codage de la couleur de cheveux intermédiaire présentent la valeur de saturation prédéfinie nulle dans l'espace chromatique HLS.
  9. Dispositif pour la détermination d'une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numérique suite à une modification de la couleur de cheveux, des couleurs de cheveux étant attribuées à une multitude de classes de couleurs de cheveux prédéfinies, la zone capillaire dans l'image numérique présentant des pixels auxquels sont attribuées des informations de codage codées conformément à l'espace chromatique HLS, respectivement au moins une valeur de luminosité prédéfinie conformément à la classe de couleurs de cheveux étant attribuée à chaque pixel dans la zone capillaire d'une classe de couleurs de cheveux dans l'espace chromatique HLS,
    comprenant une unité de processeur qui est conçue de telle sorte que sont effectuées les étapes de procédés suivantes :
    • on évalue le fait de savoir si la couleur de cheveux d'origine et la couleur de cheveux cible sont attribuées à la même classe de couleurs de cheveux,
    • pour le cas où la couleur de cheveux d'origine et la couleur de cheveux cible sont attribuées à des classes de couleurs de cheveux différentes, l'information de codage de la couleur de cheveux d'origine attribuée à la couleur de cheveux d'origine est transformée dans l'espace chromatique HLS en l'information de codage de la couleur de cheveux cible attribuée à la couleur de cheveux cible, ladite au moins une valeur de luminosité de la classe de couleurs de cheveux de la couleur de cheveux d'origine étant remplacée par ladite au moins une valeur de luminosité de la classe de couleurs de cheveux de la couleur de cheveux cible.
EP05706768A 2004-02-19 2005-02-18 Procede et dispositif pour determiner par ordinateur une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numerique apres modification de la couleur de cheveux Active EP1716542B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102004008223A DE102004008223A1 (de) 2004-02-19 2004-02-19 Verfahren und Vorrichtung zum Ermitteln einer Ziel-Haarfarbe für eine Ursprungs-Haarfarbe eines Haarbereichs in einem digitalen Bild nach erfolgter Haarfarbveränderung
PCT/DE2005/000282 WO2005081190A1 (fr) 2004-02-19 2005-02-18 Procede et dispositif pour determiner par ordinateur une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numerique apres modification de la couleur de cheveux

Publications (2)

Publication Number Publication Date
EP1716542A1 EP1716542A1 (fr) 2006-11-02
EP1716542B1 true EP1716542B1 (fr) 2010-08-11

Family

ID=34853543

Family Applications (1)

Application Number Title Priority Date Filing Date
EP05706768A Active EP1716542B1 (fr) 2004-02-19 2005-02-18 Procede et dispositif pour determiner par ordinateur une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numerique apres modification de la couleur de cheveux

Country Status (4)

Country Link
EP (1) EP1716542B1 (fr)
AT (1) ATE477560T1 (fr)
DE (2) DE102004008223A1 (fr)
WO (1) WO2005081190A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3671556A1 (fr) * 2018-12-18 2020-06-24 Henkel AG & Co. KGaA Procédé de détermination d'un niveau de gris d'un brin de cheveux

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9619119D0 (en) * 1996-09-12 1996-10-23 Discreet Logic Inc Processing image
US6259428B1 (en) * 1998-05-01 2001-07-10 National Instruments Corporation System and method for dynamically substituting colors in a graphic image
US6719565B1 (en) * 1999-10-29 2004-04-13 Kao Corporation Hair color advice system
KR100406508B1 (ko) * 2000-06-23 2003-11-20 주식회사 코난테크놀로지 컬러영상 처리 시스템 및 그 방법
US7039229B2 (en) * 2000-08-14 2006-05-02 National Instruments Corporation Locating regions in a target image using color match, luminance pattern match and hill-climbing techniques
DE10103971A1 (de) * 2001-01-30 2002-08-14 Henkel Kgaa Verfahren und Vorrichtung zum Bearbeiten eines in einem ersten Computer gespeicherten digitalisierten Bildes, Computerlesbares Speichermedium und Computerprogramm-Element
DE10142526C5 (de) * 2001-08-30 2006-02-16 Wella Ag Verfahren für eine Haarfarbberatung

Also Published As

Publication number Publication date
DE502005010068D1 (de) 2010-09-23
EP1716542A1 (fr) 2006-11-02
ATE477560T1 (de) 2010-08-15
WO2005081190A1 (fr) 2005-09-01
DE102004008223A1 (de) 2005-09-15

Similar Documents

Publication Publication Date Title
DE69301239T2 (de) Verfahren zur Bestimmung der Farbe einer Teintgrundierung zur sensitiven Wiederherstellung der Hautfarbe einer Person und Vorrichtung zu dessen Anwendung
DE69633262T2 (de) Farbumwandlungsverfahren und Vorrichtung zum Herstellen von Farbumwandlungstabellen
DE60101013T2 (de) Farbtransformationsverfahren zum Abbilden von Farben in Bildern
EP3201826B1 (fr) Procédé et dispositif de traitement de données pour le conseil, assisté par ordinateur, de couleur de cheveux
DE69522594T2 (de) Farbbildverarbeitungsverfahren
DE602005006251T2 (de) Farbsimuliersystem zum haarfärben
DE69719164T2 (de) Verfahren und Gerät zur Änderung von Bildern in Rechnersystemen
DE69328805T2 (de) Farbbilderzeugungsverfahren unter Benutzung eines Farb-plus-Grauton Farbmodels und Fehlerdiffusion
DE69120748T2 (de) Umsetzung von Farbbildern auf schwarz-weiss texturierten Bildern
DE69929238T2 (de) Verfahren zur aufnahme und wiedergabe stereoskopischer farbbilder mit polychromatischen filtern
DE19528596C2 (de) Verfahren und Vorrichtung zur Kolorierunterstützung
DE69128548T2 (de) Verbesserungen in Steuerdatenfeldern von Bildelementen
DE69216207T2 (de) Farbeinstellung zur Glättung der Grenzen zwischen Farbbildern
DE69424451T2 (de) Halbtonzitteroptimierungstechniken
DE69532025T2 (de) Farbbildverarbeitungsverfahren und -Vorrichtung
DE10053301A1 (de) Verfahren zur farblichen Wiedergabe eines Graustufenbildes
DE10017551C2 (de) Verfahren zur zyklischen, interaktiven Bildanalyse sowie Computersystem und Computerprogramm zur Ausführung des Verfahrens
WO2019034664A1 (fr) Procédé et dispositif de conseil en traitement des cheveux assisté par ordinateur
DE60100537T2 (de) Verfahren und anordnung zur wiederherstellung eines lichtsignals
DE102020105697B3 (de) Computerimplementiertes Verfahren zum Auffinden möglicher Artefakte in einem virtuell eingefärbten Histologiebild
EP1105843B1 (fr) Procede et dispositif de detection des couleurs d'un objet
EP1716542B1 (fr) Procede et dispositif pour determiner par ordinateur une couleur de cheveux cible pour une couleur de cheveux d'origine d'une zone capillaire dans une image numerique apres modification de la couleur de cheveux
DE60130614T2 (de) Farbe zu Schwarzweiss Konverter
DE69830583T2 (de) Verfahren und Vorrichtung zur Verarbeitung von Bilddaten
EP1090374B1 (fr) Procede pour traiter des donnees d'image

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20060202

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

RIN1 Information on inventor provided before grant (corrected)

Inventor name: FRAUNE, GERRIT

Inventor name: HEINEMANN, MARKUS

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: HENKEL AG & CO. KGAA

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIN1 Information on inventor provided before grant (corrected)

Inventor name: HEINEMANN, MARKUS

Inventor name: FRAUNE, GERRIT

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REF Corresponds to:

Ref document number: 502005010068

Country of ref document: DE

Date of ref document: 20100923

Kind code of ref document: P

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20100811

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20100811

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101213

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101111

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101211

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101112

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101122

26N No opposition filed

Effective date: 20110512

BERE Be: lapsed

Owner name: HENKEL A.G. & CO. KGAA

Effective date: 20110228

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502005010068

Country of ref document: DE

Effective date: 20110512

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110218

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100811

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230530

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: AT

Payment date: 20240220

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: MC

Payment date: 20240226

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240219

Year of fee payment: 20

Ref country code: GB

Payment date: 20240219

Year of fee payment: 20

Ref country code: CH

Payment date: 20240301

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240220

Year of fee payment: 20