Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/40—Extraction of image or video features
  • G06V10/56—Extraction of image or video features relating to colour

Definitions

• the invention relates to generating a color profile, and more particularly relates to generating a color profile for a digital input device, wherein the color profile can be used to transform color values of an image.
• CMSs Traditional color management systems
• CMSs use color profiles of digital input devices to derive color transformations that transform color coordinates between a device-dependent color space that depends on the device, and a device-independent color space.
• a color profile for a digital input device such as a camera or scanner can be generated by first imaging a color target (or "color checker") consisting of pre-designated color patches.
• the color target is imaged under controlled lighting conditions, which can be achieved by placing the color target inside a light booth.
• this imaging typically results in an RGB bitmap image in which the color of each color patch is encoded in an RGB value.
• a color profile for a digital input device is generated. Color values for at least one color target positioned within a first scene are measured, the color target having multiple color patches. An image of the first scene is generated using the digital input device, the first scene including the color target(s). Color values from a portion of the image corresponding to the color target are extracted and a color profile is generated, based on the measured color values and the extracted color values. The generated color profile is used to transform the color values of an image of a second scene. Using this generated color profile to transform images is likely to result in more colorimetrically accurate transformations of images created under real -world lighting conditions.
• the digital input device can comprise a digital camera.
• the color profile can include an International Color Consortium (ICC) color profile.
• the at least one color target can comprise a plurality of color targets at different positions within the first scene.
• the first scene can comprise a plurality of lighting areas.
• the plurality of color targets can be positioned at different ones of the plurality of lighting areas.
• Each of the plurality of lighting areas can have a different illuminant value and degree of brightness.
• a color value of at least one reference color patch (per target) can be measured for the at least one color target, and color values for the remaining color patches can be predicted, based on the measured color value.
• the color values for the remaining color patches can be predicted by applying a Chromatic Adaptation Transformation (CAT) to pre-measured color values for the at least one color target measured under controlled lighting conditions.
• CAT Chromatic Adaptation Transformation
• the CAT can be used to transform colorimetric values between different illuminants.
• the CAT can include CAT97, CAT2000, and XYZ scaling.
• the reference color patch can be the brightest white patch of the at least one color target.
• the extracting step can comprise selecting the portion of the image corresponding to the at least one color target, locating each color patch in the selected portion of the image using a bi-linear interpolation process, sampling the pixels within a pixel area of each color patch, and determining a color patch value for each color patch based on the sampled pixels.
• a transformation can be created from the color profile, and the transformation can be used to transform the color values of the image of the second scene.
• the transformation can be a third order polynomial transformation.
• a color profile for a digital input device is generated using a computer program.
• a first image corresponding to a color target is selected, the color target having multiple color patches.
• Each color patch in the selected portion of the image is located using a bi-linear interpolation process.
• the pixels within a pixel area of each color patch are sampled and a color patch value for each color patch is determined based on the sampled pixels.
• a color profile is generated based on the determined color patch values and the color profile is used to create a transformation that will transform the color values of a second image.
• Using the computer program to determine values of color patches in an image corresponding to a color target is likely to simplify the process of generating a color profile.
• the selected portion of the first image can be enlarged when it is selected.
• the multiple color patches can comprise four corner patches of the color target, and selecting the first image can comprise selecting four corner patches of the color target, specifying the number of color patches in the color target, determining the pixel area for each color patch, wherein the bi-linear interpolation process can use coordinates of the selected four corner patches and the specified number of color patches to locate remaining color patches of the color target.
• the number of color patches in the color target can be specified as a number of rows of color patches and a number of columns of color patches.
• the computer program can display the pixel area of each color patch and the pixel area of each color patch can be verified.
• the pixel area of each color patch can be determined based on a pixel radius and a center of the color patch.
• FIG. 1 is a flowchart depicting a process for generating a color profile for a digital input device in accordance with a representative embodiment of the invention.
• FIG. 2 is a pictorial view of a scene in accordance with a representative embodiment of the invention.
• FIG. 3 is a flowchart depicting a process for using a Chromatic Adaptation Transformation (CAT) to predict values for color patches in accordance with a representative embodiment of the invention.
• CAT Chromatic Adaptation Transformation
• Fig. 4 is a flowchart depicting a process for extracting color values from an image in accordance with a representative embodiment of the invention.
• FIG. 5 depicts an example of a user interface for selecting an image in accordance with a representative embodiment of the invention.
• FIG. 6 depicts an example of a user interface for inputting a pixel radius in accordance with a representative embodiment of the invention.
• FIG. 7 depicts an example of a user interface for displaying pixel areas in accordance with a representative embodiment of the invention.
• FIG. 8 depicts an example of a user interface for verifying pixel areas in accordance with a representative embodiment of the invention.
• Fig. 9 is a diagram illustrating a bi-linear interpolation method for determining the centers of color patches of a color target in accordance with a representative embodiment of the invention.
• Fig. 10 is an architecture diagram for a data processing apparatus, such as a general purpose computing machine, suitable for performing a process for generating a color profile in accordance with a representative embodiment of the invention.
• FIG. 1 is a flowchart depicting a process for generating a color profile for a digital input device in accordance with a representative embodiment of the invention.
• each color target e.g., color checker
• a colorimeter or spectrophotometer to obtain a set of tristimulus values (i.e., values corresponding to a device independent color space, such as, for example, the XYZ color space).
• a scene is staged to include two or more color targets, positioned in different lighting areas of the scene.
• color target 201 is placed in a first lighting area (e.g., direct sunlight) and color target 202 is placed in a second lighting area (e.g., shade).
• first lighting area e.g., direct sunlight
• second lighting area e.g., shade
• the color value (i.e., tristimulus value) for a reference patch on each color target is measured while the color target is positioned in the scene (e.g., 200), using, for example, a colorimeter or spectrophotometer.
• the reference patch can be, for example, the brightest white patch on the color target.
• the tristimulus values (e.g., XYZ) for the remaining patches on each color checker are predicted using these in-scene reference patch values and the patch values pre-measured at block 100, as described below for Fig. 3.
• a white reference standard is measured to establish a white point.
• the white point is used to create a color transformation as described below.
• the white reference standard can be, for example, a halon disk placed in the brightest lighting area, the brightest white patch of the color target in the brightest lighting area (e.g., 201), or any other suitable white reference standard.
• an image of the scene (e.g., 200) is generated using a digital input device, such as, for example, a digital camera, and at block 105 device color values (i.e., color values generated when the digital input device renders the image, such as, for example, RGB color values) are extracted from the portions of the image corresponding to each color target in the scene (e.g., 201 and 202), as will be described below for Fig. 4.
• a digital input device such as, for example, a digital camera
• device color values i.e., color values generated when the digital input device renders the image, such as, for example, RGB color values
• the XYZ values (measured and predicted at block 102) and corresponding RGB values (extracted at block 105) for each color target (e.g., 201 and 202) are combined to form a color profile (e.g., an ICC profile), which can be stored, for example, on a memory (e.g., 1002 of Fig. 10) or on a storage device (e.g. 1006 of Fig. 10). Thereafter, processing proceeds to block 107 and ends.
• a color profile e.g., an ICC profile
• This generated color profile is used to create a color transformation process that can take an arbitrary color value in the digital input device's color space (e.g., RGB) as input, and yield the device independent color value (e.g., XYZ) that would be predicted for a second scene, captured under lighting conditions similar to the lighting conditions of the profiled scene (e.g., 200).
• RGB digital input device's color space
• XYZ device independent color value
• A is the set of device dependent color values of the color profile (e.g., RGB values)
• B is the corresponding set of set of device independent values of the color profile (e.g., XYZ values)
• X is the transformation that converts color values from a device dependent color space to a device independent color space.
• X can be a linear transformation or a polynomial transformation of any order. In an example embodiment of the invention, X is a third-order polynomial transformation.
• B can be converted to a set of color values using a color space where an accepted colorimetric measurement standard can be used.
• B can be converted to a set of CIE L*a*b* (Lab) values (using the white point obtained at block 103) so that, for example, ⁇ E94 can be used as the measurement standard used to solve for X.
• An iterative minimum-search solving method such as, for example, the simplex algorithm, is used to solve for X.
• the objective function used in the solving method is seen to minimize the mean color difference (e.g., ⁇ E94 color difference) between B and the values generated by converting A using X.
• color values of an image of a second scene can be transformed from the digital input device's color space to a device independent color space.
• the color profile used to create this transformation has color values corresponding to different lighting areas, the color profile typically covers a wide luminance range, thereby resulting in more colorimetrically accurate transformations of images created under real -world lighting conditions.
• Fig. 3 is a flowchart depicting a process for using a Chromatic Adaptation Transformation (CAT) to predict tristimulus values for the color patches not necessarily measured at block 102 of Fig. 1.
• a CAT predicts what a certain color value under a particular lighting condition (illuminant) will be changed to if illuminated under a different lighting condition.
• Example CATs include XYZ scaling, CAT97, CAT2000, or any other CAT that can transform colorimetric values between different illuminants.
• An example embodiment where a CAT uses XYZ scaling to predict tristumulus values will be described, but in other embodiments other CATs may be used.
• pre represents pre-measured values of the reference patch of a color target
• scene represents the measured (in-scene) value of the reference patch of a color target
• scale represents the resultant scale factor that can be applied to the remaining pre-measured XYZ values to generate in-scene XYZ values for the color patches that have not been measured in the scene.
• the XYZ scale factor is used to generate predicted in-scene XYZ values for the color patches that have not been measured in the scene (at block 102).
• a predicted in-scene XYZ value is generated by multiplying each X, Y and Z component of a patch value pre-measured at block 101 by the respective X, Y and Z scale factors. Thereafter, processing proceeds to block 303 and ends.
• Fig. 4 is a flowchart depicting a process for extracting color values from an image in accordance with a representative embodiment of the invention.
• a data processing apparatus e.g., 1000 of Fig. 10
• the image of the scene generated at block 104 is displayed on a user output device (e.g., 1018 of Fig. 10) of the data processing apparatus, such as, for example, a display screen or a monitor.
• the data processing apparatus displays a user interface (using the user output device, e.g., 1018) that prompts a user to input the number color targets in the displayed image using a user input device (e.g., 1014 of Fig. 10).
• a user input device e.g., 1014 of Fig. 10
• the data processing apparatus displays a user interface prompting the user to select a first color target (block 402).
• the user can select the color target by, for example, dragging a bounding box (e.g., 501 of Fig. 5) around the portion of the image corresponding to the color target.
• the data processing apparatus can zoom in on (i.e., enlarge) the selected color target.
• the data processing apparatus displays a user interface prompting the user to select the center of the four corner patches of the selected color target, starting with, for example, the top-left patch, and proceeding, for example, clockwise, to the other corners.
• the data processing apparatus displays a user interface prompting the user to specify the number of color patches in the selected color target either by inputting the number of rows and columns of the color target, or by selecting a standard color target from a list of color targets, for which the number of rows and columns of color patches is known. If a standard color target is selected, the data processing apparatus can determine the total number of rows and columns of color patches by searching a look-up table (stored, for example, as data 1010 of Fig. 10) specifying the number of rows and columns for each standard color target.
• a look-up table stored, for example, as data 1010 of Fig.
• the data processing apparatus determines a pixel area for each color patch.
• the data processing apparatus can determine the pixel areas by, for example, displaying a user interface prompting the user to specify how many pixels of the image to sample for each patch.
• the user interface e.g., Fig. 6
• the user interface can prompt the user to input a pixel radius (i.e., a distance, in units of pixels, from the center of a patch) that is used to determine the area around the center of a color patch from which to sample pixels.
• the area around the center of the color patch can be, for example, a circle (as determined by the pixel radius), a square circumscribed around a circle determined by the pixel radius, or any other suitable shape.
• the data processing apparatus can determine a pixel area by, for example, testing pixels surrounding a center pixel of a color patch until patch borders are found. For example, the data processing apparatus can sample pixels surrounding the center pixel of the patch at a predetermined pixel radius and compare the color values of the pixels surrounding the center pixel with the color value of the center pixel. If the difference between the color values of the surrounding pixels and the center pixel is within a threshold, the pixel radius is increased and this process is continued until the difference in color values exceeds the threshold. The resulting pixel radius can be used to determine the area around the center of the color patch from which to sample pixels.
• the data processing apparatus locates each color patch in the portion of the displayed image corresponding to the selected color target.
• a bi-linear interpolation method (in 2D) is preferably used to determine the centers of the color patches based on the coordinates of the centers of the corner patches selected at block 403. Using the bi-linear interpolation method, rotation or skew of the color target position in the image is compensated, as will be described below for Fig. 9.
• the data processing apparatus samples the pixels of each patch within the pixel area, and at block 408 the data processing apparatus determines an average pixel color value for each patch.
• the data processing apparatus displays the pixel area of each color patch on the user output device (e.g., 1018).
• the pixel areas can be displayed, for example, by superimposing pixel area borders onto the displayed image of the selected color target, as shown in Fig. 7.
• the data processing apparatus displays a user interface (e.g., Fig. 8) prompting the user to verify the pixel areas displayed for each color patch. If the user does not approve the displayed pixel areas by, for example, selecting button
• Fig. 9 is a diagram illustrating a bi-linear interpolation method for determining the centers of color patches of a color target in accordance with a representative embodiment of the invention.
• Color target 900 is the color target selected at block 402 of Fig. 4.
• the patches on the color target are arranged in equidistant rows and columns, such that the center points of the patches are co-linear within each row or column.
• TL, TR, BL, and BR represent the centers of the top left, top right, bottom left, and bottom right corner patches selected at block 403 of Fig. 4.
• a linear interpolation is performed on the top row of the color target to find the center points of all of the patches in the top row, using the top-left (X TL ,Y TL ) and top-right (XT R ,Y TR ) patch center points, and the number of patch columns (numcols) in the color target (determined at block 404).
• the same technique is also performed on the bottom row of patches, using the bottom-left (X BL ,Y BL ) and bottom-right (XBR,YBR) patch center points.
• the linear interpolation is performed by determining a distance vector between patch centers in the top row, and a distance vector between patch centers in the bottom row. These distance vectors are calculated using the following equations:
• YDVBOT (YBR - YBL) / (numcols - 1) Equation (6)
• Equations 3 and 4 provide the X and Y components, respectively, of the distance vector for the top row
• Equations 5 and 6 provide the X and Y components, respectively, of the distance vector for the bottom row.
• TOPCOL(N) and BOTCOL(N), respectively The center point of patches in the Nth column of the top and bottom rows (TOPCOL(N) and BOTCOL(N), respectively) can be determined using the following equations.
• BOTCOL(N) BL + ((N-I) x DVBOT) Equation (8)
• the center point of the fifth patch in the top row i.e., A
• the center point of the fifth patch in the bottom row i.e., B
• a distance vector between patch centers in column(N) is calculated from the center of the patch in the top row (XTOPCOL(N), YTOPCOL(N)) and the center of the patch in the bottom row (XBOTCOL(N), YBOTCOL(N)) using the following equations:
• XDVCOL(N) (XBOTCOL(N) - XTOPCOL(N)) / ⁇ numrows - 1) Equation (9)
• YDVCOUN (YBOTCOUN) - YTOPCOL(N)) / (numrows - 1) Equation (10)
• Equations 9 and 10 provide the X and Y components, respectively, of the distance vector between the patch centers of column(N). Numrows is the number of rows of patches in the color target determined at block 404.
• ROW(M)COL(N) The center point of a patch in row(M) of column(N) (i.e., ROW(M)COL(N)) can be determined using the following equation:
• ROW(M)COL(N) TOPCOL(N) + ((M-I) x DVCOL(N)) Equation (H)
• the centers of color patches of a color target can be determined based on the centers of the top left, top right, bottom left, and bottom right corner patches selected at block 403 of Fig. 4, and the number of patch rows and columns in the color target (determined at block 404).
• the data processing apparatus 1000 includes a processor 1001 coupled to a memory 1002 via system bus 1004.
• the processor is also coupled to external Input/Output (I/O) devices via the system bus and an I/O bus 1005.
• I/O Input/Output
• a storage device having computer system readable media 1006 is coupled to the processor via a storage device controller 1008 and the I/O bus and the system bus.
• the storage device is used by the processor to store and read data 1010 and program instructions 1012 used to implement the features of generating a color profile as described above.
• the processor may be further coupled to a user output device 1018 via a user output device controller 1020 coupled to the I/O bus.
• the processor uses the user output device to display a user interface to a user to prompt the user of selections of parameters used in the color profile generation process.
• the processor may be further coupled to a user input device 1009 via a user input device controller 1016 coupled to the I/O bus.
• the processor uses the user input device to receive selections of parameters used in the color profile generation process.
• the processor may be further coupled to a communications device 1022 via a communications device controller 1024 coupled to the I/O bus.
• the processor may use the communications device to communicate with another device for transferring a color profile and/or an image.
• the processor loads the program instructions from the storage device into the memory.
• the processor executes the loaded program instructions to implement the features of the color profile generation process as described above.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Facsimile Image Signal Circuits (AREA)
• Image Processing (AREA)
• Color Image Communication Systems (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=40454492

Family Applications (1)

Country Status (3)

Cited By (1)

Families Citing this family (4)

Citations (7)

Family Cites Families (15)

• 2007
  • 2007-09-18 US US11/901,891 patent/US8019153B2/en not_active Expired - Fee Related
• 2008
  • 2008-09-17 WO PCT/US2008/076724 patent/WO2009039209A1/en not_active Ceased
  • 2008-09-17 JP JP2010525112A patent/JP5006970B2/ja not_active Expired - Fee Related

Patent Citations (7)

Also Published As

Similar Documents

Legal Events