Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T11/00—2D [Two Dimensional] image generation
  • G06T11/001—Texturing; Colouring; Generation of texture or colour

Definitions

• the present invention relates to combining images with imaging- related products to provide an improved aesthetic combination.
• Products that include images are a popular keepsake or gift for many people. Such products typically include a picture taken by an individual that is inserted into a product intended to enhance the presentation of the image or to provide storage for the image. Examples of such products include picture albums, posters, calendars, mugs, ornaments, mouse pads and post cards.
• Images can be enhanced in several ways, for example by image processing to modify the image content and thereby improve the perceived quality or value of the images. Images can also be combined with other images, for example templates including background images and one or more image openings into which an individual's image can be inserted, either mechanically or electronically with a computer, to form a combined image that displays the individual's image in a pleasing or preferred manner.
• U.S. patent 7,000,192 entitled “Method of Producing a Matted Image Usable in a Scrapbook” describes a method of producing a matted image usable in a scrapbook.
• the method comprises the steps of displaying a digital image having a matte area surrounding an image opening.
• a plurality of color selections from which a user can select a preferred color are provided and the selected color is used in the matte.
• the color selections can be obtained from an analysis of the digital image, thereby ensuring that the matte color matches a color from the digital image.
• simply selecting colors that occur frequently in an image can provide unacceptable results. For example, some colors may not be easily distinguishable to a viewer if they are part of a texture.
• some colors may be found in image locations that do not complement a matte, or may correspond to colors that are unacceptable for a matte. While a user may choose a suitable color that overcomes these deficiencies, it is desirable to automate the color selection process to enable users to be efficient and effective.
• color palettes by analyzing the colors in an image.
• Such color palettes can be used for purposes such as displaying digital images on color displays capable of representing only a limited number of colors (e.g., 256).
• a limited number of colors e.g., 256.
• Method for representing a digital color image using a set of palette colors based on detected important colors Such methods are also known to be useful for determining website color palettes.
• U.S. Patent 6,711 ,291 describes the use of colors found in an image to provide a textual caption with a selected color within the image.
• the selected color can be found within the image by forming a color histogram of the entire image. Colors within the histogram are then candidate colors. A candidate color having suitable contrast with the area in which the textual caption is to be placed is selected and the textual caption of the suitably-contrasting color combined with the image.
• a color found in the histogram at a high frequency may be, for example, part of a large textured area that includes multiple colors.
• the high-frequency color used in, for example, a matte may not provide a pleasing combination, since the specific color within the textured area may not be readily apparent to a viewer.
• a method of combining an image with an imaging-related product comprising:
• the present invention has the advantage that more suitable colors are selected from an image and employed in an imaging-related product with which the image is to be combined resulting in a more aesthetically pleasing result.
• FIG. 1 is a flow chart of a method according to an embodiment of the present invention
• FIG. 2 is a flow chart of a method according to an alternate embodiment of the present invention.
• FIG. 3 is an image useful in understanding the method of an embodiment of the present invention.
• FIG. 4 is a presentation screen useful in implementing a method of an embodiment of the present invention.
• FIG. 5 is a presentation screen useful in implementing a method of an embodiment of the present invention.
• a method of combining an image with an imaging-related product comprises a provide image step 100, a find spatially-coherent color clusters in image step 105, and a form color histogram of image color clusters step 110, the color histogram indicating the frequency of occurrence of spatially-coherent related colors in the image.
• a select image color step 115 selects one or more of the image colors from the color histogram having a frequency of occurrence in the image high enough to be visible to an image observer.
• a select image product(s) step 130 selects an imaging-related product having one or more of the selected high-frequency colors within the imaging-related product or having a color complementary to one or more of the selected high-frequency colors within the imaging-related product.
• a combine image and product step 145 combines the image and the imaging-related product to form a desired integrated product.
• the select image product(s) step 130 is used to select a plurality of imaging-related products having one or more of the selected high-frequency colors within the imaging-related product or having a color complementary to one or more of the selected high- frequency colors within the imaging-related product.
• the plurality of imaging- related products provides a customer with a corresponding plurality of product choices.
• the plurality of imaging-related products or a representation of the plurality of imaging-related products can be optionally presented to a user using a present selected product choices step 135, for example in a graphic user interface presented on a website or in a software application.
• the plurality of imaging- related products or the representation of the plurality of imaging-related products can be presented in an order, for example on a web page displaying the product choices from left to right and from top to bottom.
• the order can be derived in different ways, for example by frequency of customer choice or by a quality-of- match metric relating the matching of image and imaging-related product colors.
• a quality-of-match metric can be calculated from a degree of matching for each of the plurality of imaging-related products and the image and the order can be defined by the degree of matching. Such calculations can be done by an image processor known in the art.
• the imaging- related product can include more than one image color having a high frequency of occurrence and a plurality of image colors are used in matching and selecting the imaging-related product. Both, matching colors or complementary colors can be chosen so that the imaging-related product can include more than one color matching or complementary to an image color having a high frequency of occurrence.
• the imaging-related product can have both a neutral color and a selected color (either matching or
• the color clusters can be derived from a portion of an object that has a texture. In this case, the texture can be reproduced within the imaging-related product.
• the steps of the method of the present invention can be performed in various orders.
• the imaging-related product is constructed before the image is provided. This is useful, for example, when the imaging-related product is not printed with the image, for example when the imaging-related product is a frame or mug or cardboard matte.
• the imaging-related product is constructed after the color is selected using an optional construct selected product step 140. This can be useful when the combined image and imaging-related product are constructed together, for example as can be done for printed photocards.
• the imaging-related product for photocards can include a surround (composited matte) or background template with image openings and a predetermined design.
• the image and the imaging-related product are composited, for example with an image processor, and printed together on a common sheet of photographic paper or on printing paper.
• the actual image color selected in the select image color step 119 can be used in the imaging-related product, rather than selecting a best match from a set of available pre-constructed imaging-related products.
• an optional find spatially coherent color clusters in product step 120 can be used to find spatially coherent color clusters in the imaging-related product.
• a form product color histogram of product color clusters step 125 is then used to form a product color histogram that indicates the frequency of occurrence of spatially coherent related colors in the imaging-related product (as opposed to color clusters within the image).
• the select image product(s) step 130 determines one or more product colors having a frequency of occurrence high enough to be visible to an image observer, and matches the imaging-related product colors with the image colors to match the image with the imaging-related product. As noted above, the matches can be ordered in a list that is presented to a user for selection.
• the main subject of the image can be found using a find main subject step 116.
• a find spatially coherent color clusters in image step 117 is used to find spatially coherent color groups within the main subject.
• a form color histogram of color clusters step 118 is used to determine a color histogram of the colors in the spatially coherent color clusters.
• a select image color step 119 is used to select an image color that is not a flesh tone.
• Main- subject determination is a process well-known to those skilled in the art and is described, for example, in U.S. Patent 6,282,317.
• the imaging-related product can be any type of product known in the art including, but not limited to, a picture frame, an album cover, a mug, a calendar page, a matte, or a page surround matte or template in a photocard or album page.
• the image provided by the provide image step 100 can be a digital image.
• the provided image can be a hard copy image such as a photograph print or transparency, which can be scanned to form a digital image.
• the calculation of the color clusters and formation of the color histograms can be done by using a digital processor to process the provided image, for example with a digital computer or image or signal processor, all of which are commercially available.
• Composited products can be composited using similar computing hardware and printed, for example, using thermal-transfer printers onto thermal paper, using ink-jet printers onto specialized photo paper or plain paper, or using electro-photographic printers. Products such as framed pictures can be assembled by hand using prints of customer images.
• the method of the present invention can be practiced operationally by a user who provides a digital image 10, for example an image captured by a digital camera or scanned from a print.
• the user can then access an internet web site through a commercially-available browser and upload the digital image 10 to a storage device accessed through the web site.
• the digital image 10 can then be processed by a computer to determine spatially-coherent color clusters (e.g., color clusters 12 and 16).
• An image color histogram is then formed from the color clusters 12 and 16, and one or more image colors are selected from the image color histogram.
• FIG. 4 shows an example presentation window 30 associated with a user interface that can be provided on an internet web site or in a software application according to the method of the present invention.
• the selected one or more image colors can then be integrated into one or more imaging-related products 20, which are presented to the user as choices.
• the imaging-related products 20 represent three different matte surrounds 24, 26 and 28.
• a variety of color choices can be provided and ordered to assist the user in making a selection.
• matte surround 24 can include colors corresponding to both color clusters 12 and 16.
• matte surround 26 can include a neutral color together with a color corresponding to color cluster 16.
• Matte surround 28 illustrates a third option incorporating a single image color corresponding to color cluster 16.
• the imaging-related products 20 can be processed to determine product color clusters and corresponding product color histograms.
• the colors in the product color histograms can also be used to improve the color matching between the digital image 10 and the imaging-related products 20.
• a main subject detection algorithm is used to identify a main subject 14 in the digital image 10. Imaging-related products 20 can then preferentially be chosen containing colors corresponding to color clusters 12 that are contained within the main subject 14 rather than colors corresponding to color clusters 16 that are in the background of the digital image 10.
• the presentation window 30 also includes a sample presentation 22 of the digital image 10 shown in combination with one of the imaging-related products 20 for purposes of giving the user a preview of the combination.
• a user interface can be provided to allow the user to select different imaging-related products 20 and to update the sample presentation 22 to preview the selected imaging-related product 20.
• the user can add the combined product to a shopping cart and proceed with a product ordering process as is well-known to those skilled in the art.
• the user selection is then transferred to a production facility, together with the user's digital image for construction and shipping.
• imaging-related products for example picture frames, a pre-made element is combined with a print of the user's image.
• imaging-related product is produced at the same time as the user's image, typically with the image composited into the product by an image-processor.
• a spatially coherent color cluster is a group of similarly-colored pixels that are spatially adjacent within an image.
• a color that occurs frequently within an image does not necessarily occur within color clusters. For example, the color may occur in small areas scattered throughout the image. Therefore, use of an average color, or a color found predominantly within an image does not necessarily provide a good color match to an imaging-related product.
• imaging-related products can artistically benefit from matching accent colors within an image that can be derived from color clusters in order to ensure that the colors are easily visible to an image observer.
• Color clusters can be determined by measuring the spatial distance between pixels within the image that have similar colors as described below. Similar colors can be determined by measuring differences in hue to form a color distance value. Threshold values can be employed to define a minimum spatial cluster size and color variation (color distance) and groups of pixels meeting the threshold requirements can be determined. Typically, it is desirable that the minimum spatial cluster size is large enough to be readily perceived within an image at a suitable viewing distance.
• object segmentation within a scene can be used to determine major areas within a scene, for example as described in U.S. Patent
• an example user interface useful in selecting accent colors and then compositing the selected accent color as a matte surrounding the image according to the present invention is shown.
• a digital image 10 is displayed within a sample presentation 22 within a presentation window 30 of a computer graphical user interface.
• a selection of color options 34 corresponding to colors determined by analyzing the digital image 10 using the method of the present invention is shown to the right of the digital image 10, together with a set of user-interactive controls 32 useful in defining the color and spatial variation desired for the imaging-related product (in this example a matte surround).
• the user-interactive controls 32 can be determined and set automatically without requiring any user input.
• the composited result selected using the user-interface of FIG. 5 can then be printed, for example with an electro-photographic printer, laser thermal printer, or inkjet printer.
• the color options 34 are determined using an algorithm that creates a color histogram for the digital image 10.
• the histogram is constructed using a color representation yielding fewer bins than the total number of colors that can be expressed by typical 24-bit RGB values.
• Each pixel is tested for coherency with its neighboring pixels before being added to the histogram. This test avoids transitional pixels that occur on edges or as parts of strong patterns that can be perceived differently from non-transitional pixels, for example those found in patches of pixels that share similar color values, and provides both spatial and color coherency.
• the histogram of spatially coherent colors is examined. Colors with insufficient population (low histogram frequency) and those colors too close to neutral to be perceived as colorful are discarded.
• the remaining colors are clustered by hue into groups.
• a representative of each group is selected using different methods for correspondingly different selection criteria.
• the selected colors are displayed for the user.
• the color is applied to the background matte surrounding the image.
• colors can be clustered into groups by hue by using the following process.
• the average pixel color of the members of each histogram bin are expressed using a color space representation such as HSV, HSL or some other luminance-chrominance color space for which hue angle can be readily computed such as CIELAB or CIELUV described in chapter 13 of "Computer Graphics Principles and Practice", by Foley, et al. (Addison- Wesley, Boston, 1996).
• the distances between color bins are calculated from the differences in hue angle.
• a complete link agglomerative clustering algorithm such as the one described by Anil K. Jain in Chapter 3 of "Algorithms for Clustering Data” (Prentice-Hall, Englewood Cliffs, NJ, 1988), can be used to cluster these colors into groups.
• Representative colors for each group can be selected or created using one of several different methods to determine the degree of matching with a desired color characteristic.
• a representative color may be created that is the average color of all of the samples in the group. Alternately, the color bin closest to the average color of the group may be selected. Some applications may favor colors with certain characteristics. Colors with the highest saturation may be selected, or colors that come closest to an ideal saturation or luminance level may be preferred. Other selection criteria will be addressed later.
• a plurality of images can be processed, common colors identified, and an imaging-related product selected that employs a common color (or a color complementary to the common color, or a color found in one image and complementary to a color found in another).
• a common color or a color complementary to the common color, or a color found in one image and complementary to a color found in another.
• the imaging-related product can also be analyzed to find spatially coherent color clusters that are matched to multiple images, thus automating what can otherwise be a tedious and difficult task.
• the histogram used for selecting colors can be constructed in several manners.
• a single histogram can be created from the combined pixels of the separate images.
• a single histogram can be created by combining the histograms of the separate images.
• Such a combined histogram may be constructed from the union or intersection of the histograms of the separate images.
• colors of spatially coherent color clusters that are not near the edge or border of an image are preferentially selected to avoid clashing or blending with colors found in the imaging-related product, thus improving the aesthetic quality of the composited or combined result.
• complementary colors are those colors that are deemed by a human observer to provide a harmonious combination. Such combinations can be of colors that are strictly complementary on a color wheel, but can also include colors that are part of a common color chord, as is taught in literature on color matching, such as "The Principles of Harmony and Contrast of Colors and Their Applications to the Arts” by Chevreul (Schiffer, West Chester, PA, 1987) and “The Elements of Color", by Itten (John Wiley & Sons, Germany, 2003). Likewise, matching colors can include those colors whose hues are close enough that a user cannot distinguish the difference or whose hues are close enough so that objectionable color clashing does not occur.
• a color whose luminance is different but whose hue is similar when selecting matching colors that is, colors that are a darker or brighter shade than is found in the image can be employed.
• the image be brighter than the imaging-related product (e.g. a matte surround or frame), so a matching color might have a similar hue to a spatially coherent color cluster found within a desired image but have a reduced luminance.
• Prior art techniques for example determining matte colors by calculating an average image color or an average object color, are inadequate for providing aesthetic colors useful in imaging-related products associated with a specific image. Such techniques fail to determine accent colors effectively, since an object, or image, can have color elements within it that are markedly different from the average color of the object or image. Average colors of an image or object within an image are often gray. Furthermore, color representations on a computer are often different from the color of the actual image or imaging-related product. Hence, the present invention provides an improved method for matching images with imaging-related products. Those desiring to match images to imaging-related products also benefit from providing different color choices derived from different color clusters having different colors within an image. Such choices allow one to indulge personal preferences while ensuring suitable color matching.

Landscapes

• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Image Processing (AREA)
• Facsimile Image Signal Circuits (AREA)
• Color Image Communication Systems (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=44041666

Family Applications (1)

Country Status (5)

Cited By (4)

Families Citing this family (9)

Citations (7)

Family Cites Families (16)

• 2010
  • 2010-03-04 US US12/717,440 patent/US8630485B2/en not_active Expired - Fee Related
• 2011
  • 2011-02-22 CN CN201180011700.5A patent/CN102782726B/zh not_active Expired - Fee Related
  • 2011-02-22 WO PCT/US2011/025635 patent/WO2011109181A1/en not_active Ceased
  • 2011-02-22 JP JP2012556098A patent/JP5943845B2/ja not_active Expired - Fee Related
  • 2011-02-22 EP EP11707004A patent/EP2543021A1/en not_active Withdrawn

Patent Citations (7)

Non-Patent Citations (8)

Also Published As

Similar Documents

Legal Events