AU745947B2 - Color advisor - Google Patents

Description

S&F Ref: 489099

AUSTRALIA

PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT

ORIGINAL

Name and Address of Applicant: Actual Inventor(s): Address for Service: Invention Title: Canon Kabushiki Kaisha 30-2, Shimomaruko 3-chome, Ohta-ku Tokyo 146 Japan Josiah Poon Spruson Ferguson St Martins Tower 31 Market Street Sydney NSW 2000 Color Advisor ASSOCIATED PROVISIONAL APPLICATION DETAILS [33] Country [31] Applic. No(s) AU PP8506 [32] Application Date 05 Feb 1999 The following statement is a full description of this invention, including the best method of performing it known to me/us:- 5815c [R:\LIBU]19881 .doc:vsg -1- COLOR ADVISOR Field of the Invention The present invention relates to the selection of color for image components, such as the characters in a text string or graphical objects, intended for display or printed over some color background by a colour display apparatus or hardcopy printout.

Background Art When text is required to be displayed or printed on a colorful background incorporating images and/or graphic objects, it often transpires that the text optically merges with, or is absorbed by, the background colors and becomes difficult, and lo sometimes impossible, to read. The same problem often occurs in the display of animated images, such as those formed by graphical objects over some backgrounds. These "problems are often encountered with subtitles in motion picture films or videos. An o extreme example of this effect is the well-known Ishihara Color Blindness Test.

ooooo There are a few computational approaches that can be employed to make Is foreground image components distinctive from background image components. One approach to make foreground text readable is to introduce an additional plain color (opaque) strip over the primary (background) image before the text is painted onto the "new" strip. The disadvantage with this approach is that the strip may block out important information in the primary image.

Another approach is to use a colored transparency strip instead of an artificial solid background. This is an improvement over the previous approach but a portion of the original surface will still be distorted by the band of transparency.

Another approach analyses the color content of the underlying surface, and the text is outlined with a color complementary to the predominant color of the background while the body of the text is filled with the dominant color of the analysed region. This technique produces a so-called "wired-frame" font. The success of this approach depends upon the size of the font being used. A smaller font may not be suitable with this CFP1588AU MMedia3l 489099 [I:\ELEC\CISRA\M M EDIA\M M EDIA31 ]489099.doc:ldp approach as the color distinctions may not be readily apparent. Also, it may not be desirable to have a wired-frame font all the time.

It is an object of the present invention to address the deficiencies of existing arrangements.

Summary of the Invention In accordance with one aspect of the present invention there is disclosed a method of determining a displayable color for a current foreground image component intended to be displayed over at least one background image component having a background color, said foreground image component being one of a plurality of foreground image io components having a predetermined positional interrelationship, said method comprising the steps of: o. analysing dominant colors of a background image component over which said o• foreground image component is to be displayed to provide one or more representative colors; is examining at least one displayable color of a previously determined foreground image component from said plurality of foreground image components to determine a level of contrast with said representative colors; :wherein if the level of contrast is at least a predetermined level, assigning the displayable color of the corresponding previously determined foreground image S: 20 component to said current foreground image component; and if the level of contrast falls below said predetermined level, applying a predetermined set of rules using said dominant colors to select said displayable color from a predetermined palette of colors.

In accordance with another aspect of the present invention there is disclosed a method of determining a displayable color for each foreground image component of a group of foreground image components to be displayed over a background image having a plurality of colors, said method comprising the steps of: RA4 establishing a palette of colors from which said displayable colors are to L T be selected;

LU

O11§ 489099.doc for each said foreground image component (bl) determining from an area of said background image over which said foreground image component is to be rendered, one or more dominant colors of said background image; (b2) selecting a color from a list of previous foreground colors and comparing said selected color against said dominant colors to determine a current level of contrast; (b3) comparing said current level of contrast with a predetermined level of contrast, wherein if said current level is at least said predetermined level; (b3a) said selected color is assigned as said displayable color for said foreground image component; or (b3b) repeating steps (b2) and (b3) until each member of said list iii ~has been considered whereupon; if said list is empty or each said member has been considered: (cl) applying a predetermined set of rules using said dominant colors to determine said displayable color from said palette; and (c2) adding said displayable color to said list.

According to still another aspect of the present invention there is disclosed a Ssystem for determining a displayable colour for a current foregoing image component 20 intended to be displayed over at least one background image component having a background color, said foreground image component being one of a plurality of foreground image components having a predetermined positional interrelationship, said system comprising: means for analysing dominant colors of a background image component over which said foreground image component is to be displayed to provide one or more representative colors; means for examining at least one displayable colour of a previously determined i foreground image component from said plurality of foreground image components; 489099.doc -3ameans for determining a level of contrast with said representative colors depending on said examination; means for assigning the displayable color of the corresponding previously determined foreground image component to said current foreground image component, if the level of contrast is at least a predetermined level; and means for applying a predetermined set of rules using said dominant colors to select said displayable color from a predetermined palette of colors, if the level of contrast falls below said predetermined level.

Other aspects of the invention are also disclosed.

Brief Description of the Drawings A preferred embodiment of the present invention will now be described with i reference to the drawings in which: •o•Fig. 1 is a schematic block diagram of an implementation of the preferred embodiment: S 5is Fig. 2 is a flowchart depicting operation of the preferred embodiment; Fig. 3 illustrates one example implementation of the preferred embodiment; Fig. 4 is a block diagram of a general purpose computer upon which the preferred :embodiment of the present invention can be practiced; and Appendix A is a program/pseudo code listing of the preferred embodiment.

Detailed Description Fig. 1 schematically shows a system 100 for generating foreground color for a string of text such that each character can adapt to the color changes of the underneath background image, while a number of foreground colors used can be controlled so that variability of Colors can be manipulated.

In Fig. 1, an image 102 and text 104 are provided to a foreground color advisor 106 of the preferred embodiment. The image 102 may be formed of bitmap, pixel -AL or other known image data types including graphical object data such as computer a lmation. The text 104 may be of any style, size, or combination of both, and is !ded to be rendered in the foreground (on top of) the image data 102. The color zttended to be rendered in the foreground (on top of) the image data 102. The color 489099.doc 3b advisor 106 examines the image 102 at locations where the text 104 is present and determines color attributes for each character in the text 104 which are output 108 to be used in the rendering of the combined image.

I 0

,Z~

a *i 48999*o The color advisor 106 operates according to a number of method steps 200 as illustrated in Fig. 2.

Initially, at step 202 a user chooses a desired color style of the foreground text from a set of templates. Each template contains rules for a particular color style or scheme, for example complementary, supplementary, designer, punky, childrens, etc.

Example of such color schemes are those available with the WindowsTM operating system manufactured by Microsoft Inc.

At step 204, a string of characters is entered such that the user can use some means (using a cursor or specifying the coordinate) to indicate the start location where the text is 1o to be placed. The user at this stage may also specify the font information to be used (font type, font size, etc). Preferably, the user is provided with a means to provide the system with a maximum number of previously generated foreground colors (maxColor) to look up for use in color advising. For example, only a limited color palette may be available in some applications. Further, this may be desirable to avoid the final text appearing too Is colorful.

Step 206 which follows, commences a recursive procedure for advising a particular color for each character in the text string, which examines the first character from the string of text, and uses the current position specified by user as the start location (startx, start y) for the character. Using the font information available, step 206 20 determines the width and height occupied by the current character.

Step 208 applies color analysis to determine the dominant colors covered by the area at location (start_x, start_y), which has a width of w and a height of h. The dominant colors may be determined in a number of ways. One method is to examine the defined area and count the instances of each color within the area. This may produces first, second and third (for example) dominant colors, any of which may be considered of selected as required. A specific example of a useful color analysis method that may be alternatively used is that disclosed in Australian Patent Publication No. AU-A-88325/98 (Attorney Ref: 425248 CFP1007 Mmedia08). The result of this analysis returns a limited CFP1588AU MMedia3l 489099 [I:\ELEC\CISRAMMEDIAMMEDIA31 ]489099.doc:ldp number of representative (or dominant) colors. Typically, the number of representative colors returned will depend on the size of the area examined.

In step 210, a pointer is set to be the last element of a list of previously generated foreground colors (lprevfg_color). If no such list exists (ie. the list is empty as is the case for the first character in a text string), step 212 uses the rules from the selected template to generate an appropriate foreground color for the current character with respect to the dominant color information from the analysis of step 208. In step 214 the generated foreground color information is attached to the current character and this generated foreground color is added to the end of the list of generated foreground colors to (lprevfg_color).

Step 222 follows which assesses whether there are remaining characters in the text string. If so, control returns to step 206. This may be achieved by incrementing start-x by w, to thus obtain the starting location of the next character in the text string.

Where, at step 210 the list includes one or more previously generated colors, step 216 follows which tests the compatibility of the generated foreground color at a previous •location p from the list of previous generated foreground colors (lprevfg_color) with the dominant colors for the current area. This may be performed using any one of a number of known contrasting color determining methods.

Step 218 determines if the previous generated foreground color is at an acceptable and compatible level with the background dominant color at the current location. If so, step 220 follows by applying the foreground color information to the current character.

Since that foreground color was extracted from the top of the list, the list does not require updating. If the size of the list of previous generated foreground colors (lprevfg_color) is or exceeds the maximum size (maxColor) specified by the user, one approach is to shift out the less frequent used color.

If the color is not acceptable, the pointer p is decremented and control returns to step 210 which proceeds to examine the next color in the list in a corresponding manner.

In this fashion color advising for any character in the text string is first performed using CFP1588AU MMedia3l 489099 [I:\ELEC\CISRA\MMEDIA\MMEDIA31 ]489099.doc:ldp those colors in the list of previously used color. Specifically, in the preferred embodiment, the previously used colors are checked in order of most recent use.

If the pointer p is decremented to zero, step 212 is applied to extract an appropriate color from the template as occurs with the first character.

Next, step 222 is again encountered and acted upon as before.

When there are no more characters in the string to be processed, the string of characters and their color attributes are output to the renderer for display.

The preferred embodiment relies upon the notion that a text string, being an ordered collection of objects having distinct shape and positional interrelationship, provides the capacity for possible incremental color changes between characters to ensure visibility over a background. With this, the amount of contrast between individual text characters may be reduced where desired whilst ensuring a desired level of contrast with the background. This may, in some applications, produce a result somewhat equivalent to a color blend across the text string.

The preferred embodiment of the invention is implemented as an application program typically operating on a conventional general-purpose (host) computer system, such as the computer system 40 shown in Fig. 4, wherein the application program is implemented as software executed on the computer system 40. The computer system comprises a computer module 41, input devices such as a keyboard 42 and mouse 43, output devices including a printer 13 and a display device 11. The keyboard 42 and mouse 43 may be used in a conventional manner for the entry of the text string and the positioning of the text string to thus specify startx and start_y. A Modulator- Demodulator (Modem) transceiver device 52 is used by the computer module 41 for communicating to and from a communications network, for example connectable via a telephone line or other functional medium. The modem 52 can be used to obtain access to the Internet, and other network systems.

The computer module 41 typically includes at least one processor unit 45, a memory unit 46, for example formed from semiconductor random access memory (RAM) and read only memory (ROM), input/output interfaces including a video interface CFP1588AU MMedia3l 489099 [I:\ELEC\CISRA\MMEDIA\MMEDIA31 ]489099.doc:ldp -7- 47, and an I/O interface 48 for the keyboard 42 a mouse 43 and optionally a joystick (not illustrated). A storage device 49 is provided and typically includes a hard disk drive 53 and a floppy disk drive 54. A CD-ROM drive 55 is typically provided as a non-volatile source of data. The components 45 to 49 and 53 to 55 of the computer module 41, typically communicate via an interconnected bus 50 and in a manner which results in a conventional mode of operation of the computer system 40 known to those in the relevant art. Examples of computers on which the embodiments can be practised include IBM- PC's and compatibles, Sun Sparcstations or alike computer systems evolved therefrom.

Typically, the application program of the preferred embodiment is resident on a hard disk drive 53 and read and controlled using the processor 45. Intermediate storage of the 9. 9 program and the print list and any data fetched from the network may be accomplished 9.oO9.

using the semiconductor memory 46, possibly in concert with the hard disk drive 53. In some instances, the application program may be supplied to the user encoded on a CD- ROM or floppy disk, or alternatively could be read by the user from the network via the modem device 52.

tot .060 "Example With reference to Fig. 3, the word "WORLD" is to be displayed over some arbitrary .colorful background. After processing the first three characters according to the method 200, has been suggested to use green color, the second letter has been allocated 9*9999 20 blue and the third character, allocated the red color. At this stage, it is then necessary to determine the color for the next letter It can be seen that the area occupied by each character is not the same, in that the analysis space (h.wl) for the letter is definitely much larger than the space (h-w2) for the current letter Dominant colors over the area for character are known after the analysis. The foreground colors used by its predecessor characters are tested, from the nearest character to the furthest character, (ie. red, blue, then green). In other words, the first priority to try for the color of the letter is red, then blue and finally green as indicated by the arrows numbered 1, 2 and 3 respectively in Fig. 3. If the color used by its previous neighbours is within a threshold value to make the character stand out from its background, and is CFP1588AU MMedia31 489099 [I:\ELEC\CISRAM EDIA\MM EDIA31 ]489099.doc:Idp -8appropriate for the selected style, that previous color will be adopted by the current character Otherwise, using the available rules from the chosen template of color style, a different color is generated for if the previous colors do not work in harmony with the background colors underlying the current character.

The threshold for determining whether or not a foreground color will be sufficiently distinguishable from the background color may be determined in a variety of ways.

In the preferred embodiment, the significant background color(s) used and the suggested foreground color are converted individual components for each of hue saturation and brightness These component values are normalised. A "distance" factor between these normalised values may then be determined, for example using the 0 0 expression: distance [(hFG-hBG) 2

(SFG-SBG)

2 (bFW-bBG)]'.

The threshold value may be a predetermined value of this "distance". Where *4 15 appropriate, other color formats, such as RGB, CMY or YCrCb may be used although the 00* 0form of determination of "distance" may change.

For complementary colors, it is possible to test the difference between the hue of *1 each of the foreground and background. For example: hue max. saturation IhFG-hBG I Hfg,Hbg: 20 Alternatively, for non-complementary colors, it is possible to utilize "children" colors in order to maximize brightness. In this regard, children typically prefer bright colors, for example with a brightness greater than 0.8. A template of such colors may be used to define "children colors". Other color schemes such as "punky" or "horror" may also be appropriate.

Where a text character appears over a relatively abrupt change in background, in some embodiments a blend of colors may be applied across the text character.

Appendix A shows program/pseudo code for implementing the preferred embodiment.

CFP1588AU MMedia3l 489099 [I:\ELEC\CISRA\MMEDIA\MMEDIA31 ]489099.doc:ldp The foregoing describes only a number of embodiments of the present invention and modifications may be made thereto without departing from the scope of the present invention. For example, although finding particular application in the display of text, the present invention may be applied to any arrangement of image components, graphical s objects for example, that are desired to be displayed. This, for example, may apply in the presentation of animation overlying a pixel image. It is further noted that the scope of the color palette from which a selection is made can vary depending upon a number of user detennrminable or selectable criteria. For example where a text string is to be displayed over pixel image of a jungle where the dominant colors are a variety of shades of green representing different foliage, the contrast criteria may identify yellow as a suitable base color for the text. Using such a base color, a palette having a range of yellow colors may be selected for use in the text. Depending on the particular shade of green over which a character is to be displayed (eg. olive), a suitable color (eg. sand) may be selected from i the palette. Accordingly, where the background provides a blend of colors, the coloring of the overlying text can also display a complementing blend of contrasting colors. The net effect of such an arrangement is that, due to the color mixing insensitivity of the human eye, the text string, although displayed with a range of colors (yellows) may .oooo: appear to be formed of characters of substantially the same intensity of apparently the same color. In this fashion, visibility of the string may be optimised without distracting 20 the viewer with colors spanning a wide range.

As a consequence, the present invention may find application in a wide number of fields, such as the production of static images by printing, and the creation of static electronic images for displays. The latter may include static billboards, date/time impressing identifiers on photograph devices as well as individual computer displays.

Other applications include the presentation of subtitles to video or motion pictures, the particular embodiment being performed in real-time, for example during a television sports broadcast, or via pre-processing as in the case of the production of celluloid motion picture films. Each of such products may incorporate the invention.

CFP188AU MMedia3l 489099 [I:\ELEC\CISRA\M MEDIA\MM EDIA3 I ]489099.doc:ldp APPENDIX A The following represents a pseudo-code representation of the preferred embodiment: generateNewColor false; FOR all the available background colours from the analysis IF the current background colour (bg_color) is significant say greater than 5% of the analysed area IF bg_color.brightness is similar to the previous generated fg_color OR ((fg_color brightness 0.6) AND S(bgcolor.hue is warmColor) AND (bg_color saturation 0.7)) generateNewColor true

ENDIF

ENDIF

ENDFOR

IF generateNewColor is false dom_bg_color the first non-zero saturated dominant bg_color; IF Idom_bg_color.hue fg_color.huel 0.1 20 generateNewColor true;

ENDIF

ENDIF

CFP1588AU MMedia~l 489099 I(:\ELEC\CISRA\MM EDIA\MM EDIA31 ]489099.doc:ldp

Claims (9)

1. A method of determining a displayable color for a current foreground image component intended to be displayed over at least one background image component having a background color, said foreground image component being one of a plurality of foreground image components having a predetermined positional interrelationship, said method comprising the steps of: analysing dominant colors of a background image component over which said foreground image component is to be displayed to provide one or more representative colors; examining at least one displayable color of a previously determined foreground image component from said plurality of foreground image components to determine a iii level of contrast with said representative colors; wherein if the level of contrast is at least a predetermined level, assigning the displayable color of the corresponding previously determined foreground image component to said current foreground image component; and if the level of contrast falls below said predetermined level, applying a predetermined set of rules using said dominant colors to select said displayable color from a. a predetermined palette of colors. 0¢ i

2. A method of determining a displayable color for each foreground image component of a group of foreground image components to be displayed over a background image having a plurality of colors, said method comprising the steps of: establishing a palette of colors from which said displayable colors are to be selected; for each said foreground image component determining from an area of said background image over which said foreground image component is to be rendered, one or more dominant colors of said background image;

489099.doc 12- (b2) selecting a color from a list of previous foreground colors and comparing said selected color against said dominant colors to determine a current level of contrast; (b3) comparing said current level of contrast with a predetermined level of contrast, wherein if said current level is at least said predetermined level; (b3a) said selected color is assigned as said displayable color for said foreground image component; or (b3b) repeating steps (b2) and (b3) until each member of said list has been considered whereupon; if said list is empty or each said member has been considered: (cl) applying a predetermined set of rules using said dominant colors to determine said displayable color from said palette; and (c2) adding said displayable color to said list.

3. A method according to claim 2, wherein a maximum number of colors able to be retained in said list is able to be limited.

4. A method according to claim 2, wherein said group comprises a string of text oo characters. i

5 A method according to claim 2, 3 or 4, wherein said foreground image components have a predetermined positional relationship and step considers each said foreground image component in positional order.

6. A method according to claim 5, wherein step (bl) includes defining a position and area of said foreground image component to determine said dominant colors. S

7. A method according to any one of the preceding claims, wherein at least one said 7zij foreground image component comprises text. 489099.doc -13-

8. A method according to any one of the preceding claims, wherein said foreground image component comprises a graphical object of a group of graphical objects.

9. A method according to claim 8, wherein said group of graphical objects define a graphical image. A method of determining, from a predetermined palette of colors, a displayable color for a foreground image component intended to be displayed over at least one background image component substantially as described herein with reference to the drawings. 0 11. Apparatus for performing the method of any one of the preceding claims. Is 12. A computer program product incorporating a computer readable medium having a series of computer interpretable instructions for performing the method according to any 0 0 00 er14. An image according to claim 13, being one of: a printed image; a developed image; a static electronic image; a motion electronic image; and a motion picture film. 15. A system for determining a displayable colour for a current foregoing image Scomponent intended to be displayed over at least one background image component 489099.doc 14- having a background color, said foreground image component being one of a plurality of foreground image components having a predetermined positional interrelationship, said system comprising: means for analysing dominant colors of a background image component over which said foreground image component is to be displayed to provide one or more representative colors; means for examining at least one displayable colour of a previously determined foreground image component from said plurality of foreground image components; means for determining a level of contrast with said representative colors 1o depending on said examination; means for assigning the displayable color of the corresponding previously •'-determined foreground image component to said current foreground image component, if the level of contrast is at least a predetermined level; and Ss means for applying a predetermined set of rules using said dominant colors to select said displayable color from a predetermined palette of colors, if the level of contrast falls below said predetermined level. o• DATED this thirteenth Day of February, 2002 Canon Kabushiki Kaisha 20 Patent Attorneys for the Applicant SPRUSON FERGUSON 489099.doc