GB2231760A

GB2231760A - Neural network for color selection

Info

Publication number: GB2231760A
Application number: GB9009467A
Authority: GB
Inventors: James Chen
Original assignee: Apple Computer Inc
Current assignee: Apple Inc
Priority date: 1989-05-18
Filing date: 1990-04-27
Publication date: 1990-11-21
Anticipated expiration: 2010-04-27
Also published as: JPH0394366A; GB2231760B; DE4015791A1; GB9009467D0

Description

:2:2:9 j - 7C=,0

NEURAL NETWORK FOR COLOR SELECTION. BACKGROUND OF THE INVENTION.

1. Field of the invention.

The present invention relates to the field of artificial intelligence based solutions to problem solving; more specifically, the present invention relates to selection of visually aesthetic color combinations using an associative memory system, specifically a neural network.

2. Description of Related Art.

Overview of Artifinini Tntelligence Applications It is known in the art to employ various methods for to accomplish "artificial intelligence" or "AI" with computer systems. While a strict definition of AI is not available, generally, AI may be described as a system in which a computer is able to emulate deductive inferential reasoning, e.g., the system may reach conclusions based on certain inputs after it has been "trained" or instructed in a certain set of rules or experiences.

Such a definition encompasses so-called "heuristic search" models as well as,associative memory" systems and llconnectionist" models. Each of these terms will be defined in more detail below.

- 2 The Heuristic Search Model A heuristic search model may be thought of as a system in which a problem is solved by following the branches of a tree. A set of decisions are made; each decision corresponding to a branching point on the tree. The search is controlled by a set of rules or "heuristics"; heuristics allow the system to focus on the branches of the tree most likely to provide a solution. Heuristic search models have been commercialized in a large number of applications such as game playing, natural-language processing, and so-called "expert system" and "knowledge-based" applications.

An example of the shortcomings of such a heuristic approach is described in David L. Waltz, Prospects for Building a Truly intelligent Machine, DAEDALUS Journal of the American Academy of Arts and Sciences, Volume 117, Number 1, pp. 191- 212.

Waltz describes an application for a heuristic model for determining whether a given object Oxu is a bird. In this application, one heuristic rule may be "if x is a bird, then x can fly". However, Waltz continues to describe the "qualification problem"; that is, given a general rule, one can always devise a fact pattern which requires further qualification of the rule. For example, in view of a world in which penguins and ostriches are - 3 classified as "birds" and in which both of these species cannot fly, the rule must be modified to "if x is a bird. then x can fly, unless x is one of a set of flightless birds". Further exceptions to the rule are necessary to reach correct results when operating in a world in which a bird may have had its wings clipped, may be dead, may have Its wings soiled with oil after an oil spill in its environment, etc. The possible exceptions to the "general rule" are limitless.

Heuristic search models further suffer from shortcomings in applications where "rules" are difficult to define. The rules may evade definition because the cognitive process utilized by humans in solving a particular problem may be difficult to define; because so-called "experts" in the field differ on what the "rules" are or should be; or some combination of these problems.

As will be seen, it is primarily this latter shortcoming which leads to the present invention.

Associative Memory Systems Associative memory systems have been proposed In which a memory database is provided having some historical set of solutions to a problem or set of problems. The associative memory system solves a current problem by examining symptoms or characteristics of the current A - 4 problem and comparing those symptoms to previous solutions to the problem. A solution is chosen from the historical set of solutions where the chosen solution had a set of symptoms close the present problem.

This type of system may be differentiated from heuristic search models in its employing a ulookup" process rather than a search through a set of rules. The items or solutions "looked up" are representations of specific solutions rather than rules.

An example of such an associative memory system is an application for diagnosis of medical problems. Given a database of symptoms and patient characteristics and a set of symptoms and characteristics for a particular patient, an associative memory system has been described by Waltz in which the system finds the most similar previous patients and hypothesizes that the same diagnosis should be applied to the new patient.

An associative memory system may be designed which can "learn" by applying its database to a new set of symptoms, determining whether the proposed solution yields a desirable result, and, if a desirable result is achieved, storing the new set of symptoms and solution in the database.

-5Neural Networks A "neural network" may be described as a particular implementation of an associative memory. In a neural netwo.rk, nodes or processing units are interconnected by 5 links, each link having a weight or strength.

Neural networks are inspired by current thought on processing and storage of information in the human brain. (The brain consists of a large number of highly interconnected elements which apparently send very simple excitatory and inhibitory messages to each other and update their excitations based on these messages.) In a neural network, each individual neuron may act as an input, as an output or may be "hidden". (A hidden neuron is connected to other neurons but cannot be accessed directly from input or output channels. This will be discussed in more detail below.) The connection between any two particular neurons is strengthened each time information flows between the two particular neurons; that is, given a problem represented by input neuron A, if a solution is found represented by output neuron B, the connection between A and B is strengthened.

A further description of neural networks may be found in Jack D. Cowan and David H. Sharp, Neural Nets and -6Artificial Intelligence, DAEDALUS Journal of the American Academy of Arts and Sciences, Volume 117, Number 1,, pp. 85-121.

Training A network is typically subjected to a "trainingn process during which it is trained with a set of inputs. It is desirable that the system. during this training process, eventually reaches a stabilized configuration. A stabilized configuration may be thought of as a state of the network in which applying a set of inputs to the network will yield a desired set of outputs. Auto-associative Networks Auto-associative networks are a class of associative memory systems which are trained through use of a set of examples; these examples are often termed "exemplars". The network is trained by using the exemplars as inputs to the network and also using the exemplars as target outputs. Weights of links between input and output nodes are adjusted until applying the set of exemplars at the input nodes leads to responses at the output nodes which are sufficiently close to the target responses.

Typical known auto-associative networks are only capable of working with binary values (O's and lls). Use -7of simple binary inputs allows for simpler stabilization and training of the network.

Honfield Networks

Cowan et al. describes a particular auto-associative memory system known as the Hopfield net. The Hopfield net is a network of neuron-like elements having symmetric connections. (Symmetric connections are formally defined as follows: If x and y are two neurons, their connections are symmetric if the weight of the x to y connection is equal to the weight of the y to x connection.) Cowan-et;91. discusses and points out that Hopfield nets are not "true" neural nets because each element, or neuron, must both excite and inhibit its neighbors. More specifically, each neuron will typically excite one neighbor and inhibit another.

one problem with a Hopfield net is that they can easily get trapped in a metastable configuration. In order to avoid metastable configurations, modified Hopfield nets have been proposed using a "Monte Carlo" procedure to allow the net to escape from the metastable configuration. Using a Monte Carlo procedure, the network makes essentially random configuration changes. If the change results in a more stable state, the change is retained; if the change results in a less stable configuration the change is 4 -B- is cancelled. In this way, a stable configuration is eventually found, however, learning can be very slow. Single and Multilayer Networks Cowan et al. further describes a type of network known as a multilayer network. Such a network may be described, in a rather simplified fashion, with reference to Figure l(A) and Figure l(B) and with comparison to single layer networks. The Hopfield net, discussed above, is computationally equivilent to a single layer network.

Figure l(A) illustrates a single-layer network having 3 input nodes 101, 102 and 103 and 3 output nodes 104, 105 and 106. Each of the input nodes 101, 102 and 103 are directly connected to each of the output nodes 104, 105 and 106. A simple network, such as illustrated by Figure I(A), may be useful for certain pattern recognition and classification problems. In general, this type of network may recognize patterns which can be classified and separated by a hyperplane. This shortcoming is explained in greater detail with reference to Richard P. Lippmann. An Introduction to Computing with Neural Networks, IWE ASSP Magazine, April, 1987, pp. 4- 22.

A class of networks which may recognize more complicated patterns and which overcomes many of the problems associated with the type of network shown in 1 -9Figure l(A) is termed a "multilayer" network. Such networks typically have an input and output layer interconnected by "hidden" layers.

Lip12mann describes a single-layered network as being able to distinguish regions bounded by a hyperplane.

Multi-layer networks are arbitrary regions, the complexity of which is limited by the number of nodes in the network.

Another way of viewing the shortcomings of a single layer network is that, in general, single layer networks may be fairly proficient at recognizing patterns which are reasonably close to their training patterns; however. single layer networks perform relatively poorly at recognition of patterns which are dissimiliar to their training patterns.

is Figure l(B) illustrates a multilayered network having 3 input nodes 110, 111 and 112 and 3 output nodes 116, 117 and 118. In addition the network comprises 3 "hidden" nodes 113, 114 and 115. The term "hidden" is descriptive of the fact that the node is not directly accessible from outside the network for either input or output; rather, a hidden node is connected between input and output nodes in the network.

-10Back PrQpagation Back propagation is one technique used for training of multilayer networks. During training involving back propagation. the network is first subjected to a forward stage in which inputs to the network are simulated and responses recorded. During the backward stage, the weights of the connections between hidden units and the output units are adjusted and, after adjusting the weights of these connections, the weight of the connections between 10 the input units and the hidden units are adjusted.

Figure l(C) is illustrative of such network which may be trained with a back propagation technique. As discussed above, first the network is subjected to an input at input node 120. The responses at output nodes 122, 123 and 124 are recorded. The weights of connection 126, 127 and 128 are then adjusted and, after adjusting the weights of these connections, the weight of connection 125 Is adjusted-thus, the name back propagation. The example of Figure l(C) Illustrates a network with a single input node. Other networks may have multiple input nodes; further, other networks may have a lesser or greater number of output nodes.

-11Color Selection Systems The above-discussion has centered on the history and development of known AI systems. The present invention is centered in the use of AI systems for color selection; specifically, as will be described below, the preferred embodiment of the present Invention is useful in the selection of colors for a computer interface and has specific application In the selection of colors for elements on Apple Macintosh computer interface.

It is known in the art to utilize AI systems for color selection applications. Penny Bauersfeld, Color Palette: A Knowledae- Based System to Select Colors for a Palette, Masters Thesis, Rochester Institute of Technology. School of Computer Science and Technology, August 16, 1988, describes a system for color selection using a knowledge-based system. Bauersfeld describes a knowledge-based system which is used to advise users in selecting colors for a palette used for computer screen design applications. The knowledge-base proposed by Bauersfeld may reason about the relation of individual colors in a set with others colors in the set based on a set of rules derived from research on interface design and color theory. Bauersfeld presents argument stating "The skills required to perform color selection are cognitive, -12and are readily articulated by experts". Bauersfeld, at page 10.

A second system for color selection in computer systems is described by Barbara J. Meier, ACE: A Color Expert System for User Interface Design, Proceedings of the ACM, SIGGRAPH Symposium on User Interface Software, Banff, Alberta, Canada, October 17- 19, 1988, pp. 117-128.

Meier describes an expert system which employs a set of rules in the selection of colors for computer interfaces and the like.

Although each of Bauersfeld and Meier propose systems for color selection, both of these references rely on a set of rules or heuristics to implement their proposed color selection process. As can be appreciated, although rules are available to aid in the selection of colors, no one set of rules is available which satisfies the criteria of all of the so-called "experts" or which covers all possible color selection combinations.

Therefore, the present invention recognizes that any implementation based on a set of rules or heuristics will suffer from shortcomings similar to the general shortcomings of heuristic systems discussed above.

It is therefore an object of the present invention to develop an associative memory system for color selection.

it is a further object of the present invention to develop an associative memory system with particular application in the design of color interfaces for computer systems.

It is another object of the present invention to develop an associative memory system for implementation of a color selection system.

These and other objects of the present invention will be seen in more detail with reference to the detailed description of the preferred embodiment and the accompanying figures.

-14SUWARY OF THE INVENTION An apparatus and method for assisting in the selection of color combinations is disclosed. The present invention discloses an associative memory system having particular application in the design of color interfaces for computer systems.

Utilizing an associative memory system, a color interface may be designed based on the preferences supplied by a user of the computer system. It has been found that use of an associative memory system for the color selection process yields advantages over known "rule based" color selection systems.

The system of the present invention is implemented in a computer system having a plurality of elements on each interface screen. In particular. the preferred computer system utilizes ten separate elements. The disclosed color selection process allows a user to designate colors for any of the elements in the interface from a selection of available colors. After selecting preferred colors for the desired elements, the user invokes the associative memory system of the present invention to assist in selection of a color scheme for all ten elements. The associative memory -is- system provides a color scheme having the user's designated colors for the selected elements.

The present invention further discloses other methods useful for the selection of colors for a computer interface including a method for saving color combinations; a method for reverting to a saved color set; a method for reviewing and user "designed" color sets; and a method for setting the elements of the color interface to a monochrome mode.

The present invention further discloses a neural network implementation for the color selection system. The neural network system may be generally described as a hybrid of an auto-associative memory systems and a multi-layer neural net using back propagation for learning.

The network of the present invention is comprises a plurality of input units; each input unit corresponding to an element of the computer interface. The network further comprises a plurality of output units; again, each output unit corresponds to an element of the computer interface. Each input unit Is coupled with all output units, except for the output unit corresponding to its own element, through a sub- network. Each output unit is coupled to its corresponding input unit through a recurrent link. Thus, this main network may be thought of as the auto-associative -16network portion of the hybrid network of the present Invention.

Each sub-network comprises a plurarity of input nodes corresponding to each of the elements of a color representation system and a plurality of output nodes also corresponding to the elements of a color representation system. The input nodes are coupled to the output nodes through hidden nodes. In the preferred embodiment, RGB color space is utilized; therefore, there are three input nodes corresponding to red, green and blue, respectively, and three output nodes also corresponding to red, green and blue. Each of the three input nodes is completely connected to three hidden nodes. Each of the three hidden nodes is completely connected to the three output nodes.

Thus, each of the subnets may be thought of as a multi-layer network portion of the hybrid network of the present invention.

The present invention discloses a new technique for relaxation of the network. Relaxation of the network of the present invention utilizes a technique termed back activity relaxation. The back activity relaxation technique of the present invention is a method for reducing energy in the network by relaxing the activity (as opposed to known methods of relaxing weights) of the network.

-17BRTEF DESCRIPTION OF THE DRAWINGS

Figure l(A) is an illustration of a prior art single layer network.

Figure l(B) is an Illustration of a prior art multilayer network.

Figure I(C) is an illustration of a prior art network showing a back propagation technique.

Figure 2 is a illustration of a display screen of the preferred embodiment of the present invention.

is Figure 3 is a flow diagram illustrating a method for accessing of the color selection process as may be utilized by the present invention.

Figure 4 is a flow diagram illustrating a method for color selection as may be utilized by the present invention.

-1 B- Figure 5 is a flow diagram illustrating a method for setting the colors of elements of a display to monochrome values as may be utilized by the present invention.

Figure 6 is a flow diagram illustrating a method of the present invention for reviewing designer color sets.

Figure 7 is a flow diagram illustrating a method of the present invention for reverting to a previously saved 10 color set.

Figure 8 is an illustration of a network as may be utilized by the present invention.

is Figure 9 is an illustration of a sub-network as may be utilized by the present invention.

1 -19DESCRIPTION OF THE FERRED RMBnnTMENT

A system for selection of colors is described. In the following description, numerous specific details are set forthsuch as specific computer interfaces, specific colors, etc., in order to provide a thorough understanding of the present invention. it will be obvious, however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known circuits, structures and techniques have not been shown in detail in order not to unnecessarily obscure the present invention.

INTRODUCTION

The present invention describes a system for selection of colors having particular application in the selection of colors for a computer interface. The preferred embodiment of the present invention is implemented on the Apple Macintosh II computer, although it will be obvious to one of ordinary skili in the art that the present invention will have application in other computer systems, as well as outside of the computer arena.

The present invention further describes a preferred implementation of a color selection system comprising a neural network which is described as a multilayered -20auto-associative network with recurrent links from output back to Input. Although the network is described with respect to the specific application of the color selection problem, it will be obvious to one of ordinary skill in the 5 art that the network may be useful in other applications. Further, it will be obvious to one of ordinary skill in the art that alternative embodiments may utilize other associative memory systems to accomplish color selection. BACKGROUND COLOR INTERFACE DEVELOPMENT

In modern day computer systems, the role of color Is playing an ever increasing part. This is true in the use of computer systems for storing and presenting user information, such as color screens and slides for presentations; in the use of high-resolution color graphics for engineering and other uses; and in the design of computer system interfaces.

In particular, it has been found that use of color in the design of computer interfaces can lead to aesthetically pleasing interfaces as well as improvements In the ability to convey information with the interfaces.

However, it is desirable to allow an end-user of a computer system to alter the colors of the interface, either because of the end-user's personal preferences or to better convey information in a way useful to the particular -21user. One problem with allowing users to "customize" the colors in their interface is many users may have difficulty in obtaining aesthetically pleasing results.

Therefore, a system is desired which allows users to customize their interface while obtaining assistance with color selection in order to obtain desirable results. An analogy may be drawn to use of an interior decorator who offers assistance in the selection of a color scheme for a home or office. Typically, the interior decorator obtains information on the particular preferences of the client. Using this information. the interior decorator attempts to design a color combination which will be pleasing to the client.

one observation of the present invention is that the color expert, such as the interior decorator, works within a set of general principles of color design, however the skills of the color expert are more artistic than technical. In other words, any set of rules is incomplete and the artistic skills of the color expert are required to design a well-coordinated atmosphere. These artistic skills are difficult, if not impossible, to reduce to a set of rules necessaryr such as would be necessary to develop a color selection system based on a heuristic model.

AN ASSOCIATIVE MEMORY SYSTEM FOR COLOR SELECTION Therefore, the present invention proposes a color selection system using an associative memory network to aid in the color selection process. An associative memory network does not require the establishment of a set of rules; rather. the network is trained with a set of "good" solutions and acquires the ability to act in a manner as If it knew a set of rules.

This is similar to the manner a human color expert typically acquires information and acts to develop new color combinations, i.e., the human color expert observes and is able to construct color combinations which are pleasing, but often is unable to state specific rules or reasons for selection or approval of color combinations.

As one alternative to the preferred embodiment of the present invention, it may be desirable to implement a color selection system which uses features of a rule-based expert system in combination with the features of an associative memory system. For example. the system may first apply a set of rules and, after applying the rules utilize a neural network for selection of color combinations conforming to the set of rules. Alternatively# the system may first derive a color combination and, then, apply a set of rules to ensure compliance of the proposed color combination with -23the established rules. By way of example, a rule may be enforced which requires a certain level of contrast between text elements and background elements. Importantly. use of an associative memory system, either alone or In combination with a rule-based system, In the color selection process offers significant advantages over the prior art system using only rule- based color selection systems.

The system of the present invention and the specific application of this system to a computer Interface will be described in more detail below.

APELICATION OF A COLOR SELECTION SYSTEM TO THE MACINTOSH COMPUTER INTERFACE The preferred embodiment of the present invention is implemented on the Macintosh II computer for purposes of aiding users in the selection of colors for elements in the Macintosh computer interface (the Macintosh computer interface is commonly referred to as the "desktop").

It is, of course, obvious that the present invention has application in interfaces of alternative computer systems, as well as other alternative applications In computer systems generally, such as selection of colors for graphics designs developed on a color computer system. The present invention has further use in other color selection -24 applications. Interior decorating, make-up and wardrobe color selection are examples of such alternative uses.

The Color Selection Process of the Preferred Embodiment The interface of the preferred embodiment is shown with reference to Figure 2. As will be familiar to-many users of the Macintosh computer, the Macintosh interface comprises a number of elements such as the desktop 201. menu bar background 202, menu bar text 203, menu text 204, menu background 205, window text 206, window hilite 207, window bar 208, thumb 209 and scroll bar 210. In a color interface each of these elements may be assigned a different color. (Of course, a particular interface may also assign one or more of the elements the same color.) It should be noted that the Macintosh interface includes other elements; the preferred embodiment of the present invention allows assignment of colors to the above-specified elements, alternative embodiments may allow assignments to the full set of elements or alternative subsets.

In the system of the preferred embodiment, a particular screen may have more than one menu option in the menu bar text 203 and, consequently. more than one set of menu text 204 and menu background area 205. In the implementation of the preferred embodiment each of the options on the menu bar text 203 would appear in the same color; each of sets of menu text 204 would appear in the same color; and each of the menu background areas 205 would appear in the same color. Likewise, there may be multiple windows having window text areas 206, window hilites 207, window bars 208, scroll bars 210 and thumbs 209. These areas of each window would similarly appear in the same color from window to window in the preferred embodiment.

Figures 3 through 8 are flow diagrams illustrating the use of the color selection process of the preferred embodiment.

Figure 3 illustrates the method of the preferred embodiment for accessing the color selection system. First, a user determines it is desirable tochange the color interface of the computer, block 301. The user "launches" the color selection application, block 302. Launching of an application is generally accomplished be selecting an icon on the desktop corresponding to the application to be launched, or started, and "double-clicking" on the selected icon. ("Clicking" Is a term familiar to Macintosh users and is typically associated with the use of a mouse as a pointing device. To click on an area of a display, the mouse is used to move the cursor to the desired area of the display and the -26button on the mouse is depressed and released. Double-clicking involves depressing and releasing the button on the mouse two times in rapid succession.) As will be familiar to a user of the Macintosh computer, a menu bar is available along the top of the Macintosh screen. One alternative of to the preferred embodiment for launching the color selection application would allow the user to select a function from the menu bar by pointing to the menu bar item with a pointing device, such as a mouse. The user then selects the color selection function from the selected menu by pulling down the pointing device until the color selection function is highlighted and then releases the pointing device (e.g. the button on the mouse).

Responsive to the selection of the color selection system, a screen such as the screen shown in Figure 2 appears on the display, block 303. The screen shows the standard Macintosh interface with label boxes pointing to each of the elements which may have their colors changed.

The screen also shows a color palette 221 and a set of function buttons 220 which may be selected by the user. Each of the function buttons 221 will be described in more detail below. The color palette 221 comprises a 16x16 square of colors (a total of 256 colors).

After the color selection screen appears. the user may select any of the functions available in the color selection process, block 304.

Figure 4 illustrates the process of the preferred embodiment for selection of colors. The user first determines it is desired to change the colors of elements on the Interface. block 401.

As discussed previously. the color selection process of the preferred embodiment is designed to assist user in the color selection process by considering the user's preferences and suggesting color combinations based on those preferences. Therefore, the user first provides information to the system on the user's preferences.

To accomplish providing this input, the user selects an element, block 402. The selected element may be any one of the ten elements on the Macintosh interface, i.e., the desktop 201, menu bar background 202, menu bar text 203, menu text 204, menu background 205, window text 206, window hilite 207. window bar 208, thumb 209 and scroll bar 210.

The user selects the desired element by pointing to the label associated with the element with the pointing device and clicking on the label.

After an element is selected, the user selects a color for the element from the color palette 221. block 403. In -28the preferred embodiment, the user directs the pointing device to the color palette 221 and uses the pointing device to select a color by pointing to the desired color and clicking on it. Alternatively, embodiments of the present invention may be developed which allow for the cursor to be automatically placed in the color palette area 221 after the user selects an element.

After a color is selected, the element on the display screen appears in that color. In this way, the user can receive immediate feedback on how the selected colors interact and make changes to the selected colors until the desired combination is achieved.

After selecting the color for a first element, the user may select colors for additional elements, branch 404.

In this way, the user may provide as much, or as little, information on the preferred color scheme as desired.

After the user has selected colors for desired elements, branch 405, the user may select the "suggest colors" function by pointing to and clicking on the "Suggest Colors" button. The system, using the preferred embodiments implementation of a neural network, then suggests colors for each element which the user had not previously selected, block 406.

The "suggest colors" function presents the user's selected color scheme to the network as inputs. The network assumes the selected elements are not to have their colors changed. The network produces, as output. ten colors corresponding to each of the ten elements and presents the color scheme to the user on the display screen. The ten colors include the colors selected by the user, as well as colors determined by the network to be good "matches" for the selected colors. The determination of a good match is based on the networks previous training. The training of the network of the preferred embodiment will be described In more detail below.

After being presented with the networks suggested colors, the user may select elements and change the color scheme, block 407. The user accomplishes this by selecting elements and colors as described in connection with blocks 402 and 403.

The user may lock and unlock color choices by selecting the "lock colors" function, block 408. To select the "lock colors" function, the user points to and clicks on the "Lock Colors" button. Responsive to selecting the lock colors function, the user is presented with a set of checkboxes corresponding to each of the element in the interface. All elements which were previously explicitly -1 -30selected (by the process of blocks 402 and 403) will appear as being checked off. All elements whose colors were selected by the neural network will not be checked off. The user can check any element which is not checked off and can uncheck any element which is checked off by clicking on the appropriate box. (The checkboxes act as toggle switches, toggling between being checked and being unchecked.) There are, of course, a number of alternative methods of allowing the user to selectively lock and unlock color choices. One alternative would utilize visual indicators located either near or within each of the labels on the desktop. An example of a visual indicator would be a padlock icon. The visual indicator would then indicate the the user whether the color for an element is locked (for example, a the padlock may be highlighted when the color is locked). The user may toggle the element between being locked and unlocked by clicking on the visual indicator.

The user may then again select the "suggest colors" function and the system will suggest colors for all unlocked elements, block 409.

After a desirable color combination is achieved through the process of selecting and locking colors and utilizing the "suggest colors" function, the color -31combination may be saved by pointing to and clicking on the "Save" button. The current color combination will then be saved and used as the color combination for the Macintosh Interface after exiting the color selection process.

The preferred embodiment of the present invention includes several other function in the color selection process which the user of the process may find useful.

Figure 5 illustrates a method of the preferred embodiment utilized when the user wishes to set colors of the interface to standard monochrome values, block 501.

The user first selects the "monochrome" function by pointing to and clicking on the "Monochrome" button, block 502. Responsive to selecting the "monochrome" function, the Macintosh Interface is reset to standard monochrome values, block 503.

Figure 6 illustrates the preferred embodiments method for allowing a user who wishes to review the designer color sets used for the training the network to examine each of the exemplar color sets, block 601. The user may select the "designer color set" function, again, by pointing to and clicking on the "Designer Color Set" button. The system then allows the user to cycle through each of the color sets used for training the network, block 603. The -32development and selection of.color sets for training the network will be discussed in more detail below.

It is worth noting that the "save" function may be utilized after reviewing any of the designer color sets to save a particular designer color set to be used as the new color set for the Macintosh interface. The "save" function may also be used to save the monochrome color set as the color set for the interface. Further, any of the designer color sets or the monochrome color set may be modified by locking and unlocking elements and selecting new colors for elements as described in connection with Figure 4. The "suggest colors11 function may then be used to develop a new color set.

Referring now to Figure 7, after changing and examining new color sets, the user may wish to revert to the most recently saved color set, block 701. To accomplish this, the user points to and clicks on the "Revert" button, block 702. Responsive to clicking on the "Revert" button, the system reverts Image of interface to the most recently saved color set.

It will be obvious to one of ordinary skill in the art that the aboverecited set of features is not meant to be limiting on the scope of the present invention. Likewise, other features may be added to an embodiment of the present -33invention. For example, the currently preferred embodiment of the present invention provides the user with capability of saving one color set. A feature may be added which allows the user to save multiple color sets and to later select from the saved color sets for use and modification. A training function may be added which allows the user to design new color sets to be used to further train the network. Custom designer sets may be made available; for example, designer sets may be designed and commercially sold by "name" designers. The purchaser of such custom designer sets would be able to design color sets which would be influenced by the "name" designer's preferences. The system of the preferred embodiment is designed to read, from an external file, the values for strengths of connections. This aspect of the preferred embodiment will act to support the use of "name" designer's color sets.

THE ASSOCIATIVE MEMORY SYSTEM OF THE PREFERRED EMBODIMENT The design of the associative memory system for the present invention Involved evaluation of a number of associative memory models. After a study of known associative memory systems, it was determined that known systems would not offer a satisfactory solution to the problems posed by the desired color selection system.

The system of the present invention involves a large number of possible combinations. First, there exists a large number of possible colors from which to choose particular colors. The computer system of the preferred embodiment computationally supports 248 different colors. Second, any one of these colors may be applied to any one of a number of elements. In the preferred embodiment, there are 10 elements which may be assigned colors. Therefore, there are a total of 2480 different combinations from which the user could choose. Even in the preferred embodiment, in which the user is limited to 256 (28) possible color choices, there are 280 possible combinations.

A successful implementation of a color selection system should be able to assist with color combinations which the system has not been trained on. That is to say, a color selection system which merely recalled color combinations from a memory would not be practical given the large number of possible combinations. The trait of being able to generalize from a set of exemplars in order to provide a solution to a specific situation is termed "generalizationn.

Generalization is known in auto-associative networks. In general, autoassociative networks, like many networks, -35 are trained with a set of training examples ("exemplars"). Auto- associative networks store memory patterns (based on exemplars) in multiple point attractors and are able to generalize by clustering similar patt erns. (Instead of creating different patterns for each pattern of a class, the network creates a single attractor for the "average" or "prototype" of the patterns.) It is desired to develop a system which allows a user to select a new, unknown color and for the network to generalize its knowledge of the numerical relationship between "matching" colors to determine proper usage of the new color. An implementation of a network with this capability requires the network be trained not only to create multiple point attractors, but also to train the "landscape" around the attractor basins. This type of generalization is known in systems which may be described as feed-forward networks using a single basin. However, this type of generalization is not known in auto- associative networks.

Another difficulty arises from the fact that the colors used in the present invention are represented by a set of continuous values; in the preferred embodiment. colors are represented by RGB (red-green-blue) values. Each RGB value is represented by three 16-bit integers, one -36for red, one for green and one for blue. Typical auto-associative models store only binary patterns. An implementation which used binary encoded representation of the continuous RGB values would be prohibitive because of 5 the number of input and output units required.

Descrintion of the Network of the Preferred Embodiment Therefore, a network is disclosed which may be described as a two-layer back propagation network with recurrent links added to model an asymmetric continuous auto-associative memory.

Figure 8 illustrates the basic configuration of the color selection network of the present invention. A set of ten input nodes and ten output nodes are used to represent colors of the ten interface elements of the Macintosh display.

Figure 8 illustrates four of the input nodes, desktop 801, menu text 802, scroll bar 803 and thumb 804, as well as four of the output nodes. desktop 805, menu text 806, scroll bar 807 and thumb 808. It will be obvious to one of ordinary skill in the art how the illustration of Figure 8 showing four nodes extends to the implementation of the preferred embodiment having ten nodes.

Each of the Input nodes, (e.g., nodes 801, 802, 803 and 804) are connected to each of the output nodes (e.g.t -37nodes 805, 806, 807 and 808); except each input node is not connected to the output node corresponding to itself. By way of example, input node desktop 801 is connected with output nodes menu text 806, scroll bar 807 and thumb 808. Input node desktop 801 Is not connected with output node desktop 805. Therefore, In the preferred embodiment, each input node is connected to nine output nodes. The resulting network can be thought of as a completed connected ten-node Hopfield network, except the network includes asymmetric interconnections. (It is worth noting the lack of connection from an input unit to its corresponding output unit ensures the network does not simply learn to copy an input onto an output directly; rather, the network must learn to indirectly encode the mapping through the other interface elements.) The network also includes recurrent connections from each output node to its corresponding input node. For example, output node desktop 805 is connected with Input node desktop 801. This connection allows the state of the output node to be copied back to the input node during the relaxation process (the relaxation process will be described in more detail below).

Each of the input-output node sets shown in Figure 8 actually corresponds to a sub-network. Figure 9 is 1 illustrative of the sub-network connection between Input node menu text 802 of Figure 8 (shown as input node 910 of Figure 9) and output node scroll bar 807 (shown as output node 912 of Figure g). This connection comprises three i.nput units 910 (i.e., red 901, green 902 and blue 903), three output units 912 (i.e., red 904, green 905 and blue 906), three hidden units 911 and eighteen weighted links.

nine completely connecting the three input units 910 to the hidden units 911 and nine completely connecting the hidden units 911 to the output units 912.

The network of the preferred embodiment, therefore, comprises a total of 30 input units and 30 output units.

As has been described, each of the 30 input units and 30 output units are grouped in sets of three units.

Therefore, there are 10 groups of input units corresponding to the 10 screen elements. Each of the 10 groups of input elements are indirectly connected to 9 of the groups of output units, for a total of 90 connection of groups of input units to groups of output units. The groups of input units are connected to output units through groups of three hidden units. Therefore, the network of the preferred embodiment further comprises 270 (903) hidden units.

Specifically, each of the three input units in each input group is connected with each of the three hidden units 39 associated with that group. Each of the three hidden units Is connected with each of the three output units associated with the hidden units. In totali the system comprises 1620 forward links or connections (90 connections from input groups to output groups times; each of these connections comprising 9 connections from Input group to hidden group plus 9 connections from hidden group to output group. i.e, 90(9+9) = 9018 = 1620).

As discussed previously, the system further comprises 10 recurrent connections from output color nodes to their corresponding input color nodes. Each of the recurrent connections actually connects the red node of the output to the red node of the corresponding input, the green node of the output to the green node of the corresponding Input and the blue node of the output to the blue node of the corresponding input. i.e., a total of 3 times 10 or 30 connections.

it is important to note that, although the preferred embodiment of the present Invention is Implemented using an RGB representation of color. alternative embodiment may be implemented using other representation systems for colors. For example. common systems include HSV (hue. saturation and value) and CMY (cyan, magenta and yellow). The RGB coding system was chosen for the preferred embodiment -40because RGB encoding is the standard for the Macintosh II hardware and software. It is also worth noting that the network of the present invention has application outside of the color selection field; a network of this type may be useful in many applications where an "expert" in the field has difficulty formulating precise rules but readily renders opinions on what is "good".

Training of the NetwQrk The network of the present invention is trained using color sets designed by Ocolor experts". In particular, the system of the preferred embodiment was trained with 36 color sets designed by 5 different color experts. Back propagation was used to train the network with the color sets provided by the "experts". The training patterns were used as input patterns and as target patterns at the output layer.

of course, alternative color sets may be used to train the network. An example of alternative color sets was described above in connection with the discussion of custom designer sets designed by "name" designers. Relaxation of thg Network The network of the preferred embodiment utilizes a method termed "back activity relaxation" for relaxation of the network. (Relaxation may be thought of as the process Z of allowing the network to search for a set of well-matched colors. i.e, a solution with the lowest energy level. The term energy is often used in neural network systems to describe the level of error between the desired or target 5 output and the actual output.) The back activity relaxation algorithm adjusts the weight of linkages, searching for a set of weights which generates the minimum mean squared- error over the training set where the error is defined as the difference between the output value and target value. The underlying theory is that the least squared-error analysis produces a state where the color of each element best matches the colors of the other elements. Essentially, the network may be viewed as ten separate sub-networks, one for each element; each of the sub-networks being trained to learn the relationship between the color of its element and the color of the nine other elements. Through learning the numerical relationship of these colors, the network can respond to colors which did not appear in the training sets.

The prior art includes a back activity propagation technique which is characterized as a learning algorithm for reducing the energy of a network by relaxing the weights of the connections between nodes. Typically, a back propagation algorithm takes the partial derivative of 1 -42the global energy of the network against the local weight of a connection.

The present invention proposes a back activity relaxation technique for taking the partial derivative of energy with respect to activity of a connection, Instead of weight.

The back activity propagation algorithm is described in more detail with reference to_Appendix I.

Thus, a network is described which is capable of storing color patterns which are coded with continuous values and which is capable of learning relationships between colors of interface elements. The network comprises a recurrent network with back propagation for learning and back activity relaxation for deriving solutions. The network of the preferred embodiment may be used to model an asymmetric auto-associative memory capable of storing continuous values. Capability for modeling such an auto-associative memory is one novel aspect of the present invention.

In the present invention, one important aspect is the fact that any given input unit does not connect directly, or through hidden units, to its corresponding output unit.

It is theorized that allowing connection of an Input unit 1 -43to its corresponding output unit would lead to the networking learning to copy the input onto the output.

Thus, an apparatus and method for color selection is described. Although the present invention has been described with specific reference to a number of details of the preferred embodiment, it will be obvious that a number of modifications and variations may be employed without departure from the scope and spirit of the present invention. Accordingly, all such variations and modifications are included within the intended scope of the invention as defined by the following claims.

4 1 4 if- APPENDIX 1 Rack ActiVity Rglaxation Algorithm Specifically, for any given input pattern, define the energy as the mean. squared-error between target and output activation.

ú=17 2 (tj - ajr J Ut cii be the value of unit i in a lower layer. We would like to have a measure of how much the value of a unit needs to he changed.

Aa. -, as sai where n h an adjustment rate.

Define we can derive neij = E ailvii d OE =E 6-, t-neli = 1: --' sci,, L9ai J Snetj bai J baj Ondj With the sigmold logistic function.

sai =as('-aj) bnctj For output units ar = - Cj - as For hidden units as O.E bnttk aj E bnttk 6% where & is just V!-fi- of uniti, from the next layer. The network back P7Ppag V9 from the Output units and each input tmit. receives a gradient from its outgoing hub - The above derivation is Only for a standard feed fmwzrd net. The color net also has recurrent Uks connected from the output layer bad to the input layer. in order to simulate an auto-wwciLtive network. An output node and its corresponding input node actually represent the sum color and the in. put value is also the target activation for the output unit. Since each color is only connected to its neighhors, the gradient back propagated to a unit did not take the error incurred by the unit per me into consideration. We need to add a second term to the partial derivative as = II(E6k1Ujk - (ai - VJ) sai & where pi denotes the activation value d the output unit corresponding to the input unit..

Thus each input uniti adjusts its value by Aaj, and the relax&tion process is repeated until the activation state reaches an equilibrium point.

1.

Claims

A5 CLAIMS

1. A neural network comprising: at least one input element, said input element comprising a first plurality of nodes; at least one output element, said output element comprising a second plurality of nodes; at least one set of hidden nodes, said first plurality of nodes coupled to said hidden elements, said hidden elements coupled to said second plurality of nodes.

2. The neural network as recited by Claim 1 wherein said first plurality of nodes comprises a set of nodes corresponding to elements of a color encoding scheme.

3. The neural network as recited by Claim 2 wherein said color encoding scheme is a red-green-blue color encoding scheme and said first plurality of nodes comprising three nodes, a first of said three nodes corresponding to red, a second of said three nodes corresponding to green and a third of said three nodes corresponding to blue.

1

4 4 4G 4. A neural network for color selection comprising:

(a) a first input element comprising a first plurality of input nodes; a second input element comprising a second plurality of input nodes; (c) a first output element comprising a first plurality o f output nodes; (d) a second output element comprising a second plurality of output nodes; said first input element coupled with said second output element; said second input element coupled with said first input element.

5. A neural network for color selection substantially as herein before described with reference to the accompanying drawings.

Published 1990atThe Patent Offtce,Sta.sHouse. 66,71 High Holborn, London WC1R4TP.Further copies maybe obtalnedfrorn The PatentOfftce. Sales Branch, St Mary Cray, Orpington. Kent BR5 3RD. Printed by Multiplex techniques ltd, St Mary Cray. Kent, Con. 1187