WO2001033410A2

WO2001033410A2 - Segment-based self-learning method and system

Info

Publication number: WO2001033410A2
Application number: PCT/US2000/041871
Authority: WO
Inventors: Lee G. Cooper; Giovanni Giuffrida; Eric Bradlow
Original assignee: Strategic Data Corp.
Priority date: 1999-11-04
Filing date: 2000-11-03
Publication date: 2001-05-10
Also published as: AU3438101A; WO2001033410A8

Abstract

Segment-based, self-learning systems and methods that allow the formation and continued refinement of consumer segment profiles that originate from demographic data and provide recommendations to consumers on content offerings, advertisements, products for sale, promotions, etc. In one embodiment, the demographic data is U.S. census data, wherein zip codes can be classified into one of sixty-eight predefined demographic segments. The segments can be divided into sub-segments based upon age, gender, product category, or other data that can include profession, family size, income, etc. From the segments, knowledge rules can be formed and refined by identifying consumers by segment, tracking associated purchasing behaviors, and continually adapting the knowledge rules by segments to reflect consumer behavior. In one embodiment, probability measures can be computed and ordered according to segment and/or sub-segment to determine those recommendations likely to be accepted by consumers within a respective segment.

Description

SEGMENT-B ASED SELF-LEARNING METHOD AND SYSTEM BACKGROUND OF THE INVENTION

Field of the Invention The present invention relates generally to electronic commerce, and more particularly to consumer profiling in electronic commerce.

(2) Description of the Prior Art

Electronic commerce, otherwise known as "e-commerce", is known as the industry wherein consumers or other entities access the communications network known as the internet to procure products and services. The increasing popularity of the internet, computers, and other electronic devices is responsible for a dramatic change in the manners that people store, access, manipulate, and exchange information and other items, including traditional goods and services.

In e-commerce, internet users are typically regularly presented with a variety of displays and other information that encourage the purchase of goods, services, information, etc. For example, users can be presented with "banner" advertisements that occupy a selected portion of the display, on-line offers, sound clips, etc. It is accepted and understood that users are more likely to accept offers if the offers are targeted to the user and/or to the user's circumstances, than offers that are randomly generated with respect to the users. For example, presenting a banner advertisement for a yacht to a low-income user or a user that remains in a landlocked area, is not expected to have a large expectation of success.

Conventional methods of targeting offers to particular users are not always successful. Different methods have had limited success, and it is anticipated that this limited success may be attributed to the fact that these methods require extensive information about each customer's preferences before targeting can begin, they often segment customers based on the generic demographic information that is directly applied to select advertisements without further relation to the website, they target expected customers to a given website rather than actual customers, and/or these methods compromise consumer privacy and client data security.

One conventional method uses "cookies", or data stored on a user's computer to allow a website operator to identify a user when the user revisits a particular website. The cookies allow the website operator to use the user's past internet history to predict the likelihood of success of a particular offer to the user. Unfortunately, while traditional cookies can benefit customers by remembering their name, mailing address, and opt-in interests, some personalization companies, such as ad servers, place "profiling cookies" that share the user's personal navigation and purchase history across many sites and even link back to offline shopping databases. These cookies are considered an invasion of privacy by most users, and websites that utilize cookies are receiving considerable negative press. Additionally, users are becoming more sophisticated and are often following simple instructions to disable the user of cookies. Other targeting methods include using generic information such as the season and targeting according to the reported, aggregate demographic profile of the website on which the advertisement or offer is displayed. This method will only be useful for sites with extreme niche demographics, especially as the internet population becomes increasingly diverse. Given the wide audience for most sites popular enough to draw advertisers, a single average or aggregate demographic profile may not adequately reflect the diversity of visitors.

Collaborative filters are algorithmic techniques inferring the tastes of customers from past purchases. These models are based on the premise that customers should be grouped by purchase patterns rather than demographic or lifestyle characteristics; therefore, a customer may be compared to other customers that have bought similar products, as buying groups are generally created and defined by purchases. The purchases of other members in the buying group typically determine new offers. For instance, a model may find that a group of people bought products X, Y, and Z. Any new customer buying both X and Y receives the recommendation to buy product Z. Collaborative filters need extensive data about each customer to gain a clear picture of that customer's preferences, typically over ten interactions. Since the percentage of customers that actually buy or interact repeatedly with a particular website is very low relative to total browsers or even one-time buyers, the opportunity to accurately target "early visitors" is often lost as collaborative filtering benefits the few, repeat customers. Also, product categories with hard to define tastes would require many purchases for a usable pattern to emerge. Additionally, since batch updates to analysis can take hours or days, collaborative filters miss opportunities to optimize targeted offers in the interim. This delay can be particularly problematic for highly dynamic product categories, such as trendy music, fashions, etc., and new products that experience a market surge.

Neural networks may also be used to target offers, wherein neural networks apply rules that are learned through iterative processes. These systems, often called "black boxes" because their processes cannot be observed, were designed to work best in environments with minimal intervention; however, online marketers must be able to observe offers, understand which offers are effective, and contribute their expertise to the marketing process. Neural networks also do not provide insight into customer demographic characteristics. Just as with collaborative filters, neural networks require significant processing with batch analysis cannot satisfy a need for real-time sales opportunities.

There is a need for increased performance methods and systems for providing recommendations to consumers.

SUMMARY OF THE INVENTION

The methods and systems of the present invention can increase the likelihood of success in providing recommendations to consumers. The methods and systems disclosed herein can be implemented on a communications system such as the internet, wherein the methods and systems can use segmentation schemes to determine communications system user preferences from an array of preference options. In the disclosed methods and systems, a segmentation scheme can include an algorithm or other scheme to generate immediately identifiable groups from a customer set. In one embodiment, the customer set can be customers of a particular internet website, otherwise refened to herein as users, consumers, or visitors to the website. In an embodiment, the segmentation scheme can provide a segmentation of the customer set based on demographics.

The disclosed methods and systems can recommend offers, advertisements, products for sale, content offerings, promotions, etc., hereinafter refened to collectively as recommendations, to consumers by using the segmentation scheme to establish a plurality of consumer or user segments. In one embodiment, the methods and systems can generate, for each segment based on data collected from the website, a frequency of offer acceptance or rejection for a plurality of recommendations. The methods and systems can also determine a segment of a particular consumer, user, or visitor to the website, and present to the particular consumer one or more recommendations having a higher frequency of acceptance with the particular consumer's segment on the website, as compared to other potential recommendations presented on the website to the particular consumer's segment. The disclosed methods and systems can further incorporate a learning algorithm or technique with the segmentation scheme, wherein learning can be a dynamic updating of data based upon new information, analysis, and/or synthesis. In an embodiment, a learning algorithm or technique can capture empirical evidence indicating an increase or decrease in the likelihood that members of a particular segment visiting the website may have a preference for a particular item, service, etc., and therefore such learning algorithm can correspondingly increase or decrease the likelihood that a recommendation is made to members of that particular segment on the website. In one embodiment, the learning algorithm can be implemented for real-time application, although other update rates can be utilized. In an embodiment, multiple learning algorithms can be used to update data upon different criteria. In one embodiment wherein multiple learning algorithms are used, information can be shared or otherwise combined from the different learning algorithms.

In one embodiment, a learning algorithm can identify those recommendations for a particular segment to the website that are statistically stable when compared with other recommendations for that segment on the website, wherein statistical stability can be evaluated by computing a stability of probability measures that are associated with recommendations of a given segment. Targeted recommendations on the website for a segment can be derived from the statistically stable recommendations.

In an embodiment, the learning algorithm can provide a mechanism to determine whether a new recommendation belongs among the statistically stable group of recommendations for a segment.

In an embodiment, opportunities to provide targeted recommendations can cause a targeted recommendation selection process that can weigh a recommendation's statistical measures against business constraints that can include factors such as contractual obligations related to recommendations (e.g., advertising campaigns versus estimated numbers of consumers), profitability based on providing a particular recommendation, profitability of success of a particular recommendation, inventory limitations, age- appropriateness, etc. In one embodiment, context filters can be used to identify appropriate recommendations for a recommendation opportunity, while in another embodiment, other filtering parameters that can be specified by application, can be applied to the recommendation selection logic within a segment. In some embodiments, the methods and systems disclosed herein can target recommendations in contexts other than electronic commerce, and can include methods and systems for providing recommendations via direct mail or other advertising media. In an embodiment, the methods and systems disclosed herein can base the segmentation schemes on demographic data. In one embodiment, demographic and life- style information can be a basis of segmentation, wherein the initial basis of segmentation can be a source that is independent of the application (e.g., internet), and can include, for example, census data. In one embodiment, segmentation schemes can be derived by analyzing demographic data according to ZIP code. The methods and systems also disclose a mechanism to augment the initial basis for segmentation using information from the application to which the methods and systems herein are applied. For example, in an embodiment wherein the initial basis for segmentation is the U.S. census and the application is a particular website on the internet, information from the website such as age, gender, etc., of the website user, can be utilized to augment the U.S. census segmentation. Continuing the internet example, the disclosed methods and systems can allow a basis for segmentation to be further developed by user actions on the website, wherein such actions represent consumer preference. In an internet embodiment, preferences of a particular user can be revealed by advertisements that the user selects (i.e., "click-throughs"), purchase information related to recommended or non-recommended products, positive or negative feedback relating to recommended or non-recommended products, or other measurable indicators.

In one embodiment, the segmentation and learning schemes can be implemented within a system that provides segment members with a wide range of preferences and offerings for recommendations including advertisements, content, items for sale, affiliate offerings (i.e., cross-sell), etc., or combinations thereof. The methods and systems herein can be applied to any media wherein options can be viewed and selected, including, for example, computers in networked or stand-alone configurations, set-top boxes, mobile phones, handheld devices, personal digital assistants (PDAs), etc.

The disclosed methods and systems can combine demographic and geographic information as a basis for geodemographic segmentation. The methods and systems disclosed herein can incorporate the segmentation and learning schemes into a remotely controlled, secure, residential real-time data mining agent, wherein the data mining agent can automate the detection of patterns in a database and assist in understanding and predicting customer behavior. For example, a data mining agent can determine those customers most likely to discontinue a service, purchase merchandise or services, and/or respond to a particular offer. The disclosed data mining agent can also incorporate the notion of campaign management that includes the managing and monitoring of customer communications across all channels.

The methods and systems disclosed herein can permit an Application Service Provider (ASP) to interact with an ASP-provided database that resides at a remote location such as a merchant's server. The ASP can maintain and update the database by accessing the merchant's system via a computer network such as the internet. In an embodiment, the methods and systems are secure from the subscriber or merchant's perspective and do not affect proprietary databases within the merchant's location. Additionally, the ASP- provided database to be updated can reside behind a firewall, or on a system that is part of a virtual private network (VPN), etc.

The methods and systems disclosed herein can mine or model information that connects historic data to purchasing information or other desired outcomes, to store the rules or parameters from such model to enable real-time access, to identify as quickly as real-time, knowledge to apply to a particular situation, and similarly to generate a response. For example, in one embodiment, an internet user can visit a particular website, wherein the user can be identified as part of a particular group for which data exists in a database, the group data can be retrieved and integrated with specific user data to generate an advertisement that is provided on the user's display.

The systems and methods herein can be deployed as part of incentive programs, wherein a recommendation such as a particular preference program that includes options such as percent savings, frequent flier miles, free long distance minutes, prizes, etc., can be deemed more appropriate than another preference program, based upon the data generated by the systems and methods herein. In yet another embodiment, the systems and methods can be utilized to customize the layout of a particular consumer's viewed webpage based upon preferences, and additionally, the content of a consumer's viewed webpage can be coordinated with the preferences (e.g., news, sports, entertainment, etc.).

In one embodiment of the invention, the disclosed systems and methods can be utilized to create communities of users having similar interests. For example, chat groups can be identified based upon a user's profile and/or preferences. Such content or suggestions can be supplied in the form of a hypertext link to another webpage or internet location.

In one embodiment, the systems and methods herein can be implemented in a non- real-time scenario, wherein a subscriber can supply user data and activity information to a remote location for analysis. In such an embodiment, the remote location can return segmentation data, behavioral rules, or other data upon which users can be categorized. The methods and systems disclosed herein can also provide reports regarding recommendations, success or failure of recommendations, segments, segment profiles, segment statistics, etc.

Other objects and advantages of the invention will become obvious hereinafter in the specification and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS A more complete understanding of the invention and many of the attendant advantages thereto will be readily appreciated as the same becomes better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings, wherein like reference numerals refer to like parts and wherein:

FIG. 1 is a block diagram of an illustrative system that incorporates the principles of the invention disclosed herein, as applied to an embodiment for generating advertisements for internet pages;

FIG. 2 is a block diagram of a process to achieve demographic segments for the system of FIG. 1;

FIG. 3 shows a methodology to further refine the segments presented by FIG. 2; FIG. 4 shows stacks associated with the segments illustrated in FIG. 3, for an embodiment wherein the recommendations include advertisements only;

FIG. 5 is a block diagram illustrating an application of weights to update elements in a FIG. 4 stack; and, FIG. 6 is a block diagram illustrating a process wherein the stacks of FIG. 4 can be updated with a learning algorithm that incorporates a stack ordering algorithm.

DESCRIPTION OF ILLUSTRATED EMBODIMENTS To provide an overall understanding of the invention, certain illustrative embodiments will now be described; however, it will be understood by one of ordinary skill in the art that the methods and systems described herein can be adapted and modified to provide methods and systems for other suitable applications and that other additions and modifications can be made to the invention without departing from the scope hereof.

It shall be understood within the context of this disclosure that the term segments, when used in the context of forming segments, generating segments, etc., can otherwise be understood as forming a group or associating items, people, consumers, etc.

Referring now to FIG. 1, there is shown an illustrative system 10 that incorporates the principles of the invention. The FIG. 1 system 10 represents an internet application, however the invention herein is not limited to the internet, and can be applied to any application wherein segmentation and preference learning is desired. For example, the present invention can be applied to determine chat groups, assist in determining targets for direct mail or other traditional advertising campaigns including telephone solicitations, etc; however, in the system 10 of the FIG. 1 embodiment, the invention is applied to the internet 12 for generation of recommendations for display on web page, wherein an internet user 14 accesses the internet 12 using one of many well-known internet access devices. In the FIG.

1 system 10, the internet user 14 utilizes a personal computer (PC), however the PC can be easily interchanged by a SUN Workstation, laptop, handheld, or palmtop computer, telephone, personal digital assistant (PDA), television, or any other device that allows the display and selection of information as required by the disclosure herein. In the illustrated system 10, the PC can be equipped with an application that assists a PC user in accessing, downloading, and otherwise interacting with a web page or other information obtained through the internet 12. Similarly, the PC can be equipped with a mouse, keyboard, stylus, voice recognition software and microphone, etc., to allow the user to indicate selections. In the FIG. 1 system 10, the secure client 16 represents what shall hereinafter be refened to as a subscriber to the services provided by the systems and methods disclosed herein. In this embodiment, the subscriber 16 can be a single server or network of servers that is connected to the internet 12 via a commonly known web interface 18; and, the illustrated subscriber 16 is capable of posting information, otherwise commonly known as a web page, that is retrievable by the internet user 14 through the internet 12. Those with ordinary skill in the art will recognize that a subscriber can be any entity, individual, etc., implementing or utilizing the methods and systems disclosed herein. In the illustrative system of FIG. 1, it shall be presumed that the internet user 14 accesses a web page that is posted by or otherwise associated to the subscriber 16. The illustrated subscriber 16 further maintains a web agent 20 that supplies content to the internet 12 through the web interface 18, and similarly extracts information from the internet user 14 according to actions, selections, identity information, or otherwise collected information from the internet user 14 such as cookies, etc.. For example, the web agent can identify the internet user 14 when the internet user 14 accesses one of the subscriber's 16 web pages. Such identification can be performed using one of many well-known techniques, including cookies, and the invention herein is not limited by such information extraction technique. The web agent 20 can accordingly access a proprietary database 22 that includes information about internet users that have previously accessed the subscriber's web pages. The database 22 can include information that was previously entered by a user 14 or otherwise extracted from the user 14. The database 22 can similarly include clickstream data, information regarding previous purchases including date, identification, and price of such purchases, and any other information deemed relevant by the subscriber 16 to characterizing and otherwise segmenting internet users 14.

In the FIG. 1 system, the subscriber 16 is additionally equipped with a data mining system 24 that further includes a monitoring agent 26, a data mining engine 28, and a knowledge database 30. In the illustrated embodiment, the data mining system 24 is physically separate from the other subscriber components, and in one embodiment, the data mining system 24 can be a personal computer (PC) or other microprocessor controlled device. In such embodiment, the monitoring agent 26 and data mining engine 28 can be executable software modules, while the knowledge database 30 can be memory resident on the PC or otherwise available to the PC. In another embodiment, the data mining system 24 can be a software module resident on a PC, for example, wherein other software executable modules may also reside on the PC.

In the illustrated system 10, the monitoring agent 26 maintains the operational integrity of the data mining system 24 modules by monitoring the activities, software, connections, etc., of the various data mining system modules. Additionally, the monitoring agent 26 can receive commands and other control information from the remote administration unit 32 that can indicate to the monitoring agent 26 that the data mining system 26 should be reset, that memory or disk space should be cleansed or otherwise formatted, etc.

The illustrated data mining engine 28 also maintains a connection to the remote administration unit 32. The FIG. 1 data mining engine 28 can receive information from the web agent 20 that includes a user's identity, the user's present activity, and any historical information related to the user that can be stored and retrieved by the data mining engine 28 from the proprietary user database 22. Such historical data can include, for example, previous purchases, previous activity on the website or other website, demographic data, etc. from a third party, for example, although such examples are merely provided for illustration and not limitation. The illustrated data mining engine 28 can combine the user information from the web agent 20 with information from the knowledge database 30 to generate a recommendation for the user. In the illustrated system 10 and for the purposes of the methods and systems disclosed herein, a recommendation can herein be defined as information in the form of advertisements, products for sale, offers, promotions (e.g., coupons), content offerings, affiliate offerings (i.e., cross-selling), gift suggestions, etc., about which information can be displayed or otherwise provided (e.g., image, hypertext link, banner advertisement, etc.) on the web page that the user 14 is presently viewing. For the purposes of this disclosure, content offerings can include solicitations, links to pages or sections of books, documents, newspapers, movies, television shows, web sites, and other textual materials, although such examples are provided for illustration and not limitation.

The recommendation is therefore provided to the user for viewing through the web agent 20 and web interface 18 using well-known techniques that can include implementing CGI scripts, Perl scripts, java code, XML, HTML, etc. Additionally, the FIG. 1 data mining engine 28 can receive upgrades and updates from the remote administration unit 32.

The illustrated knowledge database 30 can be a memory that can be partitioned physically or logically and can include one or more distinct memory segments.

Additionally, the knowledge database 30 can have one or multiple databases that can hereinafter be refened to as a single database 30. In an embodiment, the database 30 can be searched and otherwise managed using well-known database management and searching techniques, and can be constructed using any well known database facilitator including SQL, MySQL, Oracle, Sybase, Informix, DB2, etc., and those with ordinary skill in the art will recognize that the invention is not limited to the database construction or management mechanism.

As will be indicated further herein, the illustrated knowledge database 30 includes learned rules and segmentation results that can be updated by the data mining engine 28 or the remote administration unit 32. The segmentation rules, learning algorithms, and updating of results shall be further described herein.

The FIG. 1 system 10 illustrates a configuration wherein the data mining system 24, or parts thereof, can be updated via commands of the remote administration unit 32, while the subscriber's remaining systems, including any proprietary databases 22, etc., can remain inaccessible to the remote administration unit 32. The illustrated subscriber 16 therefore obtains the benefit of system updates to the knowledge database 30 that can be applied to proprietary information, without sacrificing the security or integrity of proprietary data (e.g., proprietary database 22).

Referring now to FIG. 2, there is shown a block diagram providing an algorithm for developing segments from, for example, demographic data. In the segmentation illustration provided by FIG. 2, empirical data in the form of demographic data 40 is applied to a segmentation algorithm 42 to produce a set of demographic segments 44. In the illustrated system, the demographic data 40 is census data such as U.S. census data, although the invention herein is not limited to the demographic data 40 selected or the initial basis for segmentation. For example, in one embodiment, the census data 40 can be input to the illustrated segmentation algorithm 42 to characterize and categorize U.S. zip codes into one of a possible sixty-eight demographic segments 44. For example, the sixty-eight segments 44 can be further described by demographic information including mean or median values of income level, education level, ethnic background, gender, age, professional category, urban or non-urban areas, family size, home ownership, region of country, etc., and the invention herein is not limited by the use of census data, the number of segments, or the method of describing the segments (i.e., demographic attributes). The illustrated segmentation algorithm 42 therefore categorizes or segments U.S. zip codes according to a measurement of that zip code's residents with respect to the segmentation algorithm criteria. Although the FIG. 2 system indicates segmentation by demographics, segments can otherwise be formulated without departing from the scope of the invention.

Referring now to FIG. 3, there is shown a diagram indicating the subdivision of the illustrated FIG. 2 sixty-eight demographic segments 44, into sub-segments. In FIG. 3, the segment shown as "base K" 50 is illustrative of one of the sixty-eight demographic segments 44 from the system of FIG. 2, and therefore for the illustrated system, K can range from one to sixty-eight. As FIG. 3 indicates, the illustrative sixty-eight demographic segments can be further divided into sub-segments based upon age, gender, and category, although the invention herein is not limited to such sub-segmentation variables (e.g., age, gender, category, etc.), and other sub-segmentation variables can be implemented or otherwise developed without departing from the scope of the invention. The structure of FIG. 3 can also be refened to as a "lattice", wherein the lattice maintains different "nodes," and in the illustrated embodiment, the sixty-eight segments can conespond to sixty-eight lattices.

In the FIG. 3 illustration, the "category" variable is dependent upon or related to a product, service, etc. For example, if a product is compact discs (CDs), categories might include rock, pop, country, alternative, R&B, classical, contemporary, etc. Category can include subcategories and sub-subcategories, etc., thereby adding additional nodes to a lattice.

As FIG. 3 indicates, base segments can therefore be represented as a combination of the sub-segmentation categories. The FIG. 3 illustration demonstrates a "bottom-up" approach to relating the sub-segments that allows new information from one sub-segment to be transfened to other, related sub-segments. In general, in the illustrated configuration, updates to one of the sub-segments are thereafter propagated upward through the sub- segment lattice as indicated by the connecting arrows. As shown by FIG. 3, the "bottom" sub-segment 52 in the illustrated lattice, is a function of the sub-segment criteria of the nodes in the next level (i.e., 62, 54, and 64), and as the FIG. 3 lattice is traversed upwardly through the sub-segments or nodes, the sub-segment criteria are increasingly reduced until the "top" sub-segment, or base segment 50, is reached. In an embodiment, the base segment 50 can correspond, for example, to one of the sixty-eight demographic groups previously discussed.

Information in the FIG. 3 nodes is updated by traversing the lattice upwardly as indicated by the relational anows of FIG. 3, and propagating the information accordingly. For example, information related to base K 50 is updated after information in the sub- segments derivative to 50 (i.e., 56, 58, and/or 60) is updated. Additionally, in the illustrated embodiment, information in the sub-segment labeled "base K, Gender" 56 is updated after the information in the two sub-segments derivative to 56 (i.e., 62 and 54) is updated. In the illustrated systems, a user identification (ID) and a user zip code can be utilized to identify the segment and/or sub-segment of a given user. For example, if the subscriber 16 is an owner or proprietor of a web page, administrator of a web page, etc., as shown in FIG. 1, information from the user 14 in the form of a cookie, membership or registration login, shipping information, etc., can provide a information that can be related to a user ID. In an embodiment of the illustrated system, a subscriber 16 can associate users to unique user IDs, and the proprietary database 22 can be organized or otherwise accessed using the user ID to retrieve information about the user. As indicated previously, the proprietary database 22 can include demographic information, zip code information, consumer preferences, shipping preferences, historical activity on the website, etc., all of which can be provided to the data mining system 24 to identify a segment and/or sub- segment. In an embodiment, the subscriber 16 can prompt the user 14 to enter zip code information. In an embodiment, the subscriber 16 and/or the data mining system 24 can generate a cookie with a user ID and zip code information for a given user 14 of the website, and retrieve this cookie upon subsequent visits of the user 14 to the website, wherem the cookie information can be used by the subscriber 16 and similarly, by the data mining system 24. In another embodiment, the cookie generated by the subscriber 16 and/or data mining system 24 can also include the identity of the user's segment, and/or information pertaining to the segment variables (e.g., age, gender, category, etc.).

In the illustrated systems, a user zip code allows identification of a base segment, while the demographic information provides additional information to identify sub- segments and allow the lattice structure of FIG. 3, for example, to be traversed in the downward direction. In an embodiment, when a user cannot be associated with a zip code and demographic information, the user can be assigned a "default" segment that can be determined, for example, by a system administrator. In another embodiment, when the zip code for a given user is not known but demographic information is known, a base segment can be selected according to selection logic, and the demographic information may be used to identify sub-segments within the given base node. In an embodiment, if the zip code is known but there is not any demographic information for a given user, a sub-segment may not be identified. In an embodiment, when neither the zip code nor demographic information is available, the user can be assigned to a selected segment that can be specified, for example, by a system administrator using a configuration file, etc. Those with ordinary skill in the art will recognize that the determination of a user's segment and/or sub- segment can be performed in many different ways without departing from the scope of the invention.

Referring now to FIG. 4, there is shown an illustration of the information associated with the base segment and sub-segments in the illustrative system of FIG. 3, that are caused to be updated by new information in accordance with the invention herein. For the purposes of this discussion, references to a "node" shall include the base segments and all sub-segments as defined relative to the lattice illustrated in FIG. 3. The FIG. 4 illustrative node corresponds to a demographic segment that can be known as segment number fourteen, and FIG. 4 further illustrates a node therein that includes gender and age 70. As

FIG. 4 demonstrates, the node in the illustrative system is associated with six stacks, wherein in the illustrated embodiment, the stacks are dedicated to advertisements only for discussion purposes, and do not include other forms or species of recommendations that can include offers, promotions, content offerings, products etc., for displaying to users of the web page(s) of the subscriber 16. Those with ordinary skill in the art will recall that the invention is not limited to advertisements, and the stacks can similarly include all types of recommendations that can include, for example, offers, promotions, content offerings, products for sale, etc.

As FIG. 4 indicates, in the illustrated system, a node relates to a history stack 72, a momentum stack 74, a data mining stack 76, a manual stack 78, a null stack 80, and a learning stack 82. Those with ordinary skill in the art will recognize that there are other methods for associating data other than stacks, including tables, linked lists, etc., and the illustrated stacks are provided merely for illustration and discussion purposes. In the FIG. 4 system, the stacks maintain the same list of advertisements, however, as indicated by the system of FIG. 4, different stacks can apply, utilize, or maintain different learning algorithms that can cause a different ordering of the advertisements in the respective stacks for increased flexibility, depending upon system requirements. In another embodiment, different stacks within a node can have different advertisements. In some embodiments, additional or fewer stacks can be implemented. In an embodiment, information from one or multiple stacks can cause information in another or multiple stacks to be updated. For a system according to FIG. 4, the stacks can be implemented using well-known database techniques and can be implemented using Oracle, MySQL, SQL, Excel, Sybase, Informix, DB2, etc. Those with ordinary skill in the art will recognize that the stacks can be implemented using many well-known techniques such as linked-lists, queuing techniques, etc., without departing from the scope of the invention. In the illustrated system of FIG. 4, the history stack 72 maintains information based on the historical activity of the stack contents, wherein such historical activity is collected over comparatively long time periods. For example, in some embodiments, the element at the "top" of the history stack 72 can be the most popular or successful advertisement, etc., for that node during a given accounting period. For the history stack 72, the accounting period can be on the order of months or years, for example, although those with ordinary skill in the art will recognize that such an example is intended for illustration and not limitation.

Alternatively, an element at the "top" of the illustrated momentum stack 74 can indicate the advertisement that is most popular or successful within the respective node for a specified, shorter interval of time when compared to the accounting period of the history stack. The illustrated momentum stack 74 can therefore be indicative of short-term impulses in the market or short term trends, where short-term can be defined relative to the accounting period of the history stack 72, although such embodiment is merely for illustration, and different embodiments can be realized according to different system requirements without departing from the scope of the invention. For example, in the illustrated system, if the history stack 72 tracks recommendations such as advertisements, offers, products, etc. over years, the momentum stack 74 may track recommendations such as advertisements, offers, products, etc., over hours, days, weeks, or for example, months. The illustrated data mining stack 76 incorporates other factors beyond age, gender, and category into the weighting process for the recommendations. For example, the data mining stack 76 can include additional information such as profession, family size, marital status, click-stream data, etc., to order the stack elements accordingly for the node. The illustrated manual stack 78 can allow a system administrator, manager, marketing agent, etc., or other system user to create manual rules by which a stack can be ordered. The illustrated null stack 80 is a stack that does not use accumulated learning, and consequently can serve offers to a control group to monitor the ongoing effectiveness of the other stacks. The FIG. 4 learning stack 82 can provide a mechanism to allow new advertisements to be shown for gathering empirical evidence on the new advertisements while employing a learning algorithm.

In the illustrated system of FIG. 4, four of the six stacks (i.e., history, momentum, data mining, learning) learn according to specified learning algorithms, and these stacks can additionally maintain an ordering algorithm that can be implemented in conjunction with the learning algorithm. For the purposes of the discussion herein, references to the learning algorithm can be understood to include a stack ordering algorithm, wherein the learning algorithm for the illustrated embodiment shall be discussed herein with reference to FIGs. 5 and 6.

In the FIG. 4 embodiment, stacks with learning algorithms can be arranged such that the "top" item is the highest rated item according to the stack objectives, and the "bottom" item is accordingly the lowest rated item. Although terminology of top and bottom are utilized herein, such terminology is merely for explanatory purposes, and the stacks and connections thereto can be implemented using any well-known relational or queuing technique that can include pointers, linked-lists, etc. Those with ordinary skill in the art will recognize that there are many well-known techniques of implementing a priority list and indicating the highest priority item, all of which can be practiced within the scope of the invention.

Similarly, in an embodiment wherein the subscriber 16 may wish to generate multiple species of recommendations from the illustrative list of advertisements, offers, promotions, content offerings, products, etc., in one embodiment, a given stack such as the history stack 72, for example, can be subdivided into sub-stacks, wherein a sub-stack can be dedicated to advertisements, wherein another sub-stack can be dedicated to promotions, and yet another sub-stack can be dedicated to content offerings, etc., and wherein the learning algorithm for the history stack can be applied individually to the various sub-stacks, and wherein when a request is received and it is determined from the request that the recommendation be an advertisement from the history stack, for example, an element from the history advertisement sub-stack can be selected in response to the request, with other requests for other species of recommendations processed accordingly. In an alternate embodiment, a system 10 such as shown in FIG. 1 can provide multiple species of recommendations by recreating the node lattices of FIG. 3 and corresponding stacks as shown in FIG. 4 for the different species of recommendations, wherein a particular lattice conesponds to a particular species of recommendation, and requests for recommendations are appropriately forwarded by species classification, to the proper lattice. In yet another embodiment, a single lattice structure may be generated according to FIG. 3, wherein nodes may be further partitioned or otherwise understood to include areas designated to, for example, the sixty-eight base segments. Those with ordinary skill in the art will also recognize that the concept of sub-stack, multiple lattices, or single lattice are merely for illustrative purposes, and other methods of categorizing and or organizing the data and/or multiple species of recommendations to achieve the effect as indicated herein, can be implemented without departing from the scope of the invention.

As an example, if a particular system requirement uses only history stacks, but produces recommendations for species including advertisements, content offerings, and products for sale, three different stacks can respectively store the three species of recommendations, without the concept of sub-stacks or multiple lattices, wherein the three stacks can have the history stack learning algorithm and/or objectives identically and respectively applied thereto, and wherein the recommendation stack selection process includes the ability to identify the requested species of recommendation and/or the appropriate stack. In an alternate embodiment of the stated example, a single history stack, as in FIG. 4, can be implemented, wherein three sub-stacks can be generated for the different species of recommendations. In yet another embodiment, a single stack can be utilized wherein three species of pointers could relate respectively to three species of recommendations in a single stack.

Depending upon the embodiment practiced, stacks can initially be populated in many different manners according to data provided by a subscriber 16. For example, for illustrative purposes only, an embodiment such as FIG. 4 wherein the only species of recommendations is advertisements will be discussed herein. In such an embodiment, stacks within the different nodes can be initially populated using data from the subscriber's proprietary user database 22, wherein user demographics can be utilized to determine segments and nodes and initial stack orderings as indicated by previously recorded data. In another embodiment wherein data from the proprietary database 22 can be considered outdated, unreliable, or otherwise undesirable, stacks for different nodes can be populated through a process or period wherein advertisements are displayed according to existing advertisement selection criteria utilized by the subscriber 16, which can be based upon advertisement campaigns, etc. In the illustrated system, as the advertisements are shown in this manner, user information can be obtained with feedback information (i.e., user behavior resulting from the advertisements, such as clickthrough, no clickthrough, etc.), and the feedback information can be utilized to populate and update the appropriate segments and/or sub-segments, or nodes. In yet another embodiment, various advertisements may be assigned to populate the segment stacks, and the learning algorithm can be applied to properly order the advertisements.

In the website advertisement embodiment, one learning algorithm objective is to determine, from previously obtained data from the website, a probability measure that a present user visiting the website as a member of a particular segment, may accept a recommendation provided from the system of FIG. 1, wherein such recommendations can be of the form of advertisements, products for sale, content offerings, etc., and acceptance can be measured in such an embodiment, by clickthroughs, purchases, etc. Another objective of the learning algorithm in this embodiment is to form an estimate of this probability of acceptance that best approximates the actual probability of acceptance for the respective advertisements in the respective segments.

In an embodiment for presenting advertisements to a website, the learning algorithm can be implemented to collect or compile data to determine, for different segments, those advertisements that are the N most successful advertisements on the website for the respective segments, out of the M advertisements shown on the website to members of the segments. In the illustrated systems, N and M are positive non-zero integers, where N is less than M, and where N and M can vary according to application, or in some embodiments, by segment.

In one embodiment, probability measures can be associated with the advertisements for the segments, wherein the probability measures indicate the historical probability of success of the advertisement with consumers (i.e., segment members) visiting the website, and hence the probability measures can be an estimate of an actual probability of acceptance. For the purposes of the discussion herein, references to "probability measure" shall refer to the estimate of the actual probability, unless otherwise specified to be the actual probability. In an embodiment, the probability measure can be computed as the number of acceptances of an advertisement compared to the number of times the advertisement was shown, wherein the number of times that the advertisement was shown can otherwise be refened to as the number of impressions for the advertisement. In the example, the number of impressions can be tracked or otherwise recorded by the systems and methods described herein. For example, an advertisement with one impression that is successful (i.e., accepted) once, has an estimated probability of acceptance of unity from the collected empirical data. This probability measure is not "statistically significant" because, in this illustration, the sample set is not sufficient to provide confidence that the computed probability measure reflects the actual or true probability of acceptance of the advertisement within this segment. In an embodiment of the systems and methods disclosed herein, the concept of statistical significance can be related to an identification that the probability measures of the highest ranking advertisements (i.e., "top N") are generated from a statistically meaningful sample set. In the embodiments discussed herein, the concept of statistical significance can be distinct from the concept of "statistical stability", wherein statistical stability can be expressed as a condition wherein there is a statistical confidence that the identification of the top N recommendations within a segment is constant (i.e., stable), regardless of whether 5 the order of recommendations within the top N is constant. Because the top N recommendations within a segment can be determined by ranking the recommendation probability measures, the concepts of statistical significance and statistical stability are therefore related. In one embodiment, statistical significance is a necessary, but not a sufficient, condition for statistical stability. o For example, continuing the advertisement-only website display embodiment, where p _j is the observed acceptance rate of an advertisement j, p _i may be expressed as x/n,, where n, is the number of times advertisement j was provided, and x, is the number of times advertisement j was accepted. Probability measures for advertisements in a stack can be ordered, therefore, from one to M, wherein p _\ > p ₂ > ■ ■ ■ > p _u- h ^one embodiment, 5 determining whether the highest ranked N advertisements are statistically stable includes computing a probability that the advertisements not within the highest ranked N (i.e., p _N+1, P _N+2>- • •_> P _M) do not belong in the highest ranked N. For unranked advertisements, herein defined as those advertisements among the M total advertisements but not within the highest ranked N advertisements (i.e., p _N+1, p _N+2,..., p _M), a probability is computed as 0 given by equation (1).

Pr(/7_g > in(pι, pi,..., PN)) = 1 - Pr(/?_g < min(pι, pi,..., PN))

= 1 - t MPJ ~ P*) /((l I n + ll n_g)p_P(\ - p_P)) ,

(1) 5 where: p, generally represents the theoretical or true clickthrough rate of a recommendation with an observed probability measure of pi ; p_g is the observed probability measure for an unranked advertisement; p_P can be defined as the pooled estimate of the probability measures corresponding to the two advertisements being compared (i.e., corresponding to p_g and pj ), and can be further expressed as equation (2): pp = (x_g + xj) /(n_g + nj) ; and (2) ^ is the probability distribution function otherwise known as a standard normal distribution function with a mean of zero and a standard deviation of one. In an embodiment, when equation (1) is computed for unranked advertisements, and the resulting probabilities from computing equation (1) satisfy a given threshold, statistical stability can be deemed to be achieved for the segment. Because statistical significance is a necessary but not sufficient condition for statistical stability, a determination of statistical stability likewise can be interpreted as statistical significance for the top N recommendations. Those with ordinary skill in the art will recognize that the computation of statistical stability provided herein is merely for illustration and not limitation, and other methods of computing stability and accuracy of the probability measures may be utilized without departing from the scope of the invention. Additionally, although the example herein provided the computation of probability measures based on merely the number of acceptances versus the number of impressions, other methods of computing probability measures can be implemented without departing from the scope of the invention.

Depending on the embodiment and the computation of statistical stability, N may be a fixed or a variable number. For example, in a case wherein there can be fifty possible advertisements that may be shown to visitors to a subscriber website, a learning algorithm can be configured to identify the highest ranked fifteen advertisements (i.e., associated probability measures) and assess whether this set of fifteen advertisements is statistically stable. In another embodiment, the learning algorithm can be configured to identify only those highest ranked N advertisements whose probabilities of success provide statistical stability for a particular segment, wherein N can vary according to the statistics generated by the different segments.

In an another embodiment, the value of N can be determined statistically for different segments. For example, N can be determined by comparing the relative probabilities of success of an advertisement within a particular segment. Probability measures and other statistical methods can be employed to mathematically represent the distinction between the most successful N advertisements and the other advertisements.

Continuing an advertisement-only example, in one embodiment, statistical stability, as provided by equation (1), can be computed in the various nodes of the different segments at fixed intervals that can include, for example, every second, minute, hour, day, week, etc., and such intervals are provided merely for illustration and not limitation. Once a particular node within a segment is deemed to have achieved statistical stability, the advertisements within that node can be identified as statistically stable. User feedback from advertisements in the statistically stable category may still be obtained, and therefore the order of statistically stable advertisements can also accordingly change within a group of statistically stable N advertisements; however, once a group of advertisements is deemed statistically stable for a particular segment, the system of FIG. 1 10 can provide highly targeted advertisements to internet users 14 of that particular segment who are visiting the website. Referring now to FIGs. 5 and 6, there is shown block diagrams indicating a learning algorithm methodology or component by which feedback from users is obtained and weighted to reorder stacks. As indicated in the FIG. 5 embodiment, feedback is received 90, for example, from an internet user, whereupon the feedback is weighted, utilized to update a stack element, and the stack may be reordered accordingly. In some embodiments, the update to the stack element includes computing a probability measure associated with the stack element. As FIG. 5 illustrates, feedback can be in the form of clickstream data, for example, and can be accompanied by user information from the proprietary database 22 indicating the user's age, gender, previous purchase information, etc., and zip code. Referring now to FIG. 6, from the received feedback and information, and in particular, the zip code for the illustrated system, the base segment can be determined 92. In some embodiments, a sub-segment may or may not be identified. Once the appropriate sub-segment is identified 94, the stack item to be updated may be determined 96. As indicated herein, the stacks can include advertisements, offers, promotions, etc., and in the illustrated system of FIG. 5, the received feedback information from a user can be one of seven different forms of information. Depending upon the form of information, a weighting factor can be applied to the feedback 98. Those with ordinary skill in the art will recognize that the invention herein is not limited to the types or numbers of feedback, or the weightings associated thereto.

As FIGs. 5 and 6 indicate, in the illustrated system, feedback or update information in the form of "recommended positive feedback" or "unrecommended positive feedback" is weighted +2. Positive feedback can include information or comments from a user regarding a previous purchase, that indicates that the user is satisfied with the purchase. Recommended feedback indicates that the product was recommended by the methods and systems disclosed herein, whereas unrecommended refers to purchases by the user that were not recommended by the system 10. The descriptions of negative feedback are analogous to the positive feedback, while "recommended clickthroughs" indicate those advertisements, offers, etc., that were recommended by the system to the user, and wherein the user selected the advertisement, offer, etc., for more information, regardless of whether the user consummated the purchase, offer, etc. Similarly "unrecommended clickthroughs" represent those advertisements, offers, etc., that were not recommended by the system 10, but the user nonetheless selected the advertisement, offer, etc. Alternately, "recommended buys" indicate purchases of recommended items, with "unrecommended buys" indicating purchases of items that were not recommended. Those with ordinary skill in the art will recognize that some embodiments practiced according to the invention may not utilize all of the feedback or update information illustrated in FIG. 5, and such feedback information is merely presented for illustration, wherein other systems can utilize alternative or additional feedback. For example, a system that practices the invention yet seeks only to produce recommendations for advertisements, may only utilize the clickthrough feedback information. Alternately, a system producing content offerings may utilize clickthrough data and positive/negative feedback. In another embodiment wherein products for sale are offered, the recommended/unrecommended buy feedback may be utilized with the positive/negative feedback information, and perhaps clickthrough data. Those with ordinary skill in the art will recognize that a system according to the invention that produces multiple categories or species of recommendations can utilize greater subsets of the feedback information illustrated in FIG. 5; and, additionally, those with ordinary skill in the art will recognize that other feedback responses not illustrated in FIG. 5 can be implemented without departing from the scope of the invention. Referring again to the system of FIG. 6, once the feedback is appropriately weighted, the weighting can be applied to the stack(s) 100 item for the stack conesponding to the segment (or sub-segment) to which the user belongs. As indicated previously, this weighting can include or otherwise complement a computation of a probability measure. Depending upon the stack, a stack reordering algorithm can be applied 102 to further achieve the objectives of a stack (i.e., momentum, data mining, etc.). Once the stacks for the current node or sub-segment are updated accordingly, the lattice to which the node belongs can be traversed in the upward direction 104 as indicated previously in FIG. 3, repeating the process 106 by updating appropriate stacks in respective sub-segments. Those with ordinary skill in the art will recognize that the recommendation categories, feedback information, and weightings assigned to feedback information as shown in FIGs. 5 and 6, are merely illustrative to indicate one embodiment that can be employed to achieve the objectives of a particular application or system requirements as provided by the systems and methods described herein, and the invention herein can be practiced with greater or fewer categories and feedback information, and with assigned weightings of varying degrees.

In one embodiment, the weightings can vary depending upon the segment to which the user belongs.

Returning now to the sample embodiment wherein there is a single stack that includes advertisements for display on a website, learning may not be considered completed until the order of a statistically stable group in a stack is also stable, and hence a determination of statistical stability can be a first phase of learning, while the stabilization of the order of the most successful N advertisements can be a second phase of learning. In an embodiment wherein advertisements are the only form of recommendations provided, the stability of the order of the statistically stable N advertisements can also be determined using equation (1). To determine whether the order of the highest ranked N advertisements is stable, equation (1) may be utilized a total of N-l times, comparing p _N (i.e., replace with p _g in equation (1)) to min( /3 ,,..., p _N.,), then comparing p _N„, to min( p ,,..., p _N.2), ..., and comparing p ₂ to p ,. In an embodiment, when the N-l probabilities computed as described herein satisfy a given threshold, it may be determined that the order of the most successful N advertisements is stable for the segment. In an embodiment, order stability of the statistically stable N advertisements can also be computed at fixed intervals for the segments, such as every second, minute, hour, day, week, etc.

Continuing the website, advertisement, single stack example, for the illustrated system, as indicated previously, once the statistically stable N advertisements are determined for a particular segment (i.e., including sub-segment or node), those N advertisements can be the advertisements from which a targeted advertisement recommendation can be made for a subsequent similar-segment user to the website, in scenarios when it is appropriate to provide a targeted advertisement for that segment. In this example, therefore, wherein a stack includes only advertisements, and a user from a particular segment accesses the subscriber website, a targeted advertisement recommendation can be made by retrieving the top advertisement corresponding to the identified segment stack having statistically stable N advertisements; however, a recommendation for an advertisement can be affected by additional factors beyond mere positioning within a statistically stable stack in the appropriate segment or node. For example, another consideration in displaying an advertisement can include the various advertisement campaigns that place requirements on the frequency of advertisement display. Satisfying an advertising campaign requirement can determine, for example, whether an advertisement can be selected from the top N advertisements (statistically stable or not), or whether the advertisement may be selected from the group of advertisements not within the top N for the segment. For the discussion herein, targeted advertisement opportunities can therefore be distinguished from untargeted advertisement opportunities in that targeted opportunities can be those opportunities wherein the advertisement selected for display is, at least in part, selected because the advertisement is within a statistically stable group for that segment. For example, factors that can be considered in determining whether an advertisement can be served as a targeted advertisement or an untargeted advertisement can include the expected frequency of website visitors from a particular segment, etc.

In the illustrated embodiments, targeted advertisement opportunities can also be affected by more than stack positioning within a statistically stable group, and/or advertising campaign considerations. For example, consider a scenario wherein advertising campaigns are not at issue because, for example, impression budgets for all campaigns are satisfied, yet an advertisement opportunity arises that is deemed an opportunity for a targeted advertisement. The most successful advertisement in the associated stack of statistically stable N advertisements can be an advertisement for a product that is no longer in stock, and therefore it is not practical to advertise the product until such stock conditions change. Additionally, an advertisement that is lower on the stack can be significantly more profitable than a higher element, and wherein the probability of clickthrough rates between several or more advertisements within the top N can be negligible, a business decision can be made to select a more profitable advertisement. In yet another scenario, the profitability of an advertisement to the website for a particular advertisement can be greater for a lower- ranked advertisement, than for a higher-ranked advertisement in that segment, wherein in one embodiment, the profitability difference may outweigh the probability of success to allow selection of the more profitable advertisement. Although inventory considerations, profitability of the advertisements, or profit margin to the website are three considerations, the invention herein is not limited to these three business concerns, and the invention includes all types of business judgments and constraints that can be made in making a selection, wherein the selection can be more broadly understood as a maximization of the advertisement success subject to business constraints.

In an embodiment, business constraints can also include business considerations such as age-appropriateness for the item, appropriateness with regard to a user's previous purchases, whether the user has previously viewed the recommendation, etc., and the consideration of business constraints in selecting a recommendation can be viewed in some embodiments as a filtering process, wherein a subscriber 16 can specify filtering parameters that can vary according to application.

As mentioned continually throughout the above example, the above example was simplified to advertisements only with a single stack for each of the segments; however, the concept of populating stacks, applying a learning algorithm to order the stacks according to user feedback and periodically survey the stacks to identify statistical stability, and supplying recommendations by weighting stack positioning against business obligations and constraints, can apply to all species of recommendations that can include products, advertisements, content offerings, promotions, etc. For discussion purposes, returning to the website example employing only advertisements, there can be a mechanism to incorporate new advertisements to the respective segment stacks. In an embodiment, an algorithm to incorporate new advertisements can accompany the learning algorithm and or complement the learning algorithm. In an embodiment, a probability measure associated with a new advertisement to a stack can be evaluated at fixed intervals using equation (1), wherein the fixed interval can be on the order of seconds, minutes, hours, days, weeks, etc. More specifically, equation (1) can be computed wherein p_g represents the probability measure of the new advertisement, wherein equation (1) compares the new advertisement probability measure to the probability measures of the highest ranked N advertisements (whether or not such highest ranked advertisements are statistically stable and/or stable in terms of order). In a case wherein the top N advertisements in a segment or node are statistically stable, and the probability measure computed by equation (1) exceeds a given threshold, the new advertisement can be included in the statistically stable group. In an embodiment wherein there can be a fixed N, a successful new advertisement can displace an advertisement previously determined to be either in the top N, or the statistically stable group.

Referring now again to the system illustrated in FIG. 1, and for simplification, considering an embodiment wherein only advertisements are provided for recommendations, when the user 14 downloads a subscriber webpage, the web agent 20 can retrieve user information from the proprietary database 22 and supply such information to the data mining engine 28. The data mining engine 28 can identify segments and sub- segments for the user 14 using the information from the web agent 20 and/or proprietary database 22. The data mining engine 28 can thereafter access the appropriate portion of the knowledge database 30 relating to the segments or sub-segments for the user 14. In the illustrated system, a recommendation such as an advertisement (offer, promotion, etc.,) is retrieved from a conesponding stack, utilizing associated learning algorithm results and business constraints. In some embodiments, the advertisement is compared to the user data to ensure that the user did not previously view the advertisement or purchase the same item recently, that the item is age appropriate for the user, etc. In such embodiments, upon the data mining agent 28 determining that the offer is not appropriate for the user, the data mining agent can again refer to the stacks for another advertisement. In another embodiment, the subscriber 16 can elect a filtering technique that filters the advertisements before the selection occurs. For example, the advertisements can be filtered for age- appropriateness prior to selection, or in another embodiment, advertisements can be filtered by content. For example, if an advertisement opportunity appears on a home page, then a filter can identify those advertisements that satisfy this requirement. Those with ordinary skill in the art will recognize that the filter can have multiple and varied parameters, and can be implemented either before the initial selection, or after the initial selection but before the final selection of an advertisement, without departing from the scope of the invention. The selected ("final selection") advertisement can be transferred to the web agent 20 for display on the user's web page. In some embodiments wherein the subscriber 16 maintains content for the selected advertisement, the advertisement content can be provided to the user with the content from the subscriber's web page; however, in other embodiments, the user may receive the content from the subscriber's web page with a pointer or link message to the selected advertisement, wherein the advertisement can be retrieved from the advertisement web page for display to the user. The above-process can be performed in real-time to allow near-instantaneous display of the advertisement on the user's web page.

In the illustrated systems, selection of a recommendation may also include evaluating the contents of a node for existence of data. For example, consider the case wherein stacks are populated with only advertisements. A segment or node of a user or visitor to a website can be identified, and it can be determined by examining the stack contents that the user is the first user or visitor to the website for that segment. In this situation, it may be neither possible nor appropriate, depending on the embodiment, to provide a recommendation from the identified segment or node. In this instance, the lattice structure of FIG. 3 can be traversed in an upward direction to identify related segments to the identified segment, and to evaluate the data in those segments to determine whether an advertisement from the related segments should be recommended. In the FIG. 3 lattice structure, the lattice becomes less specific in describing the user as the lattice is traversed upwardly, and there are typically multiple nodes related to any given node in the upward direction; therefore, in an embodiment, a system administrator, for example, can specify the logic that can be executed to traverse a lattice and determine an order of examination for multiple related nodes. In one embodiment, this lattice traversing logic may be incorporated into a configuration file.

In the illustrated system of FIGs. 1-6, wherein a segment or sub-segment can relate to, for example, six stacks, selection logic can be imposed when only one advertisement is displayed to the user 14. In one embodiment, the data mining engine 28 can utilize selection logic that is based on probability rankings. For example, the six stacks can be equally weighted by assigning a probability of approximately seventeen percent to each stack. In such an embodiment, when the data mining engine 28 seeks information from the stacks, the probability of an advertisement, offer, etc, being retrieved from the history stack is just as probable as the item being retrieved from the null stack. In an alternate embodiment, the history and momentum stacks can be assigned weights for selections thirty percent of the time each, with the data mining, null, manual, and learning stacks being selected ten percent each. Those with ordinary skill in the art will recognize that there are many different probability assignments that can be made to assist in stack selection, and there are other, non-probability stack selection methods that can be practiced without departing from the scope of the invention. Those with ordinary skill in the art will also appreciate that the assignment of probabilities to stacks selection can be incorporated into individual stack item selection to accommodate advertising campaign constraints and other business obligations. In yet another embodiment wherein multiple stacks with multiple learning algorithms can be employed, learning can be applied across stacks. For example, a weighting algorithm could be applied across the stacks to indicate whether a particular stack within a particular segment or sub-segment, provides the most successful advertisements, offers, etc. Referring again to FIG. 1, at various times, the remote administration unit 32 can compile new rules for various segments, etc. Such rules can be transmitted to the knowledge database 30 without interfering with the subscriber's database. Additionally, updates to the data mining engine 28 can similarly be applied. In some embodiments, backups of the data mining engine 28 and knowledge database 30 can be made at the remote administration unit 32. In such scenarios, if the monitoring agent 26 determines that the data mining system 24 is not otherwise operational, the data mining system 24 operation can be coordinated through the monitoring agent 26 to the remote administration unit 32 to allow uninterrupted system 26 operation. In such embodiments, corruption of the data mining engine 28 or knowledge database 30 can be overcome with a retransmission of information from the remote administration unit 32. In these embodiments, the remote administration unit 32 can perform scheduled backups of the knowledge database 30. In an embodiment, the subscriber 16 can be provided with reports from the data mining system 24. A report can inform the subscriber 16 of those recommendations, for example, that are more successful than others, to allow a subscriber 16 to alter campaign agreements, contractual obligations, etc., accordingly. Reports can include segment characteristics including segment number, segment demographic information, the number of website users visiting the website in a particular time period, the statistics pertaining to recommendations in the different segments including probability measures, and such reporting information is provided merely for illustration and not limitation.

In an embodiment, the data mining system 24 can be configured to provide a recommendation selection to a third party that is not the subscriber 16. This sharing of information can be effectuated through the subscriber 16, or directly between the data mining system 24 and the third party. In an embodiment, the third party may receive the recommendation selection from the data mining system 24 and apply additional business constraints to the recommendation selection. In an embodiment, demographic data can be provided to a simulator to simulate the lattice structures and estimate statistics based on different lattice/node structures and combinations. The simulator can also provide estimates based on combinations of lattice/node structures and business constraints.

One advantage of the present invention over the prior art is that accurate predictions of a consumer's buying habits can be generated using combinations of segmentation schemes and learning through segment member activity.

What has thus been described are segment-based, self-learning systems and methods that allow the formation and continued refinement of consumer segment profiles that originate from demographic data and provide recommendations to consumers on content offerings, advertisements, products for sale, promotions, etc. In one embodiment, the demographic data is U.S. census data, wherein zip codes can be classified into one of sixty- eight predefined demographic segments. The segments can be divided into sub-segments based upon age, gender, product category, or other data that can include profession, family size, income, etc. From the segments, knowledge rules can be formed and refined by identifying consumers by segment, tracking associated purchasing behaviors, and continually adapting the knowledge rules by segments to reflect consumer behavior. In one embodiment, probability measures can be computed and ordered according to segment and/or sub-segment to determine those recommendations likely to be accepted by consumers within a respective segment.

Although the present invention has been described relative to a specific embodiment thereof, it is not so limited. Obviously many modifications and variations of the present invention can become apparent in light of the above teachings. For example, although the systems and methods illustrated a configuration wherein the data mining system resided completely at a subscriber location, various other configurations can be allowed. In one embodiment, the subscriber 16 can transmit a request for a recommendation, and the recommendation request can be processed at the remote administration unit, in a scenario similar to a back-up scenario presented herein. In yet another embodiment that does not require real-time decisions, information can be sent from the subscriber 16 to a remote unit. In such instances, the knowledge rules, including generation of the statistically stable recommendations, can be generated remotely and supplied to the subscriber 16, wherein the subscriber 16 can later apply the rules at the subscriber's desired time. For the methods and systems disclosed herein, although the description of the illustrated embodiments included references to providing "a recommendation", or in example embodiments, "an advertisement", the use of "a" or "an" can be understood to include multiple recommendations, advertisements, etc. Although the embodiment listed herein referred to receiving information on a computer, the invention herein is not limited, and the information can be presented on a telephone display or other electronic device. Similarly, the systems and methods may be employed to generate recommendations that are delivered non-electronically, such as through direct mailing.

Many additional changes in the details, materials, steps and anangement of parts, herein described and illustrated to explain the nature of the invention, can be made by those skilled in the art within the principle and scope of the invention. Accordingly, it will be understood that the invention is not to be limited to the embodiments disclosed herein, can be practiced otherwise than specifically described, and is to be understood from the following claims, that are to be interpreted as broadly as allowed under the law.

Claims

1. A method for learning consumer preferences, comprising, defining segments based on demographics, associating recommendations with the segments, receiving consumer responses to recommendations, relating the responses to segments associated with the consumers, and, on a segment basis, ranking recommendations based on the related responses.

2. A method according to claim 1, wherein receiving consumer responses to recommendations further comprises receiving consumer responses from at least one of advertisements, content offerings, products, and promotions.

3. A method according to claim 1, wherein receiving consumer responses to recommendations further comprises obtaining clickstream data.

4. A method according to claim 1 , wherein ranking recommendations further includes associating a probability measure with the recommendations.

5. A method according to claim 1, further comprising selecting a recommendation based on a consumer's segment and the respective segment's recommendation rankings.

6. A method according to claim 5, wherein selecting a recommendation further includes selecting the recommendation based on business constraints.

7. A method according to claim 5, wherein selecting a recommendation further includes selecting the recommendation based on at least one of profitability of the recommendation, profitability of the success of the recommendation, age appropriateness of the recommendation, and contractual obligations related to the recommendation.

8. A method according to claim 5, wherein selecting a recommendation further includes determining those recommendations that are statistically stable.

9. A method according to claim 8, wherein determining those recommendations that are statistically stable further includes, within a segment, computing probability measures to estimate consumer acceptance of the recommendations within the segment, ordering the probability measures according to numerical value, determining an integer N representing a statistically stable number of recommendations with the N highest ordered probability measures, providing a threshold, computing a statistic for recommendations whose probability measures are not within the N highest ordered probability measures, the statistic computed as,

where: p_g is the probability measure for a recommendation whose probability measure is not within the N highest ordered probability measures; p is the probability measure for the jth highest ordered probability measure; nj is the number of impressions of the recommendation conesponding to the jth highest ordered probability measure; n_g is the number of impressions of the recommendation whose probability measure is not within the N highest ordered probability measures; pp can be computed as p_P = (χ_g + xj) l(n_g + nj) ;

^ is the probability distribution function otherwise known as a standard normal distribution function with a mean of zero and a standard deviation of one; Xi is the number of favorable responses to the recommendation conesponding to the jth highest ordered probability measure; and, x_g is the number of favorable responses conesponding to a recommendation whose probability measure is not within the N highest ordered probability measures; and, comparing the threshold to the statistic for recommendations whose probability measures are not within the N highest ordered probability measures.

10. A method according to claim 1, wherein ranking recommendations further includes computing a measured probability of acceptance for the recommendations.

11. A method according to claim 1, wherein defining segments based on demographics includes providing segments according to at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities

12. A method according to claim 1, further comprising weighting the consumer responses.

13. A method according to claim 1, wherein relating the response to segments associated with the consumers further includes identifying segments related to the respective consumers based on demographics.

14. A method according to claim 13, wherein relating the responses to segments associated with the consumers further includes identifying segments related to the respective consumers based on zone improvement plan (ZIP) code.

15. A method according to claim 5, wherein selecting a recommendation further includes filtering the recommendations in the consumer's segment.

16. A method according to claim 5, wherein selecting a recommendation further includes applying a content filter to the recommendations in the consumer's segment.

17. A method of predicting consumer preferences, comprising, forming segments and associating attributes with the segments, receiving consumer behavior data, associating the consumer behavior data to segments, and, 5 adjusting consumer preferences within the segments based on the monitoring.

18. A method according to claim 17, further comprising, providing recommendations to consumers, and wherein, o receiving consumer behavior data further includes receiving consumer reactions to the recommendations, and, adjusting consumer preferences further includes weighting the recommendations based on the reactions.

5 19. A method according to claim 18, wherein weighting the recommendations further includes computing a probability measure for a recommendation to indicate a measured probability of acceptance of the recommendation.

20. A method according to claim 19, wherein adjusting consumer preferences further 0 includes ranking the probability measures.

21. A method according to claim 17, further comprising providing a consumer recommendation based on the adjusted preferences.

22. A method according to claim 21, wherein providing a consumer recommendation further comprises selecting a recommendation based on business constraints.

23. A method according to claim 22, wherein selecting a recommendation based on business constraints further includes selecting the recommendation based on at least one of profitability of the recommendation, profitability of the success of the recommendation, age appropriateness of the recommendation, and contractual obligations related to the recommendation.

24. A method according to claim 17, wherein adjusting consumer preferences further includes ranking at least one of advertisements, content offerings, products for sale, product promotions, service offerings, or service promotions.

25. A method according to claim 17, wherein forming segments and associating attributes with the segments further includes defining the segments based on demographic attributes.

26. A method according to claim 25, wherein defining the segments based on demographic attributes further includes characterizing the segments by at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

27. A method according to claim 17, further comprising, analyzing consumer characteristics by geographic region, classifying the geographic regions into segments by comparing respective consumer characteristics with segment attributes, and associating consumers with segments based on geographic region.

28. A method according to claim 17, wherein receiving consumer behavior data includes receiving clickstream data.

29. A method according to claim 17, wherein receiving consumer behavior data includes receiving at least one of recommended positive feedback, recommended negative feedback, unrecommended positive feedback, unrecommended negative feedback, recommended clicks, unrecommended clicks, recommended buys, and unrecommended buys.

30. A method according to claim 17, further comprising weighting the received consumer behavior data.

31. A method according to claim 17, wherein associating the consumer behavior data to segments further includes associating data to segments based on demographics.

32. A method according to claim 17, wherein associating the consumer behavior data to segments further includes associating data to segments based on zone improvement plan (ZIP) codes.

33. A method according to claim 17, wherein adjusting consumer preferences includes adjusting consumer preferences in real-time.

34. A method of selecting a recommendation for a consumer, comprising, providing consumer segments from demographic data, associating recommendations with the consumer segments, receiving responses from consumers based on the recommendations, ordering the recommendations within segments based on the responses, identifying a segment associated with the consumer, and, selecting a recommendation associated with the identified segment based on the respective ordering.

35. A method according to claim 34, wherein selecting a recommendation further includes weighting the recommendation against business constraints.

36. A method according to claim 35, wherein weighting the recommendation further includes considering at least one of contractual obligations related to recommendations, profitability of providing recommendations, profitability of the success of recommendations, age appropriateness of recommendations, and previous behavior of the consumer.

37. A method according to claim 34, wherein associating recommendations with consumer segments further includes associating at least one of advertisements, content offerings, products, and promotions.

38. A method according to claim 34, wherein providing consumer segments further comprises, selecting at least two segments, and, assigning demographic criteria to the segments.

39. A method according to claim 38, further comprising, analyzing census data according to geographic region, and, associating the geographic regions with the segment for which the respective analysis results relate to the segment criteria.

40. A method according to claim 38, wherein assigning demographic criteria to the segments further includes describing a segment based on at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

41. A method according to claim 38, further including forming sub-segments based on the segments and at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

42. A method according to claim 39, wherein analyzing census data according to geographic region includes characterizing the geographic region by at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

43. A method according to claim 39, wherein analyzing census data further includes analyzing census data according to zone improvement plan (ZIP) codes.

44. A method according to claim 34, wherein selecting a recommendation further includes selecting at least one of advertisements, content offerings, products, and promotions.

45. A method according to claim 34, wherein identifying a segment further includes identifying a segment based on the consumer's demographics.

46. A method according to claim 34, wherein identifying a segment further includes identifying a segment based on the consumer's zone improvement plan (ZIP) code.

47. A method according to claim 34, further comprising, categorizing by segment, consumer responses to the recommendations.

48. A method according to claim 34, wherein ordering the recommendations further comprises computing a probability measure for the recommendations based on the responses.

49. A method according to claim 48, further comprising ordering the probability measures.

50. A method according to claim 34, wherein providing consumer segments from demographic data further includes providing segments from at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities

51. A method according to claim 34, further including weighting the responses from consumers.

52. A method according to claim 34, wherein receiving consumer responses further includes receiving clickstream data.

53. A method according to claim 52, wherein receiving clickstream data further includes receiving at least one of recommended positive feedback, recommended negative feedback, unrecommended positive feedback, unrecommended negative feedback, recommended clicks, unrecommended clicks, recommended buys, and unrecommended buys.

54. A method according to claim 34, wherein selecting a recommendation includes selecting a recommendation in real-time.

55. A method according to claim 34, wherein selecting a recommendation further includes filtering recommendations.

56. A method according to claim 55, wherein filtering recommendations further includes filtering the recommendation based on content.

57. A method for selecting a recommendation for a consumer, comprising, implementing a learning algorithm that classifies consumers into segments based upon demographic data, receives responses to recommendations from the consumers, associates the responses with segments, and based on the responses, computes probability measures for the recommendations, and, selecting a recommendation based on the probability measures for a segment associated with the consumer.

58. A method according to claim 57, wherein selecting a recommendation further comprises considering business constraints.

59. A method according to claim 57, wherein selecting a recommendation further includes considering at least one of contractual obligations related to recommendations, profitability of providing recommendations, profitability of the success of recommendations, age appropriateness of recommendations, and previous behavior of the consumer.

60. A method according to claim 57, further comprising determining a segment that conesponds to the consumer.

61. A method according to claim 60, wherein determining a segment that corresponds to the consumer further comprises, receiving at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities, from the consumer and, associating the consumer with a segment by comparing the consumer information with the segment criteria.

62. A method according to claim 60, wherein determining a segment further includes, receiving the zone improvement plan (ZIP) code of the consumer, and, classifying the user based on ZIP code.

63. A method according to claim 60, wherein determining a segment further includes, receiving demographics associated with the consumer, and, classifying the user based on the demographics.

64. A method according to claim 57, wherein selecting a recommendation further comprises selecting at least one of an advertisement, a product, a promotion, or a content offering.

65. A method according to claim 57, further comprising delivering the recommendation to the consumer.

66. A method according to claim 65, wherein delivering the recommendation further comprises displaying the recommendation on a web page.

67. A method according to claim 65, wherein delivering the recommendation further comprises presenting the recommendation electronically.

68. A method according to claim 65, wherein delivering the recommendation further comprises generating an electronic link to the recommendation.

69. A method according to claim 57, wherein selecting a recommendation based on the probability measures further includes determining whether the probability measures are sufficiently stable.

70. A method according to claim 69, wherein determining whether the probability measures are sufficiently stable further comprises, ordering the probability measures according to numerical value, determining an integer N representing a statistically stable number of recommendations with the N highest ordered probability measures, providing a threshold, computing a statistic for recommendations whose probability measures are not within the N highest ordered probability measures, the statistic computed as,

= ¹ - 1! Mb ^~ P W ^{/ nj + 1/} "^s)^^{1 "} '» '

7=1 where: pg is the probability measure for a recommendation whose probability measure is not within the N highest ordered probability measures; pj is the probability measure for the jth highest ordered probability measure; nj is the number of impressions of the recommendation conesponding to the jth highest ordered probability measure; n_g is the number of impressions of the recommendation whose probability measure is not within the N highest ordered probability measures; pp can be computed as p_P = (x_g + xj) l(n_g + n ) ;

^ is the probability distribution function otherwise known as a standard normal distribution function with a mean of zero and a standard deviation of one; XJ is the number of favorable responses to the recommendation conesponding to the jth highest ordered probability measure; and, x_g is the number of favorable responses conesponding to a recommendation whose probability measure is not within the N highest ordered probability measures; and, comparing the threshold to the statistic for recommendations whose probability measures are not within the N highest ordered probability measures.

71. A method according to claim 57, wherein implementing a learning algorithm that that classifies consumers into segments based upon demographic data further includes classifying segments based upon at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities

72. A method according to claim 57, wherein implementing a learning algorithm further includes weighting the responses.

73. A method according to claim 57, wherein implementing a learning algorithm that receives responses from the consumers further includes implementing a learning algorithm that receives clickstream data.

74. A method according to claim 57, wherein selecting a recommendation includes selecting a recommendation in real-time.

75. A method according to claim 57, wherein selecting a recommendation further includes filtering recommendations associated with the selected consumer segment.

76. A method according to claim 75, wherein filtering recommendations further includes filtering the recommendation based on content.

77. A method for classifying a consumer based on geographic region, comprising, providing segments with associated demographic attributes, analyzing demographic data based on geographic region, assigning geographic regions to segments related to the analyzed data, receiving the consumer's geographic region, and, associating the consumer with the segment to which the consumer's geographic region is assigned.

78. A method according to claim 77, wherein providing segments with associated demographic attributes further includes associating with segments at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

79. A method according to claim 77, wherein analyzing demographic data based on geographic region further comprises analyzing census data.

80. A method according to claim 77, wherein analyzing demographic data based on geographic region further comprises computing statistics for at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

81. A method according to claim 77, wherein receiving the consumer's geographic region further includes receiving a zip code.

82. A method according to claim 77, wherem receiving the consumer's geographic region further includes receiving the geographic region electronically.

83. A method according to claim 77, wherein receiving the consumer's geographic region further includes receiving the geographic region via a network.

84. A system to select a recommendation for a consumer, comprising, a knowledge database for storing segmentation rules and segment results based on demographic data, and having recommendations associated with the segments, and, a processor module to access the knowledge database, interpret the segmentation rules, and select a recommendation from a segment associated with the consumer.

85. A system according to claim 84, further comprising a consumer database for storing historical data related to consumers, the database in communications with the processor module.

86. A system according to claim 84, further comprising a monitoring module to monitor the integrity of the knowledge database and the processor module.

87. A system according to claim 84, further comprising a remote administration module in communications with the knowledge database and the processor module.

88. A system according to claim 84, wherein the remote administration module further comprises a replication of the knowledge database and a replication of the processor module.

89. A system according to claim 84, wherein the knowledge database further comprises at least one grouping of recommendations associated with the segments.

90. A system according to claim 84, wherein the knowledge database further comprises at least one of an advertisement, product, content offering, and promotion for the segments.

91. A system according to claim 84, wherein the knowledge database further comprises a learning algorithm to associate probability measures to the recommendations, the probability measures providing an estimate of the probability of acceptance of the recommendations within the segments.

92. A system according to claim 84, wherein the knowledge base further includes at least one stack associated with a respective segment.

93. A network, comprising a subscriber device having, a website with space for recommendations, the subscriber device responsive to requests to provide media content, and, a recommendation selection module to identify content for display in the recommendation space, a user device having a browser program coupled to the network and capable of requesting the website and the recommendation content for display on the user device.

94. A network according to claim 93, further comprising, a recommendation server, responsive to a request from the user device to provide recommendation content, the request to provide recommendation content based on a link message from the subscriber device to the user device.

95. A network according to claim 93, wherein the subscriber device is a server.

96. A network according to claim 93, wherein the subscriber device is a personal computer (PC).

97. A network according to claim 93, wherein the subscriber device includes a SUN Workstation.

98. A network according to claim 93, wherein the user device is a personal computer (PC).

99. A network according to claim 93, wherein the user device is a SUN Workstation.

100. A network according to claim 93, wherein the user device is a telephone.

101. A network according to claim 93, wherein the user device is a television.

102. A network according to claim 93, wherein the recommendation selection module further includes a memory for storing segmentation rules.

103. A network according to claim 93, wherein the recommendation selection module further includes a processor and instructions for causing the processor to identify a segment associated with a user of the user device, and select a recommendation based on the identified segment.

104. A network according to claim 94, wherein the recommendation server further includes an advertisement server.

105. A method for selecting a recommendation for presentation to an internet user, comprising, providing consumer segments based on demographic data, associating recommendations with the segments, associating the internet user with a selected consumer segment, and selecting a recommendation from the recommendations associated with the selected consumer segment.

106. A method according to claim 105, wherein selecting a recommendation further includes selecting the recommendation based on business constraints.

107. A method according to claim 105, wherein selecting a recommendation further includes selecting a recommendation based on at least one of contractual obligations related to recommendations, profitability of providing recommendations, profitability of the success of a recommendations, age appropriateness of a recommendation, and previous behavior of the consumer.

108. A method according to claim 105, wherein providing consumer segments based on demographic data further comprises associating segments with at least one demographic criterion.

109. A method according to claim 105, wherein providing consumer segments based on demographic data further includes associating segments with at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

110. A method according to claim 105, wherein providing segments from demographic data further includes analyzing demographic data according to geographic region.

111. A method according to claim 105, wherein providing segments from demographic data further forming segments from census data.

112. A method according to claim 110, wherein analyzing census data according to geographic region includes characterizing geographic regions by at least one of income level, education level, ethnicity, gender, age, region of habitation, profession, family size, home ownership, and proximity to cities.

113. A method according to claim 105, wherein selecting a recommendation further includes selecting at least one of advertisements, content offerings, products, and promotions.

114. A method according to claim 105, further comprising, receiving a request for at least one of an advertisement, content offering, product, and promotion, and wherein selecting a recommendation further includes selecting at least one of an advertisement, content offering, product, and promotion, in response to the request.

115. A method according to claim 105, wherein associating the internet user with a selected consumer segment further includes associating the internet user with a segment based on the internet user's demographics.

116. A method according to claim 105, wherein associating the internet user with a selected consumer segment further includes associating the internet user with a segment based on the internet user's zone improvement plan (ZIP) code.

117. A method according to claim 105, wherein associating the internet user with a selected consumer segment further comprises receiving clickstream data associated with the internet user.

118. A method according to claim 105, wherein associating the internet user with a selected consumer segment further comprises retrieving cookie information from the internet user.

119. A method according to claim 105, wherein selecting a recommendation from the recommendations associated with the selected consumer segment further comprises providing a learning algorithm to identify the most successful recommendations based on the segments.

120. A method according to claim 119, wherein providing a learning algorithm further includes weighting recommendations within segments based on consumer feedback from consumers associated with the respective segments.

121. A method according to claim 119, wherein providing a learning algorithm further includes weighting recommendations based upon at least one of recommended positive feedback, recommended negative feedback, unrecommended positive feedback, unrecommended negative feedback, recommended clicks, recommended buys, and unrecommended buys.

122. A method according to claim 119, wherein providing a learning algorithm further includes computing a probability measure for the recommendations.

123. A method according to claim 122, further including ranking the probability measures within a segment.

124. A method according to claim 105, further comprising delivering the selected recommendation to the internet user.

125. A method according to claim 105, wherein selecting a recommendation further includes filtering recommendations associated with the selected consumer segment.

126. A method according to claim 125, wherein filtering recommendations further includes filtering the recommendation based on content.

127. A computer product disposed on a computer readable medium, for learning consumer preferences, the computer product comprising instructions for causing a processor to, define segments based on demographics, associate recommendations with the segments, receive consumer responses to recommendations, relate the responses to segments associated with the consumers, and, on a segment basis, rank recommendations based on the related responses.

128. A computer product disposed on a computer readable medium, for predicting consumer preferences, the computer product comprising instructions for causing a processor to, form segments and associating attributes with the segments, receive consumer behavior data, associate the consumer behavior data to segments, and, adjust consumer preferences within the segments based on the monitoring.

129. A computer product disposed on a computer readable medium, for selecting a recommendation for a consumer, the computer product comprising instructions for causing a processor to, provide consumer segments from demographic data, associate recommendations with the consumer segments, receive responses from consumers based on the recommendations, order the recommendations within segments based on the responses, identify a segment associated with the consumer, and, select a recommendation associated with the identified segment based on the respective ordering.

130. A computer product disposed on a computer readable medium, for selecting a recommendation for a consumer, the computer product comprising instructions for causing a processor to, implement a learning algorithm that classifies consumers into segments based upon demographic data, receives responses to recommendations from the consumers, associates the responses with segments, and based on the responses, computes probability measures for the recommendations, and, select a recommendation based on the probability measures for a segment associated with the consumer.

131. A computer product disposed on a computer readable medium, for classifying a consumer based on geographic region, the computer product comprising instructions for causing a processor to, provide segments with associated demographic attributes, analyze demographic data based on geographic region, assign geographic regions to segments related to the analyzed data, receive the consumer's geographic region, and, associate the consumer with the segment to which the consumer's geographic region is assigned.