KR20070015918A - Approach for estimating user ratings of items - Google Patents

Abstract

According to another embodiment of the invention, an approach is provided for how a user ranks a particular item from multiple items. The approach can be applied to various kinds of items, including rental items such as movies, music and games, and the invention is not limited to any particular kind of item. One or more items having a similar rating to a particular item are identified from the plurality of items. Next, one or more other users are identified who have given the one or more items a rating substantially similar to the rating given by a particular user. Finally, an estimate is made of how the particular user ranks the particular item based on the rating of the particular item given by one or more other users.

Description

APPROACH FOR ESTIMATING USER RATINGS OF ITEMS}

The present invention relates to article rental, and more particularly to an approach for estimating user ratings of items.

Conventional item rental models are generally based on items being rented for a fixed rental period and charge a late fee while keeping the rented items beyond a certain return date. As used herein, the term "item" refers to any trade item. Article models of this type experience some significant limitations. First, conventional rental models require customers to determine which items to rent substantially simultaneously with determining when to rent the items. One example of the limitation is the video rental business. Customers go to a video rental store and select specific movies to rent. Customers must take the movies home and return them by a certain date or pay a late fee. In this case, customers cannot decide which movies to rent before actually renting the movies. Customers can consider a particular movie, but cannot guarantee that a video rental store has a particular movie in stock. In addition, the dates are inconvenient for customers, especially for "new" movies that must be returned the next day.

Given the current needs for article rental and limitations in previous approaches, the approach to items being rented to customers does not experience limitations in connection with conventional article rental models and is highly desirable. In particular, the approach to renting item items that determine allows for distinguishing which items to rent and the consumer's decision as to when to rent the items, which is highly desirable.

Further, there is a need for an approach to renting items to customers on a continuous basis that prevents the use of rental "windows" that are dependent on fixed dates or conventional rental models.

There is also a need for an approach to rent movies, games and music to customers, which is more convenient and flexible to customers than conventional approaches.

There is also a need for an approach for estimating how a user ranks items that the user has not yet rated.

Embodiments of the present invention are described by way of example and are not limited in the accompanying drawings, wherein like reference numerals refer to like elements.

1 is a diagram illustrating an approach for renting items to customers according to one embodiment.

2 is a flow diagram illustrating an approach for renting items to customers according to one embodiment.

3 is a flow diagram illustrating a "Max Out" approach to renting items to customers according to one embodiment.

4 is a flow diagram illustrating a "Max Turns" approach to renting items to customers according to one embodiment.

5 is a diagram illustrating an approach for renting audio / video items to customers via the Internet, according to one embodiment.

6 is a flowchart illustrating an approach for renting audio / video items to customers over the Internet using both "Max Out" and "Max Turn", according to one embodiment.

7 is a flow diagram illustrating an approach for estimating a manner in which a user assigns a rating to an item in accordance with one embodiment of the present invention.

8 is a block diagram illustrating the ranking of items of a user arranged in one array.

9 is a block diagram illustrating an apparatus for recommending items in accordance with one embodiment of the present invention.

10 is a block diagram illustrating another apparatus for recommending items according to an embodiment of the present invention.

11 is a block diagram of a computer system in which embodiments of the present invention may be implemented.

In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of the present invention. It will be appreciated, however, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the present invention.

Various aspects and features of exemplary embodiments of the invention are described in detail in the following sections: I. Functional Overview II. Item Selection Criteria III. Item Delivery IV. "Max Out" V. "Max Turn" VI. Goods Management VII. Item Recommendation VIII. Execution mechanism

I. Functional Overview

1 is a block diagram 100 illustrating an approach for renting items to customers in accordance with various embodiments disclosed herein. As used herein, the term “items” refers to any commercial product that can be rented to customers. Examples of items include movies, music, and games stored in nonvolatile memory such as tape, other magnetic media, optical media, read-only memory, and the like, and the invention is not limited to some specific item types. In general, the determination of which items to rent is independent of the determination of when to rent the items. Customers can specify items to rent using one or more item selection criteria, which are separate from determining when to receive the specified items. Also, customers can have serialized, serialized rental items instead of being limited by conventional rental "windows."

According to one embodiment, customer 102 provides one or more item selection criteria to provider 104 via link 106. The link 106 can be any medium for transferring data between the customer 102 and the provider 104, and the invention is not limited to some specific media. Examples of link 106 include, but are not limited to, a network such as a LAN, a QAN or the Internet, a telecommunications link, a wired or optical link, or a wireless connection.

The item selection criteria indicate the items that the customer 102 wants to rent from the provider 104. In response to receiving item selection criteria from the customer 102, the provider 104 provides the items 102 indicated by the item selection criteria to the customer 102 via the delivery channel 108. Delivery channel 108 may be implemented by any mechanism or medium that provides for the transfer of items from provider 104 to customer 102, and the present invention is not limited to some specific form of delivery channels. Examples of delivery channel 108 include, but are not limited to, mail delivery, delivery of a delivery person, or delivery using a delivery agent. The provider 104 may be centralized or distributed depending on the requirements of a particular application.

According to one embodiment, the "max out" approach allows up to a certain number of items to be simultaneously loaned to the customer 102 by the provider 104. According to one embodiment, the "max turn" approach allows up to a certain number of items to change to occur for a particular period of time. "Max Out" and "Max Turn" can be used together or separately by various booking methods.

The above described approach for renting items to customers is described with reference to the flowchart 200 of FIG. 2. Beginning at step 202, at step 204, the customer 102 generates item selection criteria. In step 206, customer 102 provides item selection criteria to provider 104. In step 208, in response to the provider 104 receiving item selection criteria from the customer 102, the provider 104 provides one or more items indicated to the customer 102 by the item selection criteria. The process ends at 210. II. Item selection criteria

One or more item selection criteria provided to the provider 104 by the customer 102 indicate the specific items that the customer 102 wants to rent from the provider 104. Thus, item selection criteria define a customer-specific command queue that is performed by the provider 104. According to one embodiment, the item selection criteria specify the attributes of the items to be provided to the customer 102 by the provider 104. The item selection criteria may specify any form of item characteristics, and the present invention is not limited to specific item characteristics. Item characteristics include, but are not limited to, identifier characteristics, form characteristics, and cost characteristics. The item selection criteria can be changed at any time to reflect changes in the items that customers want to rent from the provider. III. Item delivery

According to one embodiment, the items are delivered by the provider 104 to the customer 102 via the delivery channel 108 in accordance with item delivery criteria. In particular, the delivery of items from the provider 104 to the customer 102 is initiated by satisfied item delivery criteria. Item delivery criteria may include a wide range of criteria, but the invention is not limited to any particular item delivery criteria. Examples of item delivery criteria include, but are not limited to, a customer request / notification, a customer notification that an item is returned, a customer return of an item, the discovery of a specific date, the passage of a specific period of time, or a customer payment cost.

Item delivery criteria may be specified to the provider 104 by the customer 102 or negotiated by the customer 102 and the provider 102 as part of a subscription service. For example, a particular subscription service may include item delivery criteria that specify that a particular number of items will be delivered monthly. As another example, the item delivery criteria would be delivered to the customer 102 by the provider 104 when the initial set of items was initiated by the subscription service, and the customer 102 when additional items returned to the provider 104. It can specify that it will be delivered by. The item delivery criteria may be applied uniformly to all items to be delivered to the customer or may be specific to the item. For example, the item delivery criteria may be specified as a specific date for all item exhibitions, namely the third Wednesday of every month. Optionally, individual item delivery dates may be assigned to each item. IV. "Max Out"

According to one embodiment, the "max out" approach is used to manage the number of items that can be loaned simultaneously to customers. According to the "max out" approach, up to a certain number of items can be loaned to the customer at the same time. Thus, the "max out" approach forms the item size of items that can be maintained by customers. A certain number of items can be specific to each customer and can be common to one or more customers. In the present example, if a particular number of items is three, up to three items may be simultaneously loaned to the customer 102 by the provider 104. If a certain number of items are simultaneously loaned to the customer 102 and a particular item delivery criteria triggers the delivery of one or more items, their items are returned when the one or more items are returned to the provider 104 by the customer 102. It is not delivered until.

According to one embodiment, if a particular number of items are currently rented to the customer 102 and the particular item delivery criteria triggers the delivery of one or more additional items, the one or more additional items are delivered to the customer 102 and added The fee applies to the customer 102. The particular number of items may increase to reflect additional items delivered to the customer 102 and increase the size of the item held by the customer 102. Optionally, the particular number of items may maintain the same item and the number of items that remain to be returned to the previous level by the customer 102 after the items are returned to the provider 104 by the customer 102. When used in combination with the "max turn" approach described below, a certain number of items may not be limited.

A “max out” approach to managing the number of items that can be loaned to customers at the same time is described with reference to the flowchart 300 of FIG. 3. Beginning at step 302, at step 304, one or more items are delivered to the customer 102 to form articles held by the customer 102. If no initial delivery of the items is required, the article of customer 102 is formed incrementally over time in accordance with one embodiment of the present invention.

In step 306, it is determined whether the item delivery criteria are met. If not met, the decision continues until the item delivery criteria are met. As mentioned above, the delivery criteria may generally include customer notification, customer notification that the item is returned, actual item return, occurrence of a particular due date, or specific overdue period.

If the item delivery criteria is met in step 308, it is determined whether a certain number of items are delivered. If not, control returns to step 304 and one or more additional items are transferred to the customer 102 by the provider 104. However, if at step 308 a certain number of items are delivered, then at step 310 it is determined whether a certain number of items, the "max out" limit, is ignored. As mentioned above, a certain number of items can be ignored by increasing a certain number of items, ie, "max out" limits, causing additional items to be delivered to the customer 102 and billed to the customer 102. . Optionally, a certain number of items are not charged and are overburdened by customer 102. The process continues for the duration of the subscription and ends at step 312. V. "Max Turn"

According to one embodiment, the "max turn" approach is used to rent items to customers. According to the "max turn" approach, the exchange of up to a certain number of items may be performed for a certain period of time. For example, referring to FIG. 1, suppose that the provider 104 agrees to rent an item to the customer 102 with a "max turn" limit for three items per month. This means that the customer 102 can exchange up to three items per month. The approach can be implemented according to the number of items a customer can rent at any given time under the "max out" approach. The approach is determined by the particular item delivery criteria used.

In one embodiment, the "max turn" approach is performed in combination with the "max out" approach to renting items to customers. In that case, up to a certain number of all items are simultaneously loaned to the customer 102, and the exchange of all items up to a certain number may be performed for a certain period of time. Thus, using the "max out" and "max turn" approaches together is a personal item for the customer 102 based on the "max out" limit that can be refreshed periodically based on the selected "max turn" limit. Form inventory.

In some cases, customer 102 may wish to exchange a certain number or more of items during a particular time period. According to one embodiment, in this case, the provider 104 agrees to rent a certain number or more of the additional items to the customer 102 and bill the customer 102 for the additional items. For example, suppose that the provider 104 agrees to rent items to the customer 102 with an opportunity (exchange) of up to three items per month. If a particular month, the customer 102 requires two additional items, two additional items are provided to the customer 102 and the cost for the two additional items is weighted to the customer 102.

In other cases, customer 102 may not use all assigned opportunities for a particular time period. In one embodiment, customers lose opportunities not used during the subscription period. For example, if customer 102 has a "max turn" limit for four item exchanges in a month, and only performed two item exchanges in a particular month, two unused exchanges are lost and cannot be used. . At the beginning of next month, the customer 102 may be eligible for four new item exchanges.

According to another embodiment, customers are allowed to perform unused opportunities in subsequent subscription periods. For example, if customer 102 has a "max turn" limit for four item exchanges in a month, and only performs two item exchanges in a particular month, two unused exchanges may be lost and used. none. At the beginning of next month, customer 102 may be eligible for six new item exchanges, two from the previous month and four from the current month.

A “max turn” approach for renting items to a customer is described with reference to flow diagram 400 of FIG. 4. Beginning at step 402, in step 404 the customer 102 and the provider 104 agree to the “max turn” negotiation terms. In particular, the customer 102 and provider 104 at least agree to the maximum opportunities allowed within a particular time period.

In step 406, in response to the one or more item delivery criteria being met, the provider 104 provides one or more items to the customer 102 via the delivery channel 108. Some item delivery criteria may be used with the "max turn" approach, and the present invention is not limited to specific delivery criteria. For example, the initial one or more items may be paid for a subscription fee performed by the customer 102 to the provider 104, in response to the initiation of a particular subscription period, or at the request of the customer 102 for the initial rental item. 102). The number of initial one or more items does not exceed the "max turn" agreement.

In step 408, in response to the one or more item delivery criteria being met, it is determined whether additional items are provided to the customer 102 within the "max turn" consent condition. For example, if the number of items loaned to the customer within the current subscription period is less than the negotiated "Max Turn", additional items may be rented to the customer 102 within the "Max Turn" consent condition. In that case, the determination may be performed by the customer 102 in response to the customer 102 returning one or more items to the provider 104 or requesting additional items.

If at step 408, a determination is made that additional items may be rented to customer 102 within the "max turn" consent condition, control returns to step 406 where one or more additional items are returned to customer 102. It is rented to). However, if a determination is made in step 408 that additional items cannot be loaned to the customer 102 within the "Max Turn" negotiation condition, then in step 410 it is determined whether to ignore the current sync conditions. If disregarded, at step 412 the terms of consent are changed to allow a greater number of conditions and the customer 102 is charged the corresponding cost, or the conditions are left unchanged and the additional items to be delivered. The cost is increased. Control returns to step 406 and one or more additional items are passed to the customer 102.

In step 410, if a determination is made that the current consent is not ignored, then in step 414, no items are delivered to the customer 102 until the next subscription period. For example, a request for additional items may be received at the end of a subscription period, and items may be delivered for subsequent subscription periods instead of immediately renting additional items. Control returns to step 406 where one or more additional items are loaned to the customer or the process ends at step 416.

The method for renting items described herein is described in connection with renting audio / video (A / V) items to customers such as movies, games and music stored on various media. 5 is a diagram illustrating a set of customers 502 wishing to rent A / V items from a provider 504. Customers 502 communicate with provider 504 via links 506 and links 510, which are global packet-switching networks referred to as the “Internet”.

Links 506 and 510 may be any medium for transferring data between customers 502 and the internet 508 and between the internet 508 and provider 504, respectively, It is not limited to any particular medium. In this example, links 506 and link 510 may be connections provided by one or more Internet service providers (ISPs) and customers 502 are configured with common Internet web browsers. Links 506 and 510 may or may not be secure, depending on the needs of the particular application.

According to one embodiment, customers 502 rent audio / video (A / V) items 512 from provider 504 according to the “max out” and / or “max turn” method described herein. The contract with the provider (504) for the risk. The invention is not limited to any particular method for making a rental agreement. For example, customers 502 and provider 504 may enter into a rental agreement over the Internet by customers 502 logging to mail, phone or a website associated with provider 504.

Customers 502 generate item selection criteria and provide them to providers 504 via links 506 and 510 and the Internet 508. The present invention is not limited to any particular method for specifying item selection criteria and providing to the provider 504. For example, according to one embodiment, customers 502 provide item selection criteria to provider 504 in one or more data files. According to another embodiment, the customers 502 log on to the provider 504's website and use a graphical user interface (GUI) to specify the attributes of the movies and music that they want to rent from the provider 504. use.

The item selection attributes may include, at least in part, any attributes that describe the music, games or music that the customers 502 want to rent. In movies, exemplary attributes may include, but are not limited to, title, category, director's name, actor's name and release year. In the game, exemplary attributes may include, but are not limited to, a title and a category. In music, exemplary attributes may include, but are not limited to, title, category, artist / group name, and year of release. Customers 502 may identify specific movies or music by item selection criteria or may provide various attributes and allow the provider 504 to automatically select specific movies and music that satisfy certain attributes. . For example, customers 502 can specify an item selection criterion that includes a horror movie released in 1999 to cause provider 504 to automatically select the horror movies that were released in 1999. In another example, customers 502 can specify item selection criteria including adventure movies starring Harrison Ford. Customers 502 may also specify an order or priority for the selected item selection criteria. For example, customers 502 may receive these movies by specifying certain movie titles and specifying the order in which they desire. In another example, customers 502 can specify a particular number of different types of movies they wish to receive.

If customers 502 and provider 504 enter a rental agreement and customers 502 provide item selection criteria to provider 504, then A / V items 512 are delivered to the delivery channel in accordance with the terms of the rental agreement. Through 514 to the customers 502. Specifically, in accordance with the "max out" method described herein, an initial set of A / V items 512 such as movies, games, and music may be directed to delivery channels 514 in accordance with the terms of the rental agreement. Is communicated to customers 502. The next A / V items 512 are delivered each time the specified item delivery criteria is met. For example, upon return of one or more A / V items 512 to a provider, a request from customers 502, such as the arrival of a particular date, such as a particular day of a particular month, for example, expiration of a particular period of time, such as 15 days. If appropriate, additional A / V items 512 may be delivered.

According to the "max out" method described herein, if the maximum number of A / V items 512 are loaned to a particular customer 512, the one or more loaned A / V items 512 may be provided to the provider 504. No further A / V items 512 are loaned until they are returned to or until an additional fee does not apply to a particular customer 502. Alternatively, the rental agreement between a particular customer 502 and provider 504 may be modified to increase the maximum number of A / V items 512 that can be loaned to a particular customer 502 at the same time.

The rental agreement between the customers 502 and the provider 504 may also specify the maximum number of turns in combination with the "max out" method. In this situation, the maximum number of turns limits how quickly customers 502 can update their A / V item 512 lists. For example, assume that a particular customer 502 has contracted with provider 504 to rent four movies for up to four times per month. Under this agreement, a particular customer 502 may maintain a personal list of up to four movies and rent four new movies each month. Thus, a particular customer 502 can completely "replace" his personal list once a month. If a particular customer 502 contracts for up to eight rentals per month, the particular customer 502 can completely replace his personal list twice a month.

The provider 504 is shown as one entity for illustrative purposes only. The provider 504 may be centralized or distributed depending on the needs of a particular application. For example, provider 504 may be a central warehouse where all A / V items 512 are provided. Alternatively, provider 504 may be implemented by a network of distributed warehouses.

6 illustrates a method for renting A / V items 512 such as movies, for example, to customers over a communication network such as the Internet using both "Max Out" and "Max Turn", according to one embodiment. It is a flow chart. Also, with respect to FIG. 5, after starting at step 602, at step 604, the customer 502 enters into a rental agreement with the provider 504. In this example, customer 502 uses a typical web browser to access an Internet web site associated with provider 504 and customer 502 maintains a list of four movies (four “max outs”) each month. Sign a rental agreement to receive up to four new movies (four "Max Turns"). In addition, the rental agreement is specified such that new movies are delivered upon receipt of the rented movie from the customer 502, i.

In step 606, the customer 502 generates and provides movie selection criteria to the provider 504 that indicates the movies that he wishes to rent. For example, the movie selection criteria may specify specific movie titles that the customer 502 wants to rent. The movie selection criteria may also specify the order or priority that the customer 502 wants to rent movies. Instead of identifying specific movie titles, movie selection criteria may specify movie preferences or any other movie preferences or attributes for the customer 502 such as, for example, the type of movies, directors, actors. In this situation, the provider 504 automatically selects specific titles that meet the movie selection criteria. For example, the movie selection criteria may specify preferences for action movies starred by a particular actor along with preferences for "newly released" movies. The provider 504 attempts to provide the clients 502 with movies that best meet the preferences indicated by the movie selection criteria.

At step 608, one or more first movies 512 are delivered to customer 502 via delivery channel 514. One or more first movies 12 may be delivered to the customer 502 by mail, courier, delivery agent or any other suitable means negotiated between the customer 502 and the provider, but the present invention It is not limited to any particular type of delivery mechanism. For illustrative purposes only, assume that movies in this example are delivered by mail between customer 502 and provider 504.

One or more first movies 512 establish a personal movie listing of the customer 502. The customer 502 can choose to receive any number of movies up to four movies, which is a "max out" limit. Typically, customer 502 will choose to receive four movies initially in the initial delivery.

Once one or more first movies 512 are delivered to the customer 502, a determination is made at step 610 as to whether any movies 512 have been returned to trigger another movie delivery. In this example, delivery of additional movies is triggered by receiving one or more movies from customer 502, such as via mail. In the situation where the customer 502 chooses not to receive the maximum number of movies 512 on the first delivery, the delivery of additional movies 512 is also the request of the customer 502 for additional movies 512. Can be triggered by For example, customer 502 may notify provider 504 via phone, email, or by accessing a website associated with provider 504.

If it is determined in step 610 that one or more movies 512 have been received from the customer 502, then in step 612 a determination as to whether the maximum number of turn (“max turn”) limits have been reached for the current cycle. This is done. In this example, a determination is made whether four or more movies were delivered by mail this month. If four or more movies were not mailed this month, control flow returns to step 608, where one or more additional movies 512 are up to four delivery channels 514 with a "max out" limit. Is delivered to the customer 502 by mail.

If at step 612 a determination is made as to whether the "max turn" limit has been met for the current cycle, i.e. in this example four movies 512 were mailed to the customer 502 this month, In step 614, a determination is made whether to exceed the current "max turn" limit. If so, then in step 616 the additional charge is applied to the customer 502 and the control flow returns to step 608 where additional movies 514 are mailed to the customer 502. If not, a decision is made at step 618 whether to continue to provide subscription services. If so, no additional movies are mailed to customer 502 during the current cycle, ie this month, and control flow returns to step 610. If at step 618 it is determined that no service continues to be provided, then the process is completed at step 620.

In some situations, the customer 502 may wish to increase or decrease the size of the customer's personal movie listing by changing the current "max out" limit. According to one embodiment, the customer 502 notifies the provider 504 that it wants to change the "max out" limit, for example by phone, mail, email, or by accessing a website associated with the provider 504. do. The movie rental agreement between customer 502 and provider 504 is then modified to reflect the change in the "max out" restriction. If the "max out" limit is increased, additional movies 512 can be mailed to the customer 502 immediately. VI. List management

The method described herein for renting items to customers provides superior list management capabilities over previous methods. Specifically, the use of item selection criteria provides efficient list management by allowing the maximum number of items to be rented at any given point in time. In addition, most customers are provided with their favorite items. For example, customers can specify priorities for items indicated in the item selection criteria. Thus, if the first choice of a particular customer is not available or already rented, the next highest priority item can be rented to the particular customer. According to one embodiment, customers may display items that are not yet available for rental. The items are delivered to customers when they become available.

For example, again with respect to FIG. 5, assume that a particular customer 502 wants to rent a movie with a title of "ABC" that has not yet been released. The particular customer 502 presents the movie to the provider 504 by item selection criteria. Since movie ABC is not yet available, the movie cannot be delivered to a particular customer 502. However, once movie ABC becomes available, the movie can be immediately delivered to the particular customer 502 as well as to other customers 502 who have requested the movie. This allows the provider 504 to maximize the number of items rented while allowing customers to rent the highest priority items they have requested.

According to another embodiment, items that are not yet known may also be rented by specifying attributes of the unknown items. For example, a particular customer 502 may request a rental of the next new movie of a particular director whose exact name is not known to that particular customer. As another example, a particular customer 502 may request to rent the next album of a particular group that is currently being prepared but not yet named. VII. Item Recommendation A. Overview

In some cases, it is desirable to know how a user evaluates an item. For example, with respect to recommending items to users, it is useful to know how a user evaluates an item, so that a decision may be made whether or not to recommend the item to the user. If the user has a high probability of appreciating the item, the recommendation for the item may be more important to the user, since the user is more likely to receive and like the recommendation. A method is provided for estimating how a user would rate an item. The method generally includes estimating how a user would rate an item based on similarities between items and similarities of ratings given for items by both the user and one or more other users. The method is applicable to any type of item, such as rental items such as movies and games, and the invention is not limited to any particular type of item. The method is broad and applicable to a variety of relationships and can be used, for example, in a recommendation engine for recommending items to users.

7 is a flowchart 700 illustrating a method for estimating how a user would rate a particular item in accordance with one embodiment of the present invention. In step 702, one or more items are identified that have ratings similar to a particular item from the plurality of items. In step 704, one or more other users are identified with given ratings for the one or more other items that are substantially similar to the given ratings for one or more other items by the user, and the one or more other users are also identified Evaluate the item. In step 706, an estimation of how a user evaluates a particular item is made based on ratings given by one or more other users for the particular item. Each of these steps is described in more detail below.

8 is a block diagram illustrating user ratings of items arranged in the array 800. Array 800 includes columns 802 of items and rows 804 of users. Each cell may be empty or contain data indicating that the user has not provided a rating for the item. For example, certain values, such as null values or other values, can be used to indicate that the user has not provided a rating for the item. Alternatively, each cell may contain data indicative of a user rating of the item. Depending on the needs of a particular application, different types of data and values may be used to indicate the user rating of the item, and the present invention is not limited to any particular data types or values. For example, the grades may be represented by numeric data in the range of 1 to 5 corresponding to the 5-star scale scale or binary data corresponding to the preferred or unfavorable grade. Embodiments of the invention are described herein in connection with user classes in the array 800 for illustrative purposes only, and the invention is not limited to user classes maintained in any particular format or data structure. In addition, the relative size of the range of items 802 and users 804 described in FIG. 8 does not mean that there are more users 804 than items 802. Embodiments of the present invention are described herein in connection with generating an estimated rating for a particular item 806 from items 802 and a particular user 808 from users 804. The estimated rating for a particular item 806 and a particular user 808 is indicated in FIG. 8 by reference numeral 810. B. Identification of other items with similar ratings to a specific item

A set of similar items 812 with similar ratings to a particular item 806 is identified. Similar items 812 are shown in FIG. 8 to be placed in proximity to a particular item 806 of the array 800 for illustrative purposes only. Similar items 812 may include any items 802. Depending on the needs of a particular application, various techniques may be used to determine similar items 812, and the present invention is not limited to any particular technique.

According to one embodiment of the present invention, similar items 812 are items in items 802 that satisfy an item similarity criterion with respect to a particular item 806. Thus, other items that meet the item similarity criteria with respect to the particular item are considered similar to the particular item. Similarity does not necessarily require symmetry. Thus, although item A may be considered similar to a particular item 806 and may be included in similar items 812 for a particular item 806, the particular item 806 is considered similar to item A and may It may or may not be sufficiently similar to item A to be included in the set of similar items for. The number of items in the similar items 812 may be limited to a specific number to reduce the amount and time of computational resources required to generate an estimated rating for a particular item 806. For example, a particular number of items in similar items 812 indicates a first specific number of items that meets an item similarity criterion for a particular item 806 or a similarity higher than some specified level or threshold. In this example, the specific number of items satisfying the item similarity criterion for the specific item 806 may be displayed.

여기서, X_i는 하나의 아이템 또는 사용자에 대한 등급들이고 Y_i는 다른 아이템 또는 사용자에 대한 등급들이며, 합산이 모든 공통 등급들을 통해 적용된다. 다른 예로서, 벡터 코사인 거리가 아래의 수학식 (2)에 도시된 바와 같이 통계적 유사성 측정으로서 이용될 수 있다:

여기서, X_i와 Y_i는 하나의 아이템 또는 사용자에 대한 등급들이다. 아래의 표 I는 아이템들 X 및 Y에 대한 예시적인 사용자 등급들, X 및 Y에 대한 사용자 등급들의 곱, X 및 Y에 대한 사용자 등급들의 제곱 및 이러한 값들의 합을 포함한다: 표 1

X 및 Y 열들의 합들은 아이템 X 및 아이템 Y 모두를 평가하였던 사용자들에 대한 등급들만을 포함한다는 것을 유의하도록 한다. 피어슨 상관 계수는

이고, 벡터 코사인 거리는

이다. Item-like criteria may include a wide variety of criteria depending on the needs of a particular implementation, and the invention is not limited to any particular item-like criteria. One example of an item similarity criterion is a statistical correlation threshold. Thus, two items that meet a statistical correlation threshold are considered similar to each other. A wide variety of statistical measures may be used for this purpose, depending on the needs of a particular implementation, and the present invention is not limited to any particular statistical similarity measure. For example, the Pearson correlation coefficient can be used as a statistical similarity measure as shown in equation (1) below:

Where X _i is ratings for one item or user and Y _i is ratings for another item or user, and the summation is applied through all common ratings. As another example, the vector cosine distance can be used as a statistical similarity measure as shown in equation (2) below:

Where X _i and Y _i are ratings for an item or user. Table I below contains example user ratings for items X and Y, the product of user ratings for X and Y, the square of user ratings for X and Y, and the sum of these values: Table 1

Note that the sums of the X and Y columns only include ratings for the users who rated both item X and item Y. Pearson's correlation coefficient

, The vector cosine distance is

to be.

만일 양 아이템 A 및 B의 공통 평가자들의 실제 수 R_ab가 E(R_ab)보다 훨씬 크면, 아이템 A 및 아이템 B는 임의의 방식으로 관련될 수 있다. 다른 한편으로, 만일 R_ab가 E(R_ab)보다 훨씬 작으면, 아이템 A 및 B는 아이템 A의 평가자들이 아이템 B와 만나지 않을 수 있거나 또는 아이템 B에 대한 의견을 가지지 않을 수 있으며 아이템 B를 낮게 평가하는 방식으로 다를 수 있다(역도 가능함). 이들 정보 피스 중 어느 하나는 부가 유사성 기준으로서 사용될 수 있다. 만일 아이템 A 및 B 중 하나가 다른 것보다 새로우면(다시 말해서, 다른 아이템이 일반적이거나 또는 알려질 때 존재하지 않으면), 아이템들의 공통 등급들의 전체 수는 앞의 기본 수식에 의하여 예상된 것보다 훨씬 작을 수 있다. 변형예로서, 등급들의 풀(pool)은 시간, 예컨대 월 정도의 슬라이스들로 재분할될 수 있으며, 앞서 기술된 공통 평가자들 분수 계산은 각각의 연속(월) 슬라이스에 적용될 수 있다.Another example of item similarity criterion is a particular number of common classes between items. When a user evaluates both items, there is a common rating for the two items. Thus, the item similarity criteria may specify the minimum number of common classes that must be shared between two items for two items to be considered similar items. Another example of an item similarity criterion is a specific fraction of common ratings between items divided by the number of ratings of an item. Thus, the item similarity criterion may specify a minimum fraction of common ratings between two items divided by the number of ratings of any one item to be satisfied for two items to be considered similar items. If the total population of users is U, the number of users who rated item A is R _a and the number of users who rated item B is R _b , and if the probability of user rating item A is independent of the probability of user rating item B Then the expected number of common classes E (R _ab ) is given by:

If the actual number R _ab of the common evaluators of both item A and B is much greater than E (R _ab ), item A and item B may be related in any way. On the other hand, if R _ab is much smaller than E (R _ab ), item A and B may not have item A's evaluators meet item B or have no opinion on item B and lower item B It can be different in the way they are evaluated (reversely possible). Any one of these pieces of information can be used as an additional similarity criterion. If one of items A and B is newer than the other (that is, not present when the other item is generic or known), the total number of items' common grades will be much smaller than expected by the preceding basic formula. Can be. As a variant, the pool of grades may be subdivided into slices of time, such as months, and the common evaluator fraction calculation described above may be applied to each consecutive (month) slice.

In some situations, using the Pearson correlation coefficient to measure the similarity between two items can make two different items look similar. This occurs at locations where the common ratings of the two items appear to be correlated due to the lower rating distributions of the two items. For example, two items may have multiple high grades. Although the items are not related, a group of rating pairs of both items among common raters, i.e., the user who rated both items, makes them appear relevant.

According to one embodiment of the invention, the rating distributions of the two items are factored. This is done by taking the percentage of each grade for each item and multiplying the percentage taken in the two-dimensional array to obtain the expected percentage for all possible pairs of grades r1-r2 for two items. . Multiplying each value of the array values by the total number of common evaluators produces an expected count of common evaluators for each class pair when the items are not related. The actual counts of common classes in each cell of the array cells are compared to see if they are larger or smaller than expected. Values higher than the expected value in the area corresponding to the similarity, such as a user evaluating both items at high speed or at low speed, indicate that the items are similar. Higher than expected common evaluator counts in domains where a user likes one item and dislikes another item indicates that the items are definitely related. Using the lower grade distribution for all evaluators makes the values for common evaluators easier to understand.

The foregoing descriptions identified several different similarity criteria that can be combined to determine overall similarity. For example, in one embodiment, the Pearson's correlation coefficient requires to be higher than a threshold (eg, 0.3), where the fraction R _ab of the common evaluators is at least any fraction 1 in the expected number of common evaluators E (R _ab ). / 2) and the absolute number of common evaluators must be at least any minimum number 200.

The methods described above show how to calculate the similarity between two items to find a set of similar items. Exactly the same methods can be applied to calculate the similarity between two users to find a set of similar users.

In some circumstances, there may be insufficient rating data to determine a set of items similar to a particular item. For example, if certain items are new, there may be very little data that allows correlations to be calculated. In this case, other semantic information can be used to determine similar items.

In one embodiment, items are further classified by semantic feature values. For example, if the items are movies, the semantic features may be genre (or category), director, male lead / female starring actors, other male and female actors, etc., and the feature values may be genre, director or actor name admit. In the music domain, the features may be style, soloists, musicians, and songwriters. Each type of feature is provided with a importance score that can be positive, negative, or zero, and the two items can be compared by counting the number of matching features and summing their importance scores. For example, if a movie director has a importance score of 3 and a male lead actor has a importance score of 1, a pair of films sharing one director and one male lead actor has a similarity score of 4, while the two actors The other pair of movies shared has a similarity score of two. The absence of matches may follow a negative score (eg, the matching member of directors scores pair -0.1 points). A set of similar items is a set number of items with a high similarity score with specific items or items whose similarity score is above a certain threshold.

In another embodiment, tools for specifying item similarity by non-mechanical means may be provided to domain experts. Once additional class data is made available, similar items designated by domain experts may be replenished or discarded. C. Identification of other users who have rated other items similar to a specific user.

Once similar items 812 are determined, one or more users of users 804 with ratings given to similar items 812 and ratings given to similar items 812 by user 808 are identified. do. All of the users 804 can be evaluated for this purpose. However, in any application where the number of users 804 is large, the computational cost may increase significantly when evaluating all of the users 804. In addition, some users may be less useful than others because they have very small classes or grade data with good statistical characteristics, as described in more detail below.

According to one embodiment of the invention, the subset of users 804 referred to herein as reference users 814 is determined based on specific user selection criteria. Reference users 814 are users from all users 804 that meet specific user selection criteria. Various user selection criteria may be used according to the requirements of a particular application, and the present invention is not limited to any particular user selection criteria. According to one embodiment of the invention, the user selection criteria include a minimum number of ratings. Thus, users of users 804 who do not have at least the minimum number of ratings are not included in reference users 814. The user selection criteria may include a maximum number of ratings. Thus, users of users 804 with more than the maximum number of ratings are not included in the reference users 814.

According to another embodiment of the invention, the user selection criteria specifies an average rating range. Thus, users of users 804 with average ratings outside of a certain range are not included in reference users 814. For example, it is assumed that grades are provided on a numerical scale of 1-5. It is also assumed that the user selection criteria specify that the tolerance average rating range is 2-4. In this situation, users 804 with an average rating less than two or greater than four are not included in the reference users 814.

According to another embodiment of the invention, the user selection criterion is that the user ratings must follow a certain tolerance of the normal distribution. Thus, among users 804 who have a rating distribution that does not approximate a normal distribution within a certain tolerance range, users are not included in the reference users 814. For example, it is assumed that the grades are on a numerical scale of the entire number from 1 to 5. Users who have one rating more than two ratings, two ratings three ratings, five ratings four ratings, or four ratings three ratings or more Not included in field 814. Specific tolerances may vary depending on the requirements of a particular application. For example, some applications do not have one more than three ratings of one of the reference users 814 but have two more ratings than three ratings and also have the same number of two ratings and one rating. Certain tolerances may be used to ensure that

According to another embodiment of the invention, the user selection criteria specify a range of acceptable standard deviations of user classes. Thus, users of users 804 with standard deviations of grades that are not within the acceptable range of standard deviations of grades are not included in reference users 814. For example, the user selection criteria may specify a standard deviation range of 10% to 20%. Thus, a user with a standard deviation of grades less than or equal to 10% or greater than 20% is not included in the reference users 814. D. Estimation of how a particular user evaluates a particular item

Generating an estimate of how a particular user 808 evaluates a particular item 806 may include identifying which users 804 have similar items 812 similar to the particular user 808. And generating an estimate based on how the users 804 evaluate the particular item 806. In a situation in which reference users 814 are identified from users 804 as described herein, the method allows any reference users 814 to evaluate similar items 812 for a particular user 808. Identifying an estimate and generating an estimate based on how the reference users 814 evaluate the particular item 806. Embodiments that generate an estimate regarding how a particular user 808 evaluates a particular item 806 are used herein in connection with using reference users 814 although the invention is not so limited.

According to one embodiment of the invention, the ratings of the reference user 814 of the similar items 812 are determined by the identification of the similar items 812 to identify the reference users 814 most similar to the specific user 808. The user 808 ratings are compared. In FIG. 8, reference users 814 that have provided ratings for the most similar items 182 with ratings for similar items 812 provided by a particular user 808 are similar reference users 816. As mentioned below. In FIG. 8, the horizontal lines shown for similar items 812 for similar reference users 816 are provided for illustrative purposes only and similar reference users 816 for all similar items 812. This does not mean that the ratings must be provided. Similarly, it is not necessary for a particular user 808 to evaluate all similar items 812.

According to one embodiment of the invention, rating similar criteria is used to identify similar criteria users 816. The rating similarity criteria may be similar to the item similarity criteria described herein. For example, similar reference users 816 may determine the statistical correlation between the reference users 814 and the ratings of similar items 812 provided by the particular user 808 and best correlate with the particular user 808. It may be determined based on selecting similar reference users 816 based on the reference users 814 being. Similarity between similar items 812 and specific items 806 may be considered when identifying similar reference users 816. In particular, the rating of the items 812 most similar to a particular item 806 may be provided with greater weight when determining similar reference users 816 from the reference users 814.

Once similar reference users 816 are identified, an estimate as to how a particular user 808 evaluates a particular item 806 depends on the ratings of the particular item 806 made by the similar reference users 816. Is made on the basis of The degree of similarity between the similar reference users 816 and the particular user 808 is determined by the number of similar items 812 evaluated by the particular user 808 and the similar reference users 816 and the specific user 808 and similar criteria. It can change based on various factors including how users 816 evaluate similar items 812. The estimate may be generated in different ways depending on the similarity between the similar reference users 186 and the rating of the similar items 812 provided by the particular user 808. For example, there may be situations where ratings provided by one or more similar reference users 816 for similar items 812 are the same as ratings provided by a particular user 808 for the same similar items 812. . This may occur, for example, when the number of similar reference users 816 and similar items 812 evaluated by a particular user 808 is very small. In such a situation, the rating of a particular item 806 estimated by a particular user 808 may be an average rating made by one or more similar reference users 816 for the particular item 806.

In situations where the similar criteria users 816 and similar items 812 by the particular user 808 are not the same, the rating of the particular item 806 estimated by the particular user 808 is determined by the particular item ( 806 may be a weighted linear sum of the ratings made by one or more similar reference users 816. The applied weight changes based on the similarity between each similar reference users 816 and a particular user 808. According to one embodiment of the invention, the weighted linear sum of the ratings for a particular item 806 is the sum of the products of similar reference user 816 ratings and weights of the particular item 806 divided by the weights. to be. Thus, the ratings of the particular item 806 provided by the reference users 816 most similar to the particular user 808 may be determined by the ratings of the particular item 806 provided by the reference users 816 that are less similar to the particular user 808. Greater weights are given than grades. For example, assume that similar reference users 816 include users A and B that are determined to have similarities of 0.8 and 0.5 (for a scale of 0-1) for a particular user 808, respectively. In addition, users A and B have given ratings for a particular item 806 of 5 and 3 (for a scale of 1 to 5), respectively. The average rating of a particular item 806 by users A and B is 4 ((5 + 3) / 2). The weighted linear sum of the ratings for a particular item 806 by users A and B is 4.23 ((5 * 0.8 + 3 * 0.5) /1.3). The weighted linear sum of the ratings is given to User A's high rating because a weight greater than User B's low weight is given by a relatively greater similarity between User A and a particular user 808 than between User B and a particular user 808. In this situation it is larger than the average of the ratings.

Various trust metrics may be calculated that indicate a certain level of confidence level of a particular item 806 by a particular user 808. According to one embodiment of the invention, the confidence metric is calculated based on the imbalance or spread between the estimated ratings, where each estimated rating is based on another user of similar reference users 816. The standard deviation of the estimated grades can be used for this purpose. Reliability calculations based on the imbalance between estimated ratings may weight these estimated ratings based on the strength of similarity between the reference user 816 and the particular user 808. The confidence metric may be formed based on similarities between a particular user 808 and each similar reference user 816, which are used to estimate the rating. In addition, the calculation of confidence may take into account the absolute strength of similarity between the reference users 816 and the particular user 808 used to generate the prediction.

There may be situations where there is an insufficient (or zero) number of similar reference users 816. In these situations, the weighted average of all reference users 814, each sharing a rating of at least similar items 812 with a particular user 808 and each evaluating a particular item 806, has a specific user 808. ) May be used to generate an estimate as to how to evaluate a particular item 806. According to one embodiment of the invention, the weights are applied to a particular user 808 based on the similarity of each reference user 814. Confidence metrics may be applied to generate an estimate regarding how a particular user 808 evaluates a particular item 806 based on all reference users 814 used to generate the estimate.

In situations where there is an insufficient (or zero) number of reference users 814, which may occur when a particular user 808 has an insufficient (or zero) number of ratings, the particular item 806 may be removed. The average rating of a particular item 806 by all the reference users 814 that have been rated may be used as an estimate of how the particular user 808 evaluates the particular item 806. Optionally, the average rating of all users 804 who rated a particular item 806 may be used as an estimate of how a particular user 808 evaluates a particular item 806. In these situations, the weighted average of all reference users 814 who each share a rating of at least one of the similar items 812 with the particular user 808 and each rated the particular item 806 will be used. Can be. E. Recommended Structure

9 is a block diagram illustrating a structure 900 for recommending items in accordance with an embodiment of the present invention. The structure 900 includes a web server 902 communicatively connected to the recommendation system 904. As described in more detail below, the recommendation system 904 is configured to process ranking data using prioritized tasks to improve the usability and accuracy of item recommendations. The recommendation system 904 includes a recommendation engine 906, a nonvolatile storage 908, and a rating database (DB) 910. The recommendation engine 906 may be implemented by any mechanism or any combination of hardware and software that performs the functions described herein. For purposes of illustration only, recommendation engine 906 is described in connection with a software process running on non-volatile memory. Nonvolatile storage device 908 may be implemented by any type of nonvolatile storage mechanism, and the present invention is not limited to any particular mechanism. Example implementations include, but are not limited to, nonvolatile memory and disk storage. Class DB 910 may be implemented by any database management mechanism, and the present invention is not limited to any particular implementation.

The recommendation engine 906 maintains ratings and prediction data 912 including the user's actual ratings and estimated ratings calculated according to the algorithms described herein or in other manners. Ratings and prediction data 912 may be maintained in volatile memory, such as random access memory (RAM), to reduce the frequency of access. According to one embodiment of the invention, the grades and prediction data 912 are maintained in a RAM array and addressed using 64-bit addressing as provided by high end computational architectures. The array is configured such that two bytes are used for each cell including a rank value, an estimated rank value, a confidence value, and one or more flags indicating the presence and validity of the values. The RAM array can be considered to be divided into 8,192 byte pages (corresponding to the physical structure of current computers). The cells are configured such that the user's ratings and estimated ratings span one or more pages for access locality.

According to one embodiment of the invention, rating and prediction data 912 is periodically stored in a data file 914 maintained on non-volatile storage 908. For example, a low level computing system may be used to ensure that data written to the RAM array is stored in the data file 914 in an appropriate manner, such that the recommendation engine 906 may select a portion of the ratings and prediction data 912. It can be shut down periodically for maintenance and upgrades without loss. Upon restarting the recommendation engine 906, the data from the data file 914 is remapped (reloaded) to the RAM array, thereby restoring all the results of the previous calculations. In one embodiment of the invention, the memory mapping function of the Solaris computing system provided by Sun Microsystems, Inc. causes the data file 914 to be sparsely placed in disk blocks, thus allowing values to be placed. Any pages in the RAM array that do not have do not occupy space on nonvolatile storage 908. This makes it possible to save significantly on hardware because a large amount of RAM arrays never contain ratings and predictive data. Class data is stored in class DB 910, which allows data file 914 to be regenerated in the event of a non-volatile storage failure. Class DB 910 may store any data contained in RAM array or data file 914 or any other data used by recommendation system 904 depending on the requirements of a particular implementation.

The use of data file 914 allows for storing more grades and prediction data 912 than can be limited to a RAM array given the actual limits given to RAM available on computer architectures. According to one embodiment of the invention, the size of the RAM array is selected large enough to store ratings and prediction data 912 for at least reference users 814 and active users of the recommendation system 904. As the user's state changes from active to inactive, the physical storage pages of the RAM array assigned to the user's rating and prediction data 912 can be replayed and reused for other users who have just been activated.

The recommendation engine 906 is configured to perform various tasks. Such a task may be prioritized in one or more work queues 916 to coordinate the activation of one or more processing threads. One or more work queues may be stored in volatile storage, such as RAM, or non-volatile storage 908. The processing thread processes the job from the job queue 916 based on the priority, and the processing of the job may create additional queues that are added to the job queue 916. For example, the recommendation engine 906 is configured to process a request for an estimated rating and a request for a recommendation. An estimated rating for a particular item for a particular user may be requested in situations where the user views data related to the particular item, such as a review, for example, on a graphical user interface, and an estimate of how the particular user evaluates the particular item. It is preferable to display the data indicating. For example, a recommendation request may be made in a situation where a user may request a list of recommended items from a web application. When a request for rating the rating of a particular item for a particular user is received from the web server 902, the recommendation engine determines if the requested rating is previously calculated and stored in the RAM array. If so, the estimated rating is immediately returned to the web server 902. If not, the request is queued for a work queue with a high priority to calculate the estimated rank (according to the described algorithm, etc.). If the computing thread completes the calculation of the estimated rank within a predetermined small thread time (eg, 0.5 seconds), the calculated estimated rank is returned. Otherwise, recommendation engine 906 is configured to provide an average rating of the particular item, such as the estimated rating. When a recommendation request for a particular user is received from the web server 902, the recommendation engine 906 identifies the set of items with the highest estimated rating and highest trust for the user. If there is no confidence prediction, no recommendation may be returned. Although embodiments of the invention have been shown and described in the drawings in the context of a single web server 902, this is for illustrative purposes only. The recommendation system 904 may be used with any number of web servers and also other types of computing entities.

The recommendation engine 906 is configured to receive and process rating data from the web server 902. Upon receipt of rating data, which may include new or changed ratings, the recommendation engine 906 stores the ratings in the RAM array using simple address calculations and simple memory records. The rank is queued in the work queue that is recorded for DB 910. In addition, the estimated rating for similar items in the item associated with the received rating data may need to be updated to reflect the received rating data. Thus, one or more jobs are added to the job queue 916 to calculate the estimated ratings for items that are received similarly. In accordance with an embodiment of the present invention, such a task is given a high priority, which means that the updated estimated rating may be applied to, for example, a transaction decision or recommendation for a user who is presumably visiting a website to provide the rating. This is because it may be requested in a relatively short time. The quality metric may be calculated from the comparison of the actual user rating to the estimated user rating. For example, assume that the estimated rating 810 has been determined for a particular item 806 and a particular user 808. It is assumed that this estimated rating 810 has been provided to the web server 902 displayed for the particular user 808 and that the particular user 808 provided the actual rating for the particular item 806. The quality metric is calculated based on a comparison of the actual grade to the estimated grade 810. Together with other quality metrics, these quality metrics can be used for the fine tuning of future estimated grades.

The recommendation engine 906 is further configured to periodically recalculate the rating and prediction data 912 periodically and if necessary. In particular, the recommendation engine 906 is configured to calculate a set of similar items for each item to reflect changes in the rating data and the new rating data. Changes in the rating data and the new rating data may be tracked, and various thresholds and aging schemes may be used to prioritize the recalculation of similar item sets. For example, an updated relatively large amount of rating data simply gives the computed similar set of items simply prioritized over older items. Since rating data may change the set of items considered similar for a particular item, it is advantageous to periodically calculate a set of similar items for a particular item if no rating data has changed for that particular item.

There may be a situation where the limitation of the power of the computational structure used to implement the recommendation system 904 prevents the recommendation system 904 from following changes in rating data. That is, the estimated rating maintained by the recommendation engine 906 may lag behind the updated rating data. One consequence of this situation may be a loss in the accuracy of the estimated rating. There may be situations where restrictions on the available size of non-volatile storage 908 may make it difficult to maintain data file 914. In conclusion, according to another embodiment of the present invention, the solution is divided into a series of individual recommendation engines and individual disk arrays that enable scaling to even higher performance and load levels. 10 shows an embodiment of such an apparatus 1000.

Apparatus 1000 includes a recommendation system 1004 and a web server 1002 configured in accordance with an embodiment of the invention. The recommendation system 1004 includes a pseudo / reference engine 1006, a nonvolatile storage device 1008, a grade DB 1010, and a recommendation engine 1012, 1014.

The single reference user / similar item data file 1016 is used to store a rating for the reference user 814. The reference user / similar item data file 1016 is maintained on the nonvolatile storage device 1008 by a similar / reference engine 1006 that maintains a set of reference users 814. The reference user 814 is selected as described above. If too many users 804 are the reference user 814, the user 804 with the most interesting ranking pattern is selected as the reference user 814. Examples of standards for selecting reference user 814 include, without limitation, the highest rating, or the highest rating of other unrated items, or users with other standards that may be appropriate. New candidate reference users are loaded from rating DB 1010 as they define, and current reference user 814 is more interested in making certain candidate reference users more appropriate as reference users for creating estimated ratings. It may be transposed if it has an off parameter. Similar / reference engine 1006 is configured to calculate a set of similar items 812 for each item 802 on a continuous basis and store the data as reference user / like item data file 1016.

The recommendation engines 1012 and 1014 are configured to generate an estimate as to how the user rates the item, as described above. The recommendation engines 1012, 1014 have exclusive read / write access to activate the user data files 1018, 1020, respectively. Active user data files 1018 and 1020 store rating and expected data for users assigned for recommendation engines 1012 and 1014. Recommendation engines 1012 and 1014 also have read-only access to the reference user / similar item data file 1016. Read-only access is cost effective on resources on disk systems and computer IO systems except when recommendation engines 1012, 1014 are initiated. In an alternative embodiment, the reference user / similar item data file 1016 copies one or more items periodically, and the final copy or copies are mapped by the recommendation engines 1012, 1024. The recommendation engines 1012 and 1014 estimate how the user evaluates the item as described to obtain a reference rating value from the reference user / similar item data file 1016 and to calculate the estimated rating for its individual read write activity. Stored in the user data files 1018 and 1020. The assignment of users to the recommendation engines 1012 and 1014 can be done using various techniques based on the needs of a particular implementation, and the invention is not limited to any particular manner or assignment of users. Moreover, the user's allocation to recommendation engines 1012 and 1014 can be dynamic and change over time, for example, depending on the loading request.

The active profile of the base user 814 of ratings and predictions is stored in one of the active user data files 1018, 1020, and the estimated rating for the base user 814 is calculated by the appropriate recommendation engine 1012, 1014. Are stored in corresponding active user data files 1018, 1020. If the reference user 814 stores his rating, this rating is also stored in the rating DB 1010. After some time, the similar / reference engine 1006 updates a threaded copy of the rating for the reference user 814 of the reference user / like item data file 1016. While the rating of the change in rating for the reference user 814 is small compared to the total number of ratings for the reference user 814, the quality of the estimated rating calculated using similar items and historical data will not be significantly affected. will be. The recommendation engines 1012, 1014 may store ratings and prediction data in one or more local RAM arrays, and periodically record ratings and prediction data in the active user data files 1018, 1020.

The work queue may be shared between one or more engines so that a new work item may be triggered by one recommendation engine, but executed by another engine if the work involves data managed by another engine.

Although embodiments of the present invention have been described in the context of using a single similarity / reference engine 1006, further scaling may be achieved using multiple similarity / reference engines, each of which is an assigned reliability for a portion of the reference user 814. May be provided. For example, even / odd items (or other item divisions) may differ in that the determination of similar item 812 needs to compare a column of ratings for pairs of items across all reference users 814. It is appropriate to split the reference users 814 by vertical segmentation by item so that it is split between / reference engines. In this situation, each portion of the rating data for each reference user 814 is useful for each similar / reference engine. In particular, each data file is owned by a single similar / reference engine with read / write access for maintenance, and each data file is read-only for all other similar / reference engines for access to reference grade data. It is useful for the foundation. VIII. Implementation mechanism

The described schemes for estimating how a user evaluates a particular item are applicable to certain types of items or content and are suitable for internet-based rental applications for renting movies, music, and games to customers. The invention may be implemented as hardware circuitry, computer software, or a combination thereof, and is not limited to any specific hardware or software implementation. The invention is ideally suited for recommendation engine applications and may be implemented as a standalone mechanism or integrated into the rental mechanism. For example, this approach is ideally suited for identifying rental items such as movies, music and games for recommendation to the user.

11 is a block diagram illustrating a computer system 1100 in which embodiments of the present invention may be implemented. Computer system 1100 includes a bus 1102 or other communication mechanism for communicating information, and a processor 1104 coupled with the bus for processing information. Computer system 1100 includes main memory 1106, such as RAM or other dynamic storage device, coupled to bus 1102 to store information and instructions to be executed by processor 1104. Main memory 1106 may also be used to store temporarily variable or other parameter information during execution of instructions to be executed by processor 1104. Computer system 1100 further includes a ROM 1108 or other static storage coupled to bus 1102 to store correction information and instructions for processor 1104. A storage device 1110, such as a magnetic disk or an optical disk, is provided and coupled to the bus 1102 to store information and commands.

Computer system 1100 may be connected via bus 1102 to a display 1112, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device comprising alphanumeric and other keys is coupled to bus 1102 to communicate information and command selections to processor 1104. Another type of user input device is a cursor controller 1116, such as a mouse, trackball, or cursor decision key, for communicating indication information and command selections to the processor 1104 and controlling cursor movement on the display 1112. Such input devices typically have two degrees of freedom in two axes, the first axis (e.g. x) and the second axis (e.g. y) allowing the device to specify a position in the plane.

The present invention relates to the use of computer system 1100 to implement the described techniques. In accordance with one embodiment of the present invention, this technique is executed by computer system 1100 in response to processor 1104 executing one or more sequences of one or more instructions contained in main memory 1106. These instructions are read into main memory 1106 from other machine readable media such as storage 1110. Execution of the sequence of instructions contained in main memory 1106 causes processor 1104 to execute the described process steps. In alternative embodiments, circuitry by means of wiring may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the present invention are not limited to any particular combination of hardware circuitry and software.

As used herein, the term "machine-readable medium" refers to any medium that participates in providing data that enables a machine to operate in a particular form. In embodiments implemented using computer system 1100, various machine readable media are involved in, for example, providing instructions to processor 1104 for execution. Such a medium may take many forms, such as, but not limited to, nonvolatile memory, volatile memory, and transmission media. For example, non-volatile media includes storage 1110 such as optical or magnetic disks. Volatile media includes dynamic memory as main memory 1106. The transmission medium includes a coaxial cable, a copper wire, and an optical fiber, including a wiring including the bus 1102. The transmission medium may take the form of acoustic or light waves such as those generated during wireless and infrared data communications.

Common forms of machine readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, or some other magnetic media, CD-ROMs, some other optical media, punch cards, paper tapes, perforations. Any other physical medium having, RAM, PROM, and EPROM, FLASH-EPROM, any other memory chip or cartridge, carrier described later, or any other medium readable by a computer.

Various forms of machine readable media may be included to convey one or more sequences of one or more instructions to the processor 1104 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the command into its dynamic memory and use the modem to deliver the command over the telephone line. A modem connected to computer system 1100 may receive data on a telephone line and use an infrared transmitter to convert the data into an infrared signal. The infrared detector can receive the data carried in the infrared signal, and appropriate circuitry can locate the data on the bus 1102. Bus 1102 delivers data to main memory 1106, and process 1104 retrieves and executes instructions from main memory. The instructions received by main memory 1106 may optionally be stored in storage 1110 before and after execution by processor 1104.

Computer system 1100 also includes a communication interface 1118 coupled to bus 1102. The communication interface 1118 provides two-way data communication that couples with a network link 1120 connected to the local network 1122. For example, communication interface 1118 may be an Integrated Services Digital Network (ISDN) card or modem that provides a data communication connection for a corresponding type of telephone line. As another example, communication interface 1118 may be a LAN card for providing a data communication connection to a bonded local area network (LAN). Wireless links may also be implemented. In this implementation, communication interface 1118 transmits and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

Network link 1120 typically provides data communication to other data devices over one or more networks. For example, network link 1120 may connect with a data device operated by a host computer or Internet service provider (ISP) 1126 via local network 1122. In turn, ISP 1126 provides data communication services over a worldwide packet data communication network, commonly referred to as “Internet” 1128. Local network 1122 and internet 1128 both use electrical, electromagnetic or optical signals that carry digital data streams. Signals over various networks and signals over a network link 1120 and over a communication interface 1118, which exchange digital data with a computer system, are examples of carrier forms that convey information.

Computer system 1100 may include program code to transmit a message or receive data via a network, network link 1120, and communication interface 1118. In the Internet example, the server 1130 may send the requested code for the application program via the Internet 1128, ISP 1126, local network 1122, and communication interface 1118. The received code may be executed by processor 1104 as received and / or stored in storage 1110 or other non-volatile storage for later execution. In this manner, computer system 1100 may obtain application code in the form of a carrier wave.

The new manner described herein for renting items to a customer allows the decision of which item to rent to be separated from the decision of when to rent the item. The customer may use the item selection standard to specify which items to rent and to receive the items at a future time, instead of going to the provider to select the items. The selection standard may be user specific or may include the required completion sequence. Moreover, the customer is not constrained by the usual date and can instead build a continuous series of rental streams of items. This approach enables more efficient list management.

The "max out" approach to list management allows a user to maintain a unique list of items that are periodically replaced by other items according to a particular event standard. The event standards that trigger delivering other items to the customer are very flexible and can meet the needs of a particular application. For example, as described, the event standard includes the return of certain items currently in use by the customer or only by customer notification. This is very convenient in the context of movie rentals, where returning a movie to a provider automatically triggers sending another movie to a customer. When used alone or in conjunction with "Max Out", the "Max Turn" approach to inventory management provides a great deal of flexibility for customers and providers. The maximum number of turns can be selected individually for a particular customer depending on the specific needs of the customer.

The "max out" and "max turn" approaches provide a great deal of flexibility in building the explanatory model to meet the needs of a particular application. In particular, the size and replacement of the customer list can be met for each customer with an individual subscription plan.

In the foregoing description, the invention has been described as applicable to implementations involving internet-based commands and other long-distance transport of mail or items, where the particular advantages of the method are very noteworthy. However, the same invention can be applied to more common video, game, or music rental storage settings, where a subscription number of movies is exchanged to subscription customers at regular times and / or during subscription periods, with no periodical return period for rentals. It can enable the rental of music, games, or music choices.

In the foregoing description, embodiments of the present invention have been described with reference to various specific embodiments that may vary from the implementation of the present invention. Thus, what is sole or exclusive indicator and what the applicant intends is defined in the claims of the present invention, including certain corrections. Accordingly, any limitation, element, property, feature, advantage, or attribute not disclosed in a claim, also defines the claims. The description and drawings are provided for the purpose of description rather than of limitation.

Claims (109)

A machine implemented method for estimating how a particular user evaluates a particular item from multiple items, Identifying one or more other items from the plurality of items that have a similar rating to the particular item; Selecting a different set of users from the plurality of different users based on user selection criteria; Identifying one or more other users who have rated a subset of the one or more other items from the set of other users, wherein the ratings are statistically similar to ratings assigned to the subset of one or more other items by the particular user. ; And Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. The method of claim 1, Identifying one or more other items having a rating similar to the particular item from the plurality of items is performed offline prior to other steps. A machine readable medium for estimating how a particular user evaluates a particular item from a plurality of items, the machine readable medium being executed by one or more processors, Identifying one or more other items from the plurality of items that have a similar rating to the particular item; Selecting a different set of users from the plurality of different users based on user selection criteria; Identifying one or more other users who have rated a subset of the one or more other items from the set of other users, wherein the ratings are statistically similar to ratings assigned to the subset of one or more other items by the particular user. ; And Instructions for causing performance of the step of generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. Machine-readable medium for the production. The method of claim 3, wherein Identifying one or more other items having a rating similar to the particular item from the plurality of items is performed offline prior to other steps. An apparatus for estimating how a particular user evaluates a particular item from multiple items, wherein the device, when executed by one or more processors, Identifying one or more other items from the plurality of items that have a similar rating to the particular item; Selecting a different set of users from the plurality of different users based on user selection criteria; Identifying one or more other users who have rated a subset of the one or more other items from the set of other users, wherein the ratings are statistically similar to ratings assigned to the subset of one or more other items by the particular user. ; And And a memory storing instructions that cause the performance of generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. Estimation apparatus for evaluation. The method of claim 5, Identifying one or more other items having a rating similar to the particular item from the plurality of items is performed offline prior to other steps. A machine implemented method for estimating how a particular user evaluates a particular item from multiple items, Identifying one or more items from the plurality of items that have a similar rating to the particular item; Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. The method of claim 7, wherein Identifying one or more other items having a rating similar to the particular item from the plurality of items is performed offline prior to other steps. The method of claim 7, wherein Identifying one or more items having a rating similar to the particular item from the plurality of items includes calculating a statistical correlation of the particular item from the plurality of items with another item. A machine implemented method for estimating on. The method of claim 7, wherein Determining whether each of the one or more items and the particular item has at least a specified number of common grades. The method of claim 7, wherein Determining whether one of the one or more items and the particular item has a specified fraction by dividing a common rating between one of the one or more items and the particular item by one of the one or more items or the number of ratings of the particular item And a step further comprising estimating the item evaluation. The method of claim 7, wherein Identifying one or more items having a similar rating to the particular item from the plurality of items includes a specified fraction of a statistically expected number, given the number of ratings of the particular item and each item for which the one or more items are individually counted. Identifying whether there is a larger number of common evaluators. The method of claim 7, wherein Identifying one or more items from the plurality of items is performed by one or more external agents. The method of claim 7, wherein Identifying one or more items having a rating similar to the particular item from the plurality of items includes statistical correlation between the particular item and the one or more items, wherein the particular item and the one or more items have at least a specified number of common ratings. One or more of whether the particular item and the one or more items have a common grade of at least a specified fraction, and whether the particular item and the one or more items have a specified fraction of the common grade of at least a statistically expected number And a step of considering the method. The method of claim 7, wherein Selecting the one or more other users from a plurality of other users based on each of the one or more other users having at least a specified minimum number of ratings. The method of claim 7, wherein And selecting the one or more other users from a plurality of other users based on each of the one or more other users having a number less than a specified maximum number of machines. Way. The method of claim 7, wherein And selecting the at least one other user from a plurality of other users based on each of the at least one other user having an average rating within a specified range. The method of claim 7, wherein Selecting the at least one other user from a plurality of other users based on each of the at least one other user having a rating distribution within a specified tolerance of a normal distribution. How to implement. The method of claim 7, wherein And selecting the at least one other user from a plurality of other users based on each of the at least one other user having a rating standard deviation within a specified range. The method of claim 7, wherein Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user may include whether the one or more other users have at least a specified minimum number of common ratings. Whether the one or more other users have fewer ratings than the specified maximum number, whether the one or more other users have an average rating within a specified range, and wherein the one or more other users have a rating distribution within a specified tolerance of the normal distribution And taking into account one or more of whether one or more other users have a rating standard deviation within a specified range. The method of claim 7, wherein Identifying the one or more other users includes selecting one or more other users whose statistical correlation with a particular user of one or more items of a common grade is greater than a specified threshold. Machine implementation method. The method of claim 7, wherein Identifying the one or more other users comprises selecting one or more other users whose vector cosine distance of the one or more items of the common class is less than a specified threshold. Way. The method of claim 7, wherein Identifying the one or more other users comprises selecting one or more other users who have given the subset of the one or more items a common rating exactly the same as the particular user. How to implement a machine. The method of claim 7, wherein Identifying the one or more other users includes selecting one or more other users who have given a common rating to the subset of the one or more items within a specified threshold of ratings by the particular user. A machine implemented method for estimating on. The method of claim 7, wherein The step of identifying the one or more other users includes whether the one or more users have a correlation of one or more items of a common rating greater than a specified threshold, the vector of one or more items of a common rating less than the specified threshold Whether there is a cosine distance, whether the one or more users gave the subset of the one or more items the same common rating, and the one or more users within a specified threshold of ratings by the particular user. And taking into account one or more of whether or not a subset has been given a common rating. The method of claim 7, wherein Identifying the one or more other users comprises selecting a specified number of other users most similar to the particular user. The method of claim 7, wherein Identifying the one or more other users comprises selecting another user whose similarity with the particular user is better than a specified threshold. The method of claim 7, wherein Identifying the one or more other users includes selecting a specified number of other users most similar to the particular user, wherein all selected other users have a similarity better than a specified threshold. Method for implementing a machine for estimation. The method of claim 7, wherein Generating an estimate as to how the user evaluates the particular item based on the rating of the particular item granted by the one or more other users determines the average rating of the particular item by the one or more other users. And implementing the estimation of the item evaluation. The method of claim 7, wherein Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users comprises: determining an average rating of the particular item by the one or more other users. And determining the method for estimating the item evaluation. The method of claim 7, wherein For each of the one or more users, generating an individual estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by each of the one or more other users; And And calculating a confidence metric based on the relationship between the individual estimates. The method of claim 7, wherein Wherein the plurality of items is a plurality of movies, the one or more items are one or more movies, and the specific item is a specific movie. The method of claim 7, wherein Wherein the plurality of items is a plurality of music items, the one or more items are one or more music items, and the specific item is a specific music item. A machine readable medium for estimating how a particular user evaluates a particular item from a plurality of items, the machine readable medium being executed by one or more processors, Identifying one or more items from the plurality of items that have a similar rating to the particular item; Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And Instructions for causing performance of the step of generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. Machine-readable medium for the production. The method of claim 34, wherein Identifying one or more other items having a rating similar to the particular item from the plurality of items is performed offline prior to other steps. The method of claim 34, wherein Identifying one or more items having a rating similar to the particular item from the plurality of items includes calculating a statistical correlation of the particular item from the plurality of items with another item. Machine-readable medium for estimation about. The method of claim 34, wherein And when processed by the one or more processors, one or more additional instructions to cause the one or more processors to perform an additional step of determining whether each of the one or more items and the particular item has at least a specified number of common grades. And a machine-readable medium for estimation about item evaluation. The method of claim 34, wherein When processed by the one or more processors, one of the one or more items and the specific item have a common rating between at least one of the one or more items and the specific item, and the number of ratings of one or more of the one or more items. And one or more additional instructions for causing the one or more processors to perform an additional step of determining whether to have a specified fraction divided by. The method of claim 34, wherein Identifying one or more items having a similar rating to the particular item from the plurality of items includes a specified fraction of a statistically expected number, given the number of ratings of the particular item and each item for which the one or more items are individually counted. And identifying whether there is a larger plurality of common evaluators. The method of claim 34, wherein And identifying one or more items from the plurality of items is performed by one or more external agents. The method of claim 34, wherein Identifying one or more items having a rating similar to the particular item from the plurality of items includes statistical correlation between the particular item and the one or more items, wherein the particular item and the one or more items have at least a specified number of common ratings. One or more of whether the particular item and the one or more items have a common grade of at least a specified fraction, and whether the particular item and the one or more items have at least a statistically expected number of specified fractions of the common grade And taking into account the machine-readable medium for estimating the item evaluation. The method of claim 34, wherein And when processed by the one or more processors, cause the one or more processors to perform an additional step of selecting the one or more other users from a plurality of other users based on each of the one or more other users having at least the minimum number of ratings. And further comprising one or more additional instructions. The method of claim 34, wherein When processed by the one or more processors, the one or more processors perform an additional step of selecting the one or more other users from a plurality of other users based on each of the one or more other users having fewer than the maximum number of ratings. Further comprising one or more additional instructions to cause the machine-readable medium for estimating the item evaluation. The method of claim 34, wherein And when processed by the one or more processors, cause the one or more processors to perform an additional step of selecting the one or more other users from a plurality of other users based on each of the one or more other users having an average rating within a specified range. And further comprising one or more additional instructions. The method of claim 34, wherein When processed by the one or more processors, the additional step of selecting the one or more other users from a plurality of other users based on each of the one or more other users having a rating distribution within a specified tolerance of a normal distribution. Further comprising one or more additional instructions for the user to perform. The method of claim 34, wherein When processed by the one or more processors, cause the one or more processors to perform an additional step of selecting the one or more other users from a plurality of other users based on each of the one or more other users having a rating standard deviation within a specified range. Further comprising one or more additional instructions for estimating the item evaluation. The method of claim 34, wherein Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user may include whether the one or more other users have at least a specified minimum number of common ratings. Whether the one or more other users have fewer ratings than the specified maximum number, whether the one or more other users have an average rating within a specified range, and wherein the one or more other users have a rating distribution within a specified tolerance of the normal distribution And taking into account one or more of whether one or more other users have a rating standard deviation within a specified range. The method of claim 34, wherein Identifying the one or more other users includes selecting one or more other users whose statistical correlation with a particular user of one or more items of a common grade is greater than a specified threshold. Machine-readable media for The method of claim 34, wherein Identifying the one or more other users includes selecting one or more other users whose vector cosine distance of the one or more items of the common class is less than a specified threshold. Media available. The method of claim 34, wherein Identifying the one or more other users comprises selecting one or more other users who have given the subset of the one or more items a common rating exactly the same as the particular user. Machine-readable medium. The method of claim 34, wherein Identifying the one or more other users includes selecting one or more other users who have given a common rating to the subset of the one or more items within a specified threshold of ratings by the particular user. Machine-readable medium for estimation about. The method of claim 34, wherein The step of identifying the one or more other users includes whether the one or more users have a correlation of one or more items of a common rating greater than a specified threshold, the vector of one or more items of a common rating less than the specified threshold Whether there is a cosine distance, whether the one or more users gave the subset of the one or more items the same common rating, and the one or more users within a specified threshold of ratings by the particular user. And taking into account one or more of whether or not a subset of has been given a common rating. The method of claim 34, wherein Identifying the one or more other users comprises selecting a specified number of other users most similar to the particular user. The method of claim 34, wherein Identifying the one or more other users comprises selecting another user whose similarity with the particular user is better than a specified threshold. The method of claim 34, wherein Identifying the one or more other users includes selecting a specified number of other users most similar to the particular user, wherein all selected other users have a similarity better than a specified threshold. Machine-readable medium for estimation on the subject. The method of claim 34, wherein Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users comprises: determining an average rating of the particular item by the one or more other users. And determining a machine-readable medium for estimating the item evaluation. The method of claim 34, wherein Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users comprises: determining an average rating of the particular item by the one or more other users. And determining a machine-readable medium for estimating the item evaluation. The method of claim 34, wherein The one or more processors, when processed by the one or more processors, For each of the one or more users, generate an individual estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by each of the one or more other users, And one or more additional instructions to perform an additional step of calculating a confidence metric based on the relationship between the individual estimates. The method of claim 34, wherein Wherein the plurality of items is a plurality of movies, the one or more items are one or more movies, and the specific item is a specific movie. The method of claim 34, wherein Wherein the plurality of items is a plurality of music items, the one or more items are one or more music items, and the specific item is a specific music item. An apparatus for estimating how a particular user evaluates a particular item from a plurality of items, wherein the device, when executed by one or more processors, causes the one or more processors to: Identifying one or more items from the plurality of items that have a similar rating to the particular item; Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And And storing memory for instructions to perform the step of generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. Estimation apparatus for. 62. The method of claim 61, Identifying one or more other items having a rating similar to the particular item from the plurality of items is performed offline prior to other steps. 62. The method of claim 61, Identifying one or more items having a rating similar to the particular item from the plurality of items includes calculating a statistical correlation of the particular item from the plurality of items with another item. Estimation apparatus for. 62. The method of claim 61, The memory is processed by the one or more processors, the one or more processors to cause the one or more processors to perform an additional step of determining whether each of the one or more items and the particular item has at least a specified number of common classes And estimating apparatus for item evaluation. 62. The method of claim 61, When the memory is processed by the one or more processors, one of the one or more items and the specific item have a common rank between at least one of the one or more items and the particular item, or one of the one or more items or the specific item. And one or more additional instructions for causing the one or more processors to perform an additional step of determining whether or not to have a specified fraction divided by the number of grades of. 62. The method of claim 61, Identifying one or more items having a similar rating to the particular item from the plurality of items includes a specified fraction of a statistically expected number, given the number of ratings of the particular item and each item for which the one or more items are individually counted. And identifying whether there is a larger number of common evaluators. 62. The method of claim 61, And identifying one or more items from the plurality of items is performed by one or more external agents. 62. The method of claim 61, Identifying one or more items having a rating similar to the particular item from the plurality of items includes statistical correlation between the particular item and the one or more items, wherein the particular item and the one or more items have at least a specified number of common ratings. One or more of whether the particular item and the one or more items have a common grade of at least a specified fraction, and whether the particular item and the one or more items have a specified fraction of the common grade of at least a statistically expected number And estimating the item evaluation according to the above. 62. The method of claim 61, When the memory is processed by the one or more processors, the one or more processors further comprise selecting the one or more other users from a plurality of other users based on each of the one or more other users having at least the minimum number of ratings. And estimating apparatus for item evaluation further comprising one or more additional instructions for performing. 62. The method of claim 61, When the memory is processed by the one or more processors, the one or more additional steps of selecting the one or more other users from a plurality of other users based on each of the one or more other users having fewer than the maximum number of ratings And estimating apparatus for item evaluation further comprising one or more additional instructions for the processor to perform. 62. The method of claim 61, The memory may be further processed by the one or more processors to select the one or more other users from a plurality of other users based on each of the one or more other users having an average rating within a specified range when the memory is processed by the one or more processors. And estimating apparatus for item evaluation further comprising one or more additional instructions for performing. 62. The method of claim 61, The memory is further processed to select the one or more other users from a plurality of other users based on each of the one or more other users having a rating distribution within a specified tolerance of a normal distribution when processed by the one or more processors. And estimating apparatus for item evaluation further comprising one or more additional instructions for one or more processors to perform. 62. The method of claim 61, When the memory is processed by the one or more processors, the additional step of selecting the one or more other users from a plurality of other users based on each of the one or more other users having a class standard deviation within a specified range is provided. And one or more additional instructions for the user to perform. 62. The method of claim 61, Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user may include whether the one or more other users have at least a specified minimum number of common ratings. Whether the one or more other users have fewer ratings than the specified maximum number, whether the one or more other users have an average rating within a specified range, and wherein the one or more other users have a rating distribution within a specified tolerance of the normal distribution And one or more of considering whether one or more other users have a rating standard deviation within a specified range. 62. The method of claim 61, Identifying the at least one other user comprises selecting at least one other user whose statistical correlation with a particular user of at least one item of a common grade is greater than a specified threshold; . 62. The method of claim 61, Identifying the one or more other users comprises selecting one or more other users whose vector cosine distance of the one or more items of the common class is less than a specified threshold. 62. The method of claim 61, Identifying the one or more other users comprises selecting one or more other users who have given the subset of the one or more items a common rating exactly the same as the particular user. 62. The method of claim 61, Identifying the one or more other users comprises selecting one or more other users who have given a common rating to a subset of the one or more items within a specified threshold of ratings by the particular user. Estimation apparatus for evaluation. 62. The method of claim 61, The step of identifying the one or more other users includes whether the one or more users have a correlation of one or more items of a common rating greater than a specified threshold, the vector of one or more items of a common rating less than the specified threshold Whether there is a cosine distance, whether the one or more users gave the subset of the one or more items the same common rating, and the one or more users within a specified threshold of ratings by the particular user. And taking into account one or more of whether or not a subset of has been given a common rating. 62. The method of claim 61, Identifying the one or more other users comprises selecting a specified number of other users most similar to the particular user. 62. The method of claim 61, Identifying the one or more other users comprises selecting another user whose similarity with the particular user is better than a specified threshold. 62. The method of claim 61, Identifying the one or more other users includes selecting a specified number of other users most similar to the particular user, wherein all selected other users have a similarity better than a specified threshold. On the estimation device. 62. The method of claim 61, Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users comprises: determining an average rating of the particular item by the one or more other users. And estimating an item evaluation. 62. The method of claim 61, Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users comprises: determining an average rating of the particular item by the one or more other users. And estimating an item evaluation. 62. The method of claim 61, When the memory is processed by the one or more processors, the one or more processors, For each of the one or more users, generate an individual estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by each of the one or more other users, And one or more additional instructions to perform an additional step of calculating a confidence metric based on the relationship between the individual estimates. 62. The method of claim 61, Wherein the plurality of items is a plurality of movies, the one or more items are one or more movies, and the specific item is a specific movie. 62. The method of claim 61, Wherein the plurality of items is a plurality of music items, the one or more items are one or more music items, and the specific item is a specific music item. Identifying one or more items having a rating similar to a particular item from the plurality of items; Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And Causing an estimation as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users, And estimating an estimate as to how the particular user evaluates the particular item from the plurality of items. 89. The method of claim 88 wherein Causing the estimation is performed before the request for the estimation, and the data representing the estimation is stored in a memory. 92. The method of claim 89, And the apparatus is configured to write data representing the estimation to a nonvolatile storage device. 92. The method of claim 90, And wherein said nonvolatile storage device is one or more magnetic or solid state disk systems, wherein all or part of said nonvolatile storage device is memory mapped to random access memory as needed. 92. The method of claim 90, And the apparatus is configured to retrieve data representing the estimate from the nonvolatile storage device and to write the data to volatile memory. 89. The method of claim 88 wherein The device is in response to any one or more other items or one or more classes updated or added to the particular item, Identify one or more other items from the plurality of items that have a similar rating to the particular item; Identify one or more users who have given the one or more items a rating substantially similar to the rating given to the one or more other items by the particular user; Generate a modified estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more users; And store data indicative of a modified estimate of how the particular user evaluates the particular item. 89. The method of claim 88 wherein The device is in response to any one or more other items or one or more classes updated or added to the particular item, Identify one or more estimated grades that are affected by the one or more grades that are updated or added; Generate one or more update estimates for the one or more estimates that are affected by the one or more ratings that are updated or added; And store data indicative of the at least one update estimation class. 89. The method of claim 88 wherein And the apparatus is configured to periodically generate and update similarity data identifying similarities between items from the plurality of items based on a rating associated with the plurality of items. 89. The method of claim 88 wherein The device is configured to select the one or more other users from a plurality of users based on selection criteria including whether a particular user has a specified number of ratings, to generate and store data indicative of the selected one or more other users. Estimation device, characterized in that. 89. The method of claim 88 wherein And further comprising one or more priority job queues to which tasks to be processed are assigned based on priorities associated with each task, wherein the highest priority is assigned to the tasks associated with updating the estimated class for active users. Estimating device. 89. The method of claim 88 wherein Wherein the plurality of items is a plurality of movies, the one or more items are one or more movies, and the specific item is a specific movie. 89. The method of claim 88 wherein Wherein the plurality of items is a plurality of music items, the one or more items are one or more music items, and the specific item is a specific music item. An apparatus comprising a plurality of processes assigned to generate a plurality of estimated ratings about how a user evaluates items from the plurality of items, the apparatus comprising: a method of estimating how a particular user evaluates a particular item from the plurality of items Occurrence is, Identify one or more items from the plurality of items that have a similar rating to the particular item; Identify one or more other users who have assigned the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And estimating how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. 101. The method of claim 100, And at least one process configured to generate and store grade data indicative of the estimated grade, wherein each process is not granted access to the grade data generated and stored by another process. 102. The method of claim 101, wherein Select reference users and assign similarities between items from the plurality of items based on ratings associated with the plurality of items to generate and store similarity data indicative of identified similarities between items from the plurality of items And at least one process having read / write access to the similarity and reference user data, and at least one other process having read only access to the similarity data. . 103. The method of claim 102, The similarity and reference user data is periodically copied, wherein at least one recommendation process has access to the copied similarity and reference user data. 101. The method of claim 100, Further comprising an additional process assigned to select the one or more other users from the plurality of users based on selection criteria including whether the user has a specified number of ratings, and generate data indicative of the selected one or more other users; And wherein said further process has read / write access to said data, said plurality of processes having read-only access to said data. 101. The method of claim 100, Wherein the plurality of items is a plurality of movies, the one or more items are one or more movies, and the specific item is a specific movie. 101. The method of claim 100, Wherein the plurality of items is a plurality of music items, the one or more items are one or more music items, and the specific item is a specific music item. A machine implemented method for estimating how a particular user evaluates a particular item from multiple items, Identifying one or more items similar to the particular item from the plurality of items by associating semantic feature values with the plurality of items and the particular item, wherein each feature type is a matching score that may be positive, zero, or negative; With a mismatch score, scoring the similarity of items, Find features with matching values and add them with the associated matching score, By finding features with mismatched values and summing them with the relevant mismatch scores, Items with a combined similarity score above a specified threshold are considered similar, or a specified number of items with the highest similarity score are considered similar; Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And Generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. A machine readable medium for estimating how a particular user evaluates a particular item from a plurality of items, the machine readable medium being executed by one or more processors, Identifying one or more items similar to the particular item from the plurality of items by associating semantic feature values with the plurality of items and the particular item, wherein each feature type is a matching score that may be positive, zero, or negative; With a mismatch score, scoring the similarity of items, Find features with matching values and add them with the associated matching score, By finding features with mismatched values and summing them with the relevant mismatch scores, Items with a combined similarity score above a specified threshold are considered similar, or a specified number of items with the highest similarity score are considered similar; Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And Instructions for causing performance of the step of generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. Machine-readable medium for the production. An apparatus for estimating how a particular user evaluates a particular item from multiple items, wherein the device, when executed by one or more processors, Identifying one or more items similar to the particular item from the plurality of items by associating semantic feature values with the plurality of items and the particular item, wherein each feature type is a matching score that may be positive, zero, or negative; With a mismatch score, scoring the similarity of items, Find features with matching values and add them with the associated matching score, By finding features with mismatched values and summing them with the relevant mismatch scores, Items with a combined similarity score above a specified threshold are considered similar, or a specified number of items with the highest similarity score are considered similar; Identifying one or more other users who have given the one or more items a rating substantially similar to the rating given to the one or more items by the particular user; And And a memory storing instructions that cause the performance of generating an estimate as to how the particular user evaluates the particular item based on the rating of the particular item granted by the one or more other users. Estimation apparatus for evaluation.

Images