WO2000030010A1

WO2000030010A1 - Computer-implemented educational product recommendation system

Info

Publication number: WO2000030010A1
Application number: PCT/US1999/027140
Authority: WO
Inventors: Jeffrey A. Pucci; Richard Viard; Albert L. Noyes; David A. Blohm; Darien Fitzgerald; Frederic L. Day, Iii; Susan Mara Graham
Original assignee: Smarterkids.Com
Priority date: 1998-11-16
Filing date: 1999-11-16
Publication date: 2000-05-25
Also published as: AU1626700A; CA2351680A1; AU763950B2; GB2361563A; EP1151406A1; GB0114514D0

Abstract

The invention is a computer-implemented product recommendation system. The system includes a storage for data characterizing relationships between product attributes, such as education skills, within a set product attributes. The system also includes a storage for data associating each of a number of products, such as educational products, with one or more of the product attributes. The system also includes software for accessing attribute relationship data from the storage for data characterizing relations between the product attributes, determining related product attributes based on the desired product attributes and the attribute relationship data, accessing product association data from the storage for data associating products with product attributes, and determining a matching set of the products using the product association data and the desired and related product attributes.

Description

COMPUTER-IMPLEMENTED EDUCATIONAL PRODUCT RECOMMENDAΗON SYSTEM

BACKGROUND

This invention relates to a computer implemented product recommendation system. A number of current Internet-based systems provide customers with the ability to browse an inventory of available products, such as books or toys, using a Web browser applications (e.g., Netscape Navigator or Microsoft Internet Explorer designed for the Worldwide Web) which executes on the customer's personal computer and which communicates over the Internet with a Web server application at centralized server computer. Many of these systems provide the ability to search for and select particular products, for example, by supplying keywords that are present in a product description. Some systems also allow the customer to specify a particular category of products, for example products appropriate to a particular age group, products in a particular price range, types of toys (e.g., dolls, action figures, games, etc.), or products from a particular manufacturer or publisher. A customer can then select and purchase a product, for example, by providing a credit card number to charge for the product, and an address to which the product is to be shipped.

SUMMARY

In a general aspect, this invention addresses some of the shortcomings of many currently available product search and selection approaches. In particular, the invention provides a way of recommending products to a customer by matching information characterizing an individual for whom the products are intended to information characterizing each of the products in a product inventory. The invention is particularly direct to recommendation of educational products based on test or survey results related to a child. In one aspect, in general, the invention is a computer-implemented system for recommending educational products, such as books, toys, games, puzzles, crafts and software. The system includes a storage for data characterizing relationships between product attributes within a set product attributes, and a storage for data associating each of a number of products with one or more of the product attributes. The set of product attributes includes a set of educational skills. The system also includes software stored on a computer-readable medium for causing a computer to perform the functions of determining one or more desired product attributes, accessing attribute relationship data from the storage for data characterizing relations between the product attributes, determining related product attributes based on the desired product attributes and the attribute relationship data, accessing product association data from the storage for data associating products with product attributes, and determining a matching set of the products using the product association data and the desired and related product attributes. The invention can include one or more of the following features:

The storage for data associating each of the products with one or more of the product attributes includes a database table for holding a number of records. Each record associates one of the products with one of the product attributes.

The relationships between product attributes include inclusion of particular of the educational skills within some category of educational skills. Determining related product attributes then includes determining educational skills that are in the same category of educational skills as a desired educational skill. The relationships between product attributes can further include inclusion of a number of categories of educational skills within a broader category of educational skills. Determining one or more desired educational skills includes providing a questionnaire including a plurality of questions and accepting responses to the questions.

Determining an equivalent grade level associated with each of the determined educational skills based on the accepted responses. Determining one or more desired educational skills includes accepting test scores obtained on a standardized test and determining an equivalent grade level associated with each of the one or more educational skills based on the accepted test scores. Determining an equivalent grade level includes comparing the accepted test scores with statistical data characterizing a distribution of test scores for a grade level.

Determining one or more desired product attributes includes determining test scores for an individual, and mapping the tests scores to a quantity related to one of the educational skills, such as an equivalent grade level associated with the skill. For instance, the system can directly administer a set of questions and grade answers from the individual to determine test scores, or the system can accept test scores for a test administered outside the control of the system, such as a standardized test administered to school children.

The invention includes one or more of the following advantages: Use of a common set of product attributes, such as educational skills, for both characterizing products, and for representing desired properties for products provides an efficient mechanism for matching desired attributes to products.

Products can be matched to varying numbers of attributes, thereby providing a mechanism for representing products that are focused on a single attribute, as well as those spanning a number of attributes.

Mapping test scores for an individual into one or more educational skills and associated grade equivalents provides a mechanism for using the results of a variety of tests without having to modify the data that associates the products with educational skills. In this way, particularly relevant educational products can be recommended to children based on their scores on a diagnostic test administered by the system, as well an on their score on standardized tests, such as those administered by a school.

Other features and advantages of the invention are apparent from the following description, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a client/server software system, which is used to recommend products to a user of the system;

FIG. 2 is a block diagram of data stored and maintained on the server computer;

FIG. 3 is a diagram of database tables and interrelationships of the tables;

FIGS. 4A-D are a tabular representation of a skill hierarchy stored in the skill hierarchy table;

FIG. 5 is a diagram of a many-to-many skill/product mapping; FIGS. 6A-C are diagrams which illustrate approaches for obtaining information about a child;

FIG. 6A is a data flow diagram that illustrates direct input from a user, and use of a learning style survey to determine learning styles of the child;

FIG. 6B is a data flow diagram that illustrates use of a skills test to determine a skills assessment for the child;

FIG. 6C is a data flow diagram that illustrates use of scores obtained from a standardized test to determine a skills assessment for the child;

FIG. 7 is a sample set of learning style questions;

FIGS. 8A-E is a sample skills test; and FIG. 9 is a diagram that illustrates the relationship of skills and selected products.

DESCRIPTION

Referring to FIG. 1, a number of client computers 110 and a server computer 140 are coupled over communication links 128 and 132, respectively, to Internet 130. Together, software that executes on a client computer 110 and on server computer 140 forms a client/server software system. A user 120 uses this software system to obtain a list of recommended products for a child 122 based on inputs related to that child that are provided by user 120, and optionally to select and purchase some of the recommended products. The system determines characteristics of child 122 from the inputs provided by user 120 and matches those characteristics to particular products.

In particular, the system uses the input provided by user 120 to determine learning styles or skills that are related to the child, and matches those learning styles or skills to an inventory of educational products. The system sorts the matched products according to a variety of properties of the products and recommends the matched product to the user according to their sorted order, for example by displaying a sorted list of the product names and related graphics for the top matching products. In embodiments that include the function of on-line purchasing, user 120 can select and purchase one or more of the recommended products.

Although the description of this embodiment focuses on a system for providing recommendations for educational products for a child, alternative embodiments provide recommended products for other classes of individuals, including adults, and for other types of products or items that are not necessarily commercial products. In general, user 120 is not necessarily the same person as child 122, for instance user 120 may be the child's parent. However, the user and the child can indeed be the same individual.

In some operating modes, the system uses test scores 114 to determined characteristics of child 122. In one such operating mode, a testing application 116 executes on client computer 110 using executable software provided on a CD-ROM 118. Child 122 responds to questions generated by testing application 116, and the responses are used to determine scores for individual or groups of questions. In another such operating mode, child 122 has previously taken a standardized test 124, and user 120 provides test scores 114 based on the graded standardized test. In some embodiments, a third-party computer 160 is also coupled to Internet

130 over a communication link 164. Client computer 110 communicates with third party computer 160, which then communicates with server computer 140, in addition to or instead of client computer 110 communicating directly with server computer 140. In one such embodiment, third party computer 160 collects some inputs related to child 122, and forwards data derived from those inputs to server computer 140. Server computer 140 then either communicates directly with client computer 110, or indirectly though third party computer 160, to provide product recommendations to user 120.

A server-based system of the type described above has been deployed and is in commercial use. A server computer 140 is accessible to client computers 110 at the

URL (uniform resource locator) http://www.smarterkids.com/. The content of the

Web pages that are publicly accessible at that site are incorporated herein by reference. A third-party computer 160 that communicates both with a client computer

110 and a server computer 140 has been deployed and is accessible at the URL http://www.wehelpkids.com/ . The content of the Web pages that are publicly accessible at that site are also incorporated herein by reference.

Server computer 140 includes a database 150 that includes information related to an inventory of products. A database server application 148 is coupled to database 150 and provides an interface to the information stored in database 150 to other software modules that execute on server computer 140. In this embodiment, database 150 is a relational database, which includes a number of interrelated tables. Database server application 148 is an SQL server that accepts queries according to an SQL language syntax and provides responses to those queries. Database server application 148 has the capabilities of executing stored procedures. Each stored procedure implements a relatively complex query that may involve multiple separate accesses to database 150. These stored procedures are stored on server computer 140 in database procedures 146. Database procedures 146 are stored on a file system, such as on a magnetic disk storage, on which the procedures a stored in individual files in a source (SQL) or compiled form. Server computer 140 also executes a Web server application 142. In some embodiments, the functions performed by server computer 140 are split among several server computers, for example, having Web server application 142 execute on a different computer than database server application 148. Furthermore, these servers may be geographically separated, for example also coupled over Internet 130. Web server application 142 responds to communication it receives from a Web browser 112 over Internet 130. Web server application 142 listens for connections on a particular Internet Protocol (IP) port and receives standard http (hyper-text transport protocol) requests on that port that identify particular Web pages that the requestor, typically Web browser 112 on a client computer 110, wants to receive. Web pages 144 includes individual Web pages which are stored as individual files on a file system. Web server application 142 responds to http requests from Web browser 112 by providing HTML (hyper-text markup language) format data to the Web browser, which in turn uses the HTML data to display information to user 120. Requests from a Web browser 112 can include data, for example obtained by user 120 filling out a form or selecting options by selecting choice buttons with a mouse.

Web server application 142 optionally communicates with a commerce application 156 that provides an interface for Web server application 142 to external commercial transaction and order fulfillment services 158, which allow user 120 to order products that have been recommended by the system. Server computer 140 also includes a database maintenance application 152.

An administrator 154 of the system uses the maintenance application to update the data stored in database 150. For example, as new products are added to the inventory, administrator 154 enters information related to the new product and this information is stored in database 150. Referring to FIG. 2, Web pages 144 includes two types of pages. One type is static Web pages 210, which are HTML format pages that are passed by Web server application 142 (FIG. 1) directly to a requesting Web browser 112 without modification. The other type is active server pages 220. An active server page 220 includes a procedure specification that, when requested by a Web browser, that page is executed under the control of Web server application 142 rather than being passed directly to the Web browser. Execution of the procedure specified by an active server page 220 produces HTML format data that is passed by Web server application 142 to the Web browser. In this embodiment, active server pages 220 include Visual Basic language procedure specification that are executed under to the control of Web server application 142. Active server pages 220 can include references to services provided by database server application 148 (FIG. 1). For instance, a Visual Basic procedure in active server page 220 accesses database server application 148 through an application program interface (API) for the database server application. During execution of the stored procedure, Web server application 142 can therefore access data stored in database 150. Active server pages 220 can also include references to stored database procedures 230. Each stored database procedure 230 includes one or more SQL statements. Web server application 142 invokes a stored database procedure 230 during execution of an active server page 220. Database server application 148 controls the execution of the stored database procedure 230 to provide return data to Web server application 142.

Together, static Web pages 210, active server pages 220, and stored database procedures 230 determine the user interface through which user 120 interacts with the system. Database 150 includes a number of separate tables. A product table 240 includes information related to an inventory of products available in the system. A skill hierarchy table 244 relates to a set of skills, or learning goals. Both product table 240 and skill hierarchy table are dynamic in that they can be modified, for example, as new products are added or as characterizations of existing products are refined. This set of skills is generic, not being particularly related to any one of the products. The set of skills represented in skill hierarchy table 244 is used to represent attributes or characteristics of the products represented in product table 240. The set of skills is also used in part to represent attributes that are desired to be associated with recommended products for child 122. A third table, skill/product table 242 is used to associate particular products with particular skills in a many-to-many mapping between products and skills. Product table 240, skill hierarchy table 244, and skill/product table 242 together provide data needed to recommend particular products based their relationships to a desired skill or set of skills.

Database 150 also includes permanent user data 248, which include information about users 120 and children 122 that is persistent between sessions in which the user interacts with the server computer. Information about a user 120 includes their first and last name, and a password that is needed to access the system.

Information about a child 122 includes their name, their date of birth, and their current grade. Database 150 also includes a user cache 246, which is used to store intermediate results while the system interacts with a user 120 during a session. User cache 246 is used to avoid recomputing results during a particular session.

Information in user cache 246 has a limited lifetime. For example, data in user cache

246 is periodically removed if it has not been accessed for a predetermined amount of time.

Referring to FIGS. 3, database 150 (FIGS. 1, 2) includes tables that relate skills and products. Product table 240, skill/product table 242 and skill hierarchy table 244 are interrelated and include entries that refer to entries in other of these tables. Product table 240 includes one product record 330 per product in the inventory of the system. Each product record 330 includes a number fields related to the product represented by that record. A product identification 312 includes a unique identifier that is used to refer to this product in other tables of the database. A description 314 includes fields that described the product including, for example, a text description, a universal product code, and a manufacture or supplier. Each product record also includes a learning styles field 316. Learning styles

316 includes a number of Boolean indicators used to signify whether or not that product is related to one of a corresponding number of standard learning styles. In this embodiment, seven learning styles are used. The specific learning styles are motivated Howard Gardner's theory of Multiple Intelligences. These learning styles are: 1. Physical,

2. Interpersonal,

3. Intrapersonal,

4. Linguistic, 5. Mathematical,

6. Musical, and

7. Visual.

Each product record 330 also includes a grade range 318, which includes a lower and upper grade for which the product is generally appropriate. Note that more refined grade ranges are associate with particular skills related to the product, as is described below with reference to skill/product table 242.

Each product record 330 also includes a learning approach field 320. Learning approach 320 includes a number of Boolean fields, each indicating whether the product is related to a corresponding learning approach. These learning approaches are:

1. Instruction,

2. Reference,

3. Drill & Practice,

4. Exploration & Discovery, 5. Tools, and

6. Educational Game. Finally, each product record 330 includes a number of ratings 322 related to a corresponding number of ratings categories. Each rating field includes a numeric rating, from 1 (worst) to 5 (best), for each of 5 ratings categories. These ratings categories are:

1. Educational Value,

2. Fun,

3. Ease of Use,

4. Depth/Reusability, and 5. Reviewer's Opinion. Skill hierarchy table 244 defines a hierarchical relationship of a set of skills.

A skill hierarchy 380 is illustrated in a portion of a tabular form. A complete listing of skill hierarchy 380 used in this embodiment is shown in FIGS. 4A-D. Each skill is characterized by three identifiers: a major skill 382, a minor skill 384, and a detailed skill 386. For example, as shown in the figure, the "reading & writing" major skill includes the "general," "phonics," and "writing" minor skills. Each combination of a major and minor skill includes one or more detailed skills. For example, the "reading

& writing/phonics" major/minor skill includes the detailed skills "beginning sounds" and "ending sounds." As is discussed further below, during the matching process, the relationship of different detailed skills according to whether they fall under the same major/minor skill combination, the same major skill, or under different major skills can affect the selection of products.

Skill hierarchy table 244 includes the interrelationship of the detailed skills, and associates each detailed skill with a skill identifier 362. Therefore, each skill identifier 362 is associated with a particular combination of a major skill 364, a minor skill 366, and a detailed skill 368. Note that in general, each major skill includes a "general" minor skill, and each combination of a major skill and a "general" minor skill includes a "general" detailed skill. Therefore, a major skill as a whole is referred to by the identifier for the "general/general" minor/detailed skill combination for hat major skill. Similarly, a minor skill is referred to by the identifier of the "general" detailed skill for that minor skill.

Skill/product table 242 provides a link between product records 330 in product table 240 and skills hierarchy 380. As a whole, skill/product table 242 represents a many-to-many skill/product mapping 390 between products and skills in skills hierarchy 380. Each record of skill product table 242 is related to an association between a single of the products and a single of the skills. A representative skill product record 352 relates a product with product identification 113 with a skill with skill identification 25. Each skill/product record includes a skill identification field 342 and a product identification field 344. Each record also includes a grade range field 346. Unlike grade range field 318 in product table 318, this grade range field relates to both the product identified in the product identification field 344 and the skill identified in skill identification 342. A particular product may have educational value for one skill in a first grade range, while it has educational value for another skill in a second grade range. The values in grade range 346 in skill/product table 242 provide a mechanism for representing such skill-dependent grade ranges. Each skill/product record also has a rating field 348. A particular product may have strong educational value for one skill, a moderate value for a second skill, and a minor value for a third skill. Rating 348 encodes the strength of the educational value using a three level encoding, using the numbers 1 to 3, with 3 being the greatest strength and 1 being a lesser strength. Skill product records 352, 350, and 354 are all related to a product will product identification 113 and skills 25, 12, and 28, respectively, and strengths of the relationships, 3, 2, and 1, respectively.

In this embodiment, a reviewer assigns the strength of these relationships according to the following definitions: 3. Product addresses the skill explicitly and exclusively. If the product addresses more than one skill and the treatment of this skill compares favorably with products that address this skill exclusively, the reviewer could also assign this score, however this should be done with reservation. 2. Product addresses the skill explicitly, but as part of a general treatment of the subject area. An example would be a workbook that addressed spelling, phonics, vocabulary, and usage. Each of these skills would be rated 2 for this product.

1. Product addresses the skill as a secondary subject. The product does not explicitly address the skill. An example would be a science project book that utilizes math computation and reading comprehension skills.

Math computation and reading comprehension for this product would be rated as 1

Referring to FIGS. 4A-D, a snapshot of a typical skills hierarchy 380 includes 161 different skills, arranged in seven major skills: Creative Arts, Developmental, Logical Reasoning, Math, Reading & Writing, Science, and Social Studies. It should be recognized that the skill hierarchy may be extended as new products are added to the inventory, and that skills may be added to the skill hierarchy to characterize new or existing products. For example, detailed skills for "Math/ Algebra" (FIG. 4B), which here only includes the "General" detailed skill, could be added by an administrator 154 (FIG. 1) if a new products being added to the inventory were particularly related to a particular aspect of algebra. Alternative embodiments of the system may use fewer or a greater number of skills, which may be arranges under a different number of major skill categories. Alternative embodiments of the system may also use a different hierarchical structure to arrange skills. For example, a structure may be used which allows an arbitrary number of hierarchical levels. Additionally the number of skills in each category, and the number of categories in each hierarchical level, may be different.

Referring to FIG. 5, skill/product mapping 390 illustrates the nature of the association of skills and products. Each association, which corresponds to a single record in skill/product table 242 (see FIG. 3), is illustrated as a single link between one of the skills illustrated as points on the left and one of the products illustrated as points on the right. Links 392, 394, and 396 correspond to skill/product records 350, 352, and 354, respectively, in FIG. 3. Only a small number of links are illustrated between the products and the skills. In practice, each product is linked at least one, and typically, multiple skills, and each skill is linked to multiple products.

Referring to FIGS. 6A-C, information related to a child 122 is obtained from a user 120 or in some operating modes, is obtained directly from the child. Referring to FIG. 6A, a user 120 provides an age 620 and a grade 622 for the child. The user provides this by filling out a form that is displayed by Web browser 122 on client computer 110 based on HTML data provided by Web server application 142 to Web browser 112 (see FIG. 1).

In operation of the system, user 120 accesses the system controlling Web browser 112 to request a particular Web page from server computer 140. This begins a interaction between user 120 and the system that follows the general model of many Web based systems. The interaction alternates between the system providing HTML data to Web browser 112, which the Web browser uses to form a display to the user, and the user selecting a next Web page or providing data that results in a next request being sent by Web browser 112 to server computer 140. Several operating modes are supported by the system. In general, in each operating mode, user 120 provides information about child 122 to the system, and the system in return recommends products to the user based on that information. The operating modes differ in the nature of the information provided by the user, and in the particular matching algorithm used by the system to choose products to recommend. In the discussion below, the different modes of providing information to the system are first described, followed by a description of the corresponding matching algorithms.

In a first operating mode, user 120 selects a primary learning style 626, and optionally selects a secondary learning style 628 from a list of the seven learning styles identified above that is displayed to the user by the system. The system then uses these learning styles to match the products in the inventory.

In a second operating mode, rather than directly selecting primary learning style 626 and secondary learning style 626, user 120 interacts with a learning style survey 612. Learning style survey 612 is implemented by having Web browser 112 display a series of questions to user 120. The questions are obtained from Web server application 142. The user's responses to the questions are passed back to Web server application 142, where the primary and secondary learning styles are determined from the responses. Three sets of 28 learning style questions 610 are stored on server computer 140. Each set is associated with a different grade range. The system selects the appropriate set based on grade 622 that was provided to the system by user 120. The system presents the questions in the selected set to the user. A sample set of 28 questions is shown in FIG. 7. For each of the enumerated statements in FIG. 7, user 120 is asked to respond with one of three choices related to how well the statement matches child 122 ("rarely," "sometimes," "usually"). Each question is associated with one of the seven learning styles, yielding four questions per style. A response of "usually" adds 3 to a tally for the corresponding style, "sometimes" 2, and "rarely" 1.

The learning style with the highest tally is selected by the system to be primary learning style 626, and the second highest tally is selected to be secondary learning style 628. In a third operating mode, user 120 enters a learning goal 624 for the child by selecting from a list of skills in skill hierarchy 850 (FIGS. 4A-D). In one alternative approach to selecting the learning goal, user 120 first selects a major skill from a first

"pull-down" list. Having selected the major skill, the system provides a list of minor skills for that major skill for the user to select from. Finally, the system provides a list of detailed skills that the user can select from. The system then uses the selected learning goal 624 to match to and sort the products in the inventory. This matching procedure is described in detail below.

In a fourth, combined, operating mode, the system makes use of learning goal 624, which is provided directly by user 120, as well as primary learning style 626 and secondary learning style 628, which are either provided directly by user 120 as in the first operating mode, or are produced by learning style survey 612, as in the second operating mode.

Referring to FIG. 6B, in a fifth operating mode, rather than user 120 directly providing learning goal 624 to the system, the system determines a skill assessment 640, which includes one or more skills 622 and a corresponding grade equivalent 644 for each of those skills. In this operating mode, called an "on-line skills checkup," child 122 responds to a set of skill-dependent questions 630 that are presented to the child by a skills test 632. Skills test 632 is implemented as a procedure in which Web browser 112 presents skill-dependent questions 632 to child 122, who responds to the questions on computer 110. In alternative embodiments, the questions are presented to the child "off-line," for example printed on paper, and user 120 enters the answers. Skills test 632 makes use of score norms 634 associated with the skills tests. Score norms 634 provide the data needed to convert scores on the skills test into grade equivalent 644. One example of a skills test 632 is illustrated in FIGS. 8A-E. This test is related to the "Reading&Writing"/"Reading"/"Comprehension"

(major/minor/detailed) skill, and is intended for children in grades 4-6. A child is presented one or more of the five passages shown in FIGS. 8A-E in order, and answers a set of six questions about each passage. The passages are intended to correspond to grade levels 3 through 7. If a child gets five out of the six questions for a grade level correct, the child has demonstrated that grade level and moves to the next story. In this example, skills assessment 640 includes grade equivalent 644 that is determined from the score norms 634 for this test, that is, that a child obtains at least five out of the six questions correct on the highest grade level passage.

Other skills tests 634 address multiple skills. For example, various detail skills under the "Math'V'Decimal" major/minor skill category (e.g., "Addition," "Subtraction," "Division") are addressed by one skill test 634. Each question is associated with a one of the detail skills. Using score norms 634 for those questions, a different grade equivalent 644 may be determined for each detail skill.

A sixth operating mode uses a similar approach to the on-line skills checkup. However, rather than having questions presented by Web browser 112, an off-line testing application 116 (FIG. 1) is executed on computer 110. Testing application is similar to an on-line skills checkup, however, a wider range of skills is typically addressed and more questions are asked of the child. Execution of testing application 116 may perform the entire skills test 632 (FIG. 6B) producing skill assessment 640, or alternatively, part of skills test 632 may be performed at server computer 140. For example, testing application 116 may produce raw test scores, and a procedure on server computer 140 would then application score norms 634 to determine the grade equivalents for the tested skills. Rather than entering answers or test scores directly to Web browser 112, the system has the provision to transfer data stored by testing application 116 to server computer 140, for example, under the control of user 120 through a Web browser interface, or directly from testing application 116.

Referring to FIG. 6C, a seventh operating mode is related to the on-line and off-line modes illustrated in FIG. 6B. In this mode, child 122 has taken a standardized test 652, such as the Michigan Educational Assessment Program

(MEAP) Information test or the Massachusetts Comprehensive Assessment System test. These tests include a set of standardized test questions 650. Typically, government sponsored agency scores the tests and produces standardized test scores 654 for each child, broken down into a number of subject areas.

In this operating mode, user 120 enters standardized test scores 654 into the system through the Web browser interface. The system performs a skill mapping 656 to convert the scores into a skill assessment 660, of the type produced by the on-line test. Skill mapping 656 performs two functions. First, each subject area for which a standardized score is obtained typically maps into multiple skills in the skill hierarchy. Second, for each of the subject scores the system determines a grade equivalent. In normed tests, the system derives the grade equivalent based on statistical data associated with the test and which characterizes the performance of individuals at various grade levels that have previously taken the test. In criterion tests, the system derives the equivalent grade level based on data which characterizes at which grade level various skills tested should be mastered. The result is that a skill assessment 660 typically includes more skills 662 than the number of subject scores in standardized test scores 654, and for each skill, skill mapping 656 has assigned a grade equivalent 664. In some embodiments, the function of obtaining standardized test scores 654 and transferring them to server computer 140 is performed by a third party using third party computer 160. For example, the third party may have the ability to obtain the standardized test scores directly from a government agency.

Other than in the first and second operating modes, the system matches products based at least in part on a skill associated with child 122. Referring back to FIG. 6 A, in the first and second operating mode, user 120 has either directly specified primary learning style 626 and possibly secondary learning style 628, or these learning styles were determined using learning style survey 612. User 120 has also provided grade 622 for the child. Based on the learning styles and the child's grade, Web server application 142

(FIG. 1) initiates a stored procedure 230 (FIG. 2) that performs a query and sort of the products that match the child's grade and learning styles. All products for which the child's grade is in the product's grade range 318 (FIG. 3), and for which either the primary or the secondary learning styles are indicated in the product's learning approach 316 (FIG. 3) are matched. The products are sorted according to a score that is 1 if only the secondary learning style matched, 2 if only the primary learning style matched, or 3 of both the primary and secondary learning styles matched. Within each of these scores, the products are further sorted by an average of the product's ratings 322 (FIG. 3). The highest scoring products are then presented to the user, for example, on a series of screens. In some embodiments, particular learning approaches 320 (FIG. 3), such as "Drill&Practice" are prioritized higher within products that match the learning styles equally.

In the remaining operating modes, a selected learning goal 624, which is a skill from the skill hierarchy, or skills in a skills assessment, are used in the matching process. Referring to FIG. 9, the approach to associating skills with products is illustrated. At the left of the figure, a list of desired skills 910 includes two skills 912 that are to be matched to the inventory of products 970, shown on the right. The mapping of a particular skill 912 to one or more products 970 makes use of skills hierarchy 380 and skill/product mapping 390, which are store in skill hierarchy table 244 (FIG. 2) and skill/product table 242 in database 150.

First, a skill 912 is mapped to all skills within the same major skill 382 in skill hierarchy 380. The mapping is weighted so that matching skill has a higher weight, than skills that are within the same minor skill, which have a higher weight than skills that are only within the same major skill. In this embodiment, these weights are 3, 2, and 1, respectively. Therefore, skill hierarchy 380 provides a basis for expanding a small number of targeted skills 912, into a larger number of skills, each weighted by its relationship to a target skill.

Second, product skill mapping 390 is used to associate particular products with the expanded set of skills. Recall that each association 956, which corresponds to one record in skill/product mapping table 242 (FIG. 3) is also associated with a rating, which is a number in the range 1 to 3 with 3 being the strongest association and 1 being a weaker association. In one embodiment, a score for a product is the highest product of the weighting of an expanded skill and the weighting of an association linking that expanded skill and the product. Optionally, the relative contributions of the weighting of an expanded skill and that of a rating of an association can be adjusted by multiplying one or the other by a fixed constant. If an

3-rated association links a target skill 912 and a product, that product receives a high score. If only a 1 -rated association 956 links a skill within the same major skill as the target skill and a product, that product receives a relatively low score.

Alternative embodiments combine the effects of different target skills 912, or different expanded skills in different ways. In one alternative, the best match is used to score a product. In another alterative, a sum of scores related to a product is used. That is, if a product is associated with multiple of the target skills, its score is the sum of the scores associated with each of the target skills, rather than the maximum of those scores. In another embodiment, matching is equivalent to matching each of the target skills separately, and then combining the sorted lists, thereby ensuring that at least some products related to each of the target skills are at the top of the list of recommended products. In operating modes in which a grade equivalent is associated with a target skill, as in the case in which a skill assessment for one or more skills has been generated, only associations with a grade range including the grade equivalent for a skill are used. In some alternative embodiments, user 120 is asked by the system whether products that emphasize the child's strength or products that emphasize the child's weaknesses are desired. If the user respond that emphasizing the child's strength is desired, only the skills associated with the highest determined grade equivalents are used, or alternatively, the skills for which the grade equivalent is greater than the actual grade entered for the child. Similarly emphasizing weaknesses involves using skills with relatively low grade equivalents. In some embodiments, yet other numerical scores are combined into a total score for each product. For instance, score related to one or more of the product's ratings may be combined, thereby including more products with high ratings at the top of the recommended list. Also, a "product lifetime" related to the number of grades left between a child's grade level and a product's maximum recommended grade is factored in, thereby including products that should be relevant to the child for more years higher in the list of recommended products. Another alternative score to combine into a product's overall score is a "focus" score that is higher for a product that is associated with a smaller number of skills than for a product that is associated with a large number of skills.

In some embodiments, the numerical score obtained for a product based on the matching to one or more target skills is combined with a score determined from a child's primary and secondary learning styles.

In alternative embodiments, alternative structures than skills hierarchy 380 are used to represent the interrelationship of skills or other attributes used to characterize the products. For example, pair-wise relationships between skills may be explicitly stored, or skills may belong to overlapping categories. In each of these alternatives, specification of one desired skill allows the system to derive a set of related skills that may be matched with products in the product inventory. Referring back to FIG. 1, administrator 154 uses database maintenance application 152 to maintain database 150. When a new product is added to the inventory of products, a new product record 330 (FIG. 3) is added to product table 240. In addition, one or more skill/product records are added to skill/product table 242, each record associating the new product with one of the existing skills in the skill hierarchy. If the new product is related to a skill not yet in the skill hierarchy, that new skill can first be added by adding a record in skill hierarchy table 244. Then a skill/product record associating the new product to the new skill is added to skill/product table 242. In this way, the set of skills is extensible without affecting existing data that associates previously entered products with previously defined skills. Referring back to FIG. 1, in alternative embodiments, other forms of data networks couple client computers 110 and server computer 140. For example, a private "intranet" can couple the client and server computers. Also, rather than coupling the computers over a data network, client computers 110 can alternatively connect directly to server computer 140, for instance using dialup telephone modem connections, or dedicated connections such a leased telephone lines.

The system described above makes use of Web browser 112 executing at client computer 110 interacting with Web server application 142 executing at server computer 140. Alternative types of applications and servers can be used. For example, a client application can execute at client computer 110 and communicate with an server application executing at server computer 140 using a proprietary protocol.

Furthermore, the system shown in FIG. 1 makes use of a client/server architecture in with a particular division of functions between the client and server computers. In alternative embodiments, different divisions of functions are used, and different types of databases and database procedures may be used. For instance, server computer 140 may host database 150, but the software that performs the functions of the database procedures, such as Java applets, can be transferred to client computer 110 and executed there to access database 150 remotely over Internet 130. In FIG. 1, server computer 140 is shown as a single computer. In alternative embodiments, the functions of server computer 140 can be distributed among several computers. For instance, database server application 148, database procedures 146 and database 150 may be hosted on a different computer than Web server application 142 and Web pages 144. Furthermore, multiple Web server applications 142 and Web pages 144 may be hosted on different computers, and interconnected by links in the Web pages returned to the client computers.

It is to be understood that the foregoing description is intended to illustrate and not to limit the scope of the invention, which is defined by the scope of the appended claims. Other embodiments are within the scope of the following claims.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented educational product recommendation system comprising: a storage for data characterizing relationships between product attributes within a set product attributes, wherein the set of product attributes includes a set of educational skills; a storage for data associating each of a plurality of products with one or more of the product attributes; and software stored on a computer-readable medium for causing a computer to determine one or more desired product attributes, access attribute relationship data from the storage for data characterizing relations between the product attributes, determine related product attributes based on the desired product attributes and the attribute relationship data, access product association data from the storage for data associating products with product attributes, and determine a matching set of the products using the product association data and the desired and related product attributes.

2. A system of claim 1 wherein the storage for associating each of the plurality of products with one or more of the product attributes includes a database table for holding a plurality of records, each record associating one of the products with one of the product attributes.

3. The system of claim 1 wherein the relationships between product attributes include inclusion of particular ones of the educational skills within the same category of educational skills, and wherein determining related product attributes includes determining educational skills that are in the same category of educational skills as a desired educational skill.

4. The system of claim 3 wherein the relationships between product attributes includes inclusion of a number of categories of educational skills within a broader category of educational skills.

5. The system of claim 3 wherein determining one or more desired product attributes includes determining test scores for an individual, and mapping the tests scores to a quantity related to one of the educational skills.

6. The system of claim 5 wherein the quantity related to one of the educational skills is an equivalent grade level.

7. Software stored on a computer-readable medium for causing a computer system to : determine one or more desired product attributes from a set of product attributes, wherein the set of product attributes includes a set of educational skills, access attribute relationship data from a storage for data characterizing relationships between product attributes in the set of product attributes, determine related product attributes based on the desired product attributes and the attribute relationship data, access product association data from a storage for data associating each of a plurality of products with one or more of the product attributes, and determine a matching set of the products using the product association data and the desired and related product attributes.

-23/1-

8. A method for recommending educational products comprising: determining one or more desired product attributes from a set of product attributes, including determining one or more desired educational skills; accessing attribute relationship data from a storage for data characterizing relationships between product attributes in a set of product attributes; determining related product attributes based on the desired product attributes and the attribute relationship data; accessing product association data from a storage for data associating each of a plurality of products with one or more of the product attributes; and determining a matching set of the products using the product association data and the desired and related product attributes.

9. The method of claim 8 wherein determining one or more desired educational skills includes: providing a questionnaire including a plurality of questions; and accepting responses to the questions.

10. The method of claim 9 further comprising: determining an equivalent grade level associated with each of the one or more educational skills based on the accepted responses.

11. The method of claim 8 wherein determining one or more desired educational skills includes: accepting test scored obtained on a standardized test; and determining an equivalent grade level associated with each of the one or more educational skills based on the accepted test scores.

12. The method of claim 11 wherein determining an equivalent grade level includes comparing the accepted test scores with statistical data characterizing a distribution of test scores for a grade level. _

13. The method of claim 8 wherein determining related product attributes based on the desired product attributes and the attribute relationship data includes determining related educational skills that are in a same category of educational skills as at least some of the desired educations skills.

14. The method of claim 13 wherein determining one or more desired product attributes includes determining test scores for an individual, and mapping the test scores to a quantity related to one of the educational skills.