US9064017B2 - Systems and methods for providing information incorporating reinforcement-based learning and feedback - Google Patents
Systems and methods for providing information incorporating reinforcement-based learning and feedback Download PDFInfo
- Publication number
- US9064017B2 US9064017B2 US13/486,923 US201213486923A US9064017B2 US 9064017 B2 US9064017 B2 US 9064017B2 US 201213486923 A US201213486923 A US 201213486923A US 9064017 B2 US9064017 B2 US 9064017B2
- Authority
- US
- United States
- Prior art keywords
- user
- state
- information
- action
- information resources
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G06F17/30867—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
Definitions
- the invention relates to providing information based on feedback.
- Particular embodiments provide systems and methods for feedback-driven provision of information in relation to a plurality of information resources.
- the information provided comprises recommended actions for users in connection with information resources, which may include educational information or content.
- Provided information and/or recommended actions may be personalized for individual users and/or groups of users.
- the internet which is accessible to many students, provides an often overwhelmingly large amount of content.
- Other sources of educational resources e.g. libraries, book stores, educational service providers
- Such content may be related to the academic interests and needs of students, but is sometimes completely irrelevant.
- Unknowingly due, for example, to the amount of content and/or a lack of organization of educational resources tailored to a student's needs, a student can make poor choices about what educational resources to study or observe, and for how long. If a student is motivated to learn, he or she can be persuaded to choose educational resources that suit their learning style and pace; however, it can be difficult to locate such resources.
- Issues relating to the overwhelming nature of amount, variety and/or lack of organization of content available from the internet and from other sources is not limited to educational content. These issues apply generally to other information resources available from such sources. There is a general desire to provide users with feedback-driven recommendations for actions in connection with such information resources.
- Feedback can be a powerful tool for assessing the value of information to a user or users.
- FIG. 1 is a schematic system diagram of a possible architecture of a learning system incorporating reinforcement learning techniques according to a particular embodiment
- FIG. 2A shows a resource item data structure suitable for use by the FIG. 1 learning system according to a particular embodiment
- FIG. 2B shows an action item data structure suitable for use by the FIG. 1 learning system according to a particular embodiment
- FIG. 2C shows a feedback item data structure suitable for use by the FIG. 1 learning system according to a particular embodiment
- FIG. 2D shows a user item data structure suitable for use by the FIG. 1 learning system according to a particular embodiment
- FIG. 3A is a schematic example of a user action log suitable for use by the action interface of the FIG. 1 learning system according to a particular embodiment
- FIG. 3B is a schematic resource-space (or model-space) diagram corresponding to the FIG. 3A example user action log;
- FIG. 4A shows an example of a state table which may be created by a ranking engine of the FIG. 1 learning system according to a particular embodiment
- FIG. 5A illustrates a method for using reinforcement learning together with one or more feedback metrics to update values of a state table maintained by the FIG. 1 learning system according to a particular embodiment
- FIG. 5B is a schematic depiction of a method for providing information about recommended actions for a user of the FIG. 1 learning system according to a particular embodiment
- FIG. 5C schematically depicts a method for selecting a target state which may be suitable for use in the action recommendation method of FIG. 5B ;
- FIG. 5D schematically depicts an optional method for determining a next best action or actions for a user after just having completed an action according to a particular embodiment
- FIG. 5E schematically depicts a method for providing information about a path comprising a structure (e.g. ordered) series of recommended actions which may be suitable for use in the action recommendation method of FIG. 5B according to a particular embodiment;
- FIG. 5F schematically depicts a method for providing information about a set of one or more recommended actions which may be suitable for use in the action recommendation method of FIG. 5B according to a particular embodiment
- FIG. 5G schematically illustrates one method for using a diagnostic assessment which may be suitable for use in the action recommendation method of FIG. 5B according to a particular embodiment.
- aspects of the invention provide methods and systems for providing information based on feedback.
- Feedback may be incorporated into the information provided using reinforcement learning.
- Information provided by the methods and systems of particular embodiments can comprise information about feedback-driven recommendations for actions in connection with accessible information resources.
- information resources comprise educational information or content.
- Feedback may comprise one or more feedback metrics which may be related to a user's interaction with the information resources.
- feedback comprises an assessment which comprises a feedback metric (or metrics).
- reinforcement learning techniques use these feedback metrics to ascribe, or otherwise determine, one or more values for an action or a series of actions taken by a user in connection with the information resources. Such values may be used as estimates of the value of the same action or series of actions for other users.
- the set of accessible information resources may be referred as a state space and information about current position of a user in the state-space (e.g. a history of the information resources with which the user has interacted) may be referred to as the user's state.
- a user interacts with an information resource.
- Such interaction of the user with an information resource may be referred to as an action.
- a current state of a user coupled with an action which will transition the user to a new state may be referred to as a state-action pair.
- the interaction of states and actions and how an action taken by a user transitions the user from one state to another state may be referred to as a model.
- reinforcement learning techniques may use feedback to ascribe, or otherwise determine, one or more values for state-action pairs.
- State-action pairs together with their corresponding values may be maintained in a state table.
- Such a state table may be used as a basis for providing information about recommended actions to a variety of users.
- Information provided may be personalized for individual users and/or groups of users.
- FIG. 1 is a schematic system diagram of a possible architecture of a system 100 capable of feedback-driven information retrieval according to a particular embodiment.
- system 100 comprises a learning system and the feedback-driven information retrieved by learning system 100 comprises feedback-driven recommendations for user actions in relation to information resources.
- Such information resources may comprise educational information or content.
- Learning system 100 can access information resources from one or more information resource repositories 150 .
- information resource repositories include the internet 150 A, one or more general purpose information resource databases 150 B and information resources which may be accessed from a learning management system 150 C.
- learning system 100 can interact with a different number (more or fewer) of information resource repositories, different types of information resource repositories and/or the like.
- information resource repositories 150 may comprise electronically accessible information repositories.
- information resource repositories 150 may be accessible by user 142 and/or by learning system 100 over a suitable data communications network (e.g. the internet, a private network and/or the like).
- information resources provided on a given information resource repository 150 may be created and/or managed by more than one entity.
- the internet (repository 150 A) is completely open-ended and accessible information resources provided on the internet (repository 150 A) may be contributed by an arbitrary number of independent content providers.
- a particular information resource repository 150 may comprise a topical information repository 150 which may accept information resources from a number of expert contributors.
- an information repository 150 could be a topical repository 150 on the topic of astronomy, in which case it may accept contribution of information resources from a number of astronomy experts.
- Information resource repositories 150 described herein are merely representative examples of suitable types of information repositories 150 and, unless specifically claimed, are not meant to be limiting.
- Information resource repositories 150 may hold a wide variety of information resources having a corresponding wide variety of forms.
- information resources can comprise textual resources, audio resources, image-based resources, video resources, interactive resources, questions, assessments, executable applications, instructions or directives on how to access and/or use other resources, discussion posts or forums, instructor notes, hints, blogs, any combinations or sub-combinations of these types of resources and/or the like.
- learning system 100 can accommodate any form of informational resource.
- information resources may be pre-organized or otherwise mapped or classified in some manner within the repository prior to being made accessible to learning system 100 . This is not necessary, however.
- information resources are not organized in any particular manner.
- Learning system 100 interacts with information resource repositories 150 through information resource interface (or for brevity resource interface) 112 .
- resource interface 112 is implemented as an application program interface (API) capable of electronically accessing information resources from, or otherwise interacting with, repositories 150 , although this is not necessary.
- Resource interface 112 may pull information resources from repositories 150 and/or repositories 150 may push information resources to resource interface 112 .
- resource interface 112 may pull information resources from one or more repositories (e.g. the internet 150 A) while other repositories (e.g. learning management system 150 C) push information resources to resource interface 112 .
- resource interface 112 may crawl one or more of repositories 150 to gather data in respect of the information resources accessible therein.
- resource interface 112 creates information resource item data structures which may be stored in information resource and classification database 114 for access by the other components of system 100 .
- FIG. 2A shows an information resource item data structure (or for brevity resource item) 200 suitable for use by learning system 100 according to a particular embodiment.
- Resource item 200 of the illustrated embodiment comprises three fields: resource ID; resource type; and resource URI.
- resource item data structure 200 may comprise additional or alternative fields.
- Resource ID is a field which uniquely identifies a particular resource item.
- Resource URI is a pointer to the information resource item in one or more of repositories 150 .
- resource URI may comprise the actual universal remote locator (URL) corresponding to the information resource.
- the resource URI field of resource item 200 may comprise any suitable pointer to the underlying information resource.
- Resource item 200 also comprises a resource type field.
- a resource item may be classified into one of four types: general information resource, taxonomy, assessment and question.
- a resource item 200 with an assessment type means that the underlying information resource provides some capacity for testing the user or otherwise providing feedback as to the user's progress in connection with one or more learning objectives.
- an assessment may comprise a plurality of questions which may be posed to a user and then evaluated. This is not necessary, however, and in general, an assessment can comprise any means of providing user feedback in connection with one or more pedagogical objectives.
- Assessment type resource items can come from a variety of different sources.
- a curriculum-setting authority such as a school board or a government education branch and/or the like
- private entities such as the proprietors of learning management system 150 C and/or resource databases 150 B
- individual teachers who may or may not be affiliated with repositories 150
- learning management system 100 may comprise a specialized user interface (not shown) for adding independently created assessments, but this is not necessary.
- a question type resource item 200 may be the building block of an assessment type resource item. That is, an assessment type resource item may comprise a plurality of question type resource items. As explained in more detail below, in some embodiments, it can be useful to parse assessment type resource items into questions, so that diagnostic assessments can be used to make personalized recommendations for a user. Taxonomy type resource items are described in more detail below in the context of mapping engine 116 .
- General type resource items may comprise resource items other than assessment type resource items and question type resource items.
- Resource items of the general type may be referred to herein as information resource or just resources.
- information resources may have a wide variety of forms.
- information resources can comprise textual resources, audio resources, image-based resources, video resources, interactive resources, questions, assessments, executable applications, instructions or directives on how to access another resources, discussion posts or forums, instructor notes, hints, blogs, any combinations or sub-combinations of these types of resources and/or the like.
- different information resources may have different pedagogical value. This is particularly the case when information resources are accessed from a wide variety of repositories 150 which may include the internet 150 B.
- learning system 100 may use feedback to provide information about recommended actions for users in relation to information resources that will help such users achieve their learning objectives.
- learning system 100 may comprise an assessment creation engine (not shown) which may create one or more questions and/or assessments based on accessible information resources. Assessments created in this manner may comprise a plurality of question type resource items which may be determined by mapping engine 116 (discussed further below) to have a threshold degree of similarity or some other suitable relationship with one another.
- Learning system 100 is not limited to the four resource types discussed herein. In some embodiments, learning system 100 may comprise other additional or alternative resource types.
- the assessment type, question type and/or general type resource items could be further sub-classified.
- general type resource items could be further sub-divided into video type, graphic type, textual type, interactive type, etc.
- assessment type resource items could be sub-classified into: random type (which comprises a random plurality of questions), adaptive type (which comprises questions based, at least in part on answers given to previous questions) and/or the like.
- question type resource items could be sub-classified into: multiple choice type, short answer type, true/false type and/or the like.
- mapping engine 116 may perform semantic analysis (or other data mining analysis) in relation to accessible information resources. Semantic analysis, which may be performed by mapping engine 116 using any suitable algorithm(s) or technique(s), may be used to establish mappings or other similar relationships as between information resources accessible to resource interface 112 . For example, mapping engine 116 may establish such mappings as between information resources in any particular repository 150 and/or across all accessible repositories 150 . Semantic analysis may be performed by mapping engine 116 on accessible information resources regardless of whether such information resources are pushed to, or pulled by, resource interface 112 .
- semantic analysis or other data mining analysis
- mapping engine 116 comprises clustering (e.g. hierarchical agglomerative clustering (HAC)) and/or some other suitable mapping technique(s) which may be based on semantic analysis.
- clustering e.g. hierarchical agglomerative clustering (HAC)
- mapping techniques may provide metrics which assess the semantic similarity of any one information resource item to other information resource items.
- a metric which may be used to assess similarity in the mapping techniques performed by mapping engine 116 comprises cosine similarity, although it will be appreciated that other suitable similarity algorithms and/or metrics could be used.
- mapping techniques represent one class of semantic analysis techniques and that mapping engine 116 may additionally or alternatively make use of other semantic analysis (or data mining analysis) techniques.
- Mapping engine 116 may be generally capable of establishing mappings to assess the similarity of any accessible information resources.
- mapping engine 116 may initially (or otherwise) be provided with taxonomy information (not explicitly shown).
- taxonomy information may comprise taxonomy type information resources and may be provided to mapping engine 116 through resource interface 112 , from resource and classification database 114 or otherwise.
- taxonomy information may comprise a set of one or more topics, skills and/or objectives for learning.
- taxonomy information may include course syllabi, table(s) of contents and/or the like.
- a taxonomy type information resource may be the base unit of a set of taxonomy information.
- Taxonomy type information resources may be presented to mapping engine 116 as documents or the like. For each such taxonomy type information resource, mapping engine 116 may then determine mappings between the taxonomy type information resource and other information resources accessible to mapping engine 116 (e.g. general type information resources, question type information resources, assessment type information resources and/or other taxonomy type information resources).
- mapping engine 116 may then determine mappings between the taxonomy type information resource and other information resources accessible to mapping engine 116 (e.g. general type information resources, question type information resources, assessment type information resources and/or other taxonomy type information resources).
- a user 142 interacts with learning system 100 via a user interface 154 .
- user interface 154 may comprise a specialized (e.g. independent) application which, when executed, provides user 142 with a comprehensive interface to learning system 100 .
- user 142 may access resources from repositories 150 directly via user interface 154 .
- user interface 154 may create (or may have access to) resource interaction activity data 156 .
- resource interaction activity data 156 may comprise an identification of user action (e.g.
- resource interaction activity data 156 may comprise the output of LMS gradebook(s) or the like which may provide grades for assessments taken by user 142 .
- an assessment manager 128 may be provided to determine or otherwise obtain grades for assessments taken by user 142 . As discussed in more detail below, such grades may be used by system 100 as feedback metrics.
- user interface 154 to system 100 may operate passively while user 142 accesses information resources from repositories 150 via another independent application program (not expressly shown).
- user interface 154 to system 100 may operate in parallel with, or in the background of, the independent application.
- the independent application may provide an independent user interface through which user 142 accesses information resources from repositories 150 .
- such an independent could comprise an internet browser or a LMS user interface application.
- user interface 154 to system 100 may passively monitor the actions of user 142 as they interact with resources from repositories 150 via the internet browser and may create (or may have access to) browser activity data 152 .
- browser activity data 152 may comprise an identification of user actions (e.g. the resource ID of resources that user 142 has accessed or otherwise interacted with) and a temporal indication (e.g. one or more time stamps) of when and/or how long user 142 interacted with a particular resource.
- browser activity data 152 may comprise grades for assessments taken by user 142 .
- an assessment manager 128 may be provided to determine or otherwise obtain grades for assessments taken by user 142 . As discussed in more detail below, such grades may be used by system 100 as feedback metrics.
- System 100 may be accessed by a plurality of users 142 , who may use the same user interface 154 or different user interfaces 154 . To keep track of individual users 152 , system 100 may create user item data structures which may be stored in user database 158 for access by the other components of system 100 .
- FIG. 2D shows a user item data structure 240 suitable for use by learning system 100 according to a particular embodiment.
- User item data structure 240 of the FIG. 2D embodiment comprises two fields: user ID and user profile.
- User ID is a field which uniquely identifies a particular user.
- the user profile field is optional and may comprise any of a wide variety of information specific to a particular user, including (by way of non-limiting example): the user's education stage (e.g.
- the user profile field could be further sub-classified to provide user item data structure 240 with a plurality of user profile sub-fields.
- User data structure 240 shown in the FIG. 2D includes two fields: user ID and user profile. In other embodiments, user data structure 240 may comprise additional or alternative fields.
- Action interface 120 is implemented as an application program interface (API) capable of electronically tracking the actions of user 142 , although this is not necessary.
- Action interface 120 provides a link between resource items 200 ( FIG. 2A ) and user items 240 ( FIG. 2D ).
- Action interface 120 may pull user action data from any one of more of: user 142 , browser activity data 152 , resource interaction activity data 156 , user interface 154 , assessment manager 128 and/or the like. This interaction is shown schematically in FIG. 1 by the dashed box around user 142 , browser activity data 152 , resource interaction activity data 156 , user interface 154 and assessment manager 128 .
- action interface 120 may pull user action data from one or more of repositories 150 (not shown). Additionally or alternatively, any one or more of browser activity data 152 , resource interaction activity data 156 , user interface 154 , assessment manager 128 , repositories 150 and/or the like can push user action data to action interface 120 .
- Action interface 120 may track the actions of users 142 in relation to the resources in repositories 150 using an action log which may be stored in action database 124 .
- each record in an action log stored in action database 124 is based on an action item.
- FIG. 2B shows an action item data structure 220 suitable for use by learning system 100 according to a particular embodiment.
- Action item data structure 220 of the FIG. 2B embodiment comprises three fields: user ID; resource ID and time stamp(s). As discussed above, user ID and resource ID uniquely identify a particular user and a particular information resource item.
- the time stamp field(s) of action item data structure 220 may comprise a log of the time(s) during which a particular user accessed a particular resource item.
- time stamp field(s) could comprise a single time stamp field indicating that the user accessed the information resource at a particular time or for a particular duration.
- time stamp field(s) could include a plurality of time stamps which could be used to provide more comprehensive information about when and for how long the user accessed the information resource. For example, multiple time stamps could be used to indicate that the user accessed the resource between 13:30 and 14:14 on 21 May 2012 and between 13:40 and 14:45 on 22 May 2012.
- Feedback interface 118 is implemented as an application program interface (API) capable of electronically procuring feedback metrics, although this is not necessary.
- API application program interface
- Feedback interface 118 provides learning system 100 with feedback metrics (discussed further below).
- feedback metrics obtained by feedback interface 118 may comprise feedback relating to the interaction of user 142 with assessments (e.g. resource items of the assessment type).
- assessments e.g. resource items of the assessment type.
- assessments e.g. resource items of the assessment type
- assessments e.g. resource items of the assessment type
- assessments e.g. resource items of the assessment type
- an assessment grade or some other suitable assessment metric may be generated.
- such an assessment metric could be generated by the gradebook (not shown) of a LMS 150 C (or any other LMS) and reported as part of resource interaction activity data 156 .
- Feedback interface 118 may pull feedback metrics (e.g. user assessment grades or metrics) from any one of more of: user 142 , browser activity data 152 , resource interaction activity data 156 , user interface 154 , assessment manager 128 and/or the like. This interaction is shown schematically in FIG. 1 by the dashed box around user 142 , browser activity data 152 , resource interaction activity data 156 , user interface 154 and assessment manager 128 . Additionally or alternatively, any one or more of user 142 , browser activity data 152 , resource interaction activity data 156 , user interface 154 , assessment manager 128 and/or the like can push user assessment data to feedback interface 118 . In some embodiments, feedback interface 118 can additionally or alternatively receive feedback metrics from external source (not shown), such as instructors, external assessment source(s) (which may include other system(s) capable of providing feedback), peers and/or the like.
- external source not shown
- external assessment source(s) which may include other system(s) capable of providing feedback
- Feedback interface 118 may optionally track the feedback metrics of users 142 in a feedback database 122 .
- FIG. 2C shows a feedback item data structure 230 suitable for use by learning system 100 according to a particular embodiment.
- Feedback item data structure 230 of the FIG. 2B embodiment comprises four fields: user ID; resource ID; time stamp(s); and feedback metric.
- user ID and resource ID uniquely identify a particular user and a particular information resource item.
- the time stamp field(s) of feedback item data structure 230 may be similar to those of action item data structure 220 discussed above.
- the time stamp field(s) of feedback item data structure 230 comprise a log of the time(s) during which a particular user accessed a particular assessment type resource item.
- the feedback metric field of feedback item data structure 230 may comprise one or more numbers which may, in the case of feedback derived from user interaction with an assessment type resource, indicate the performance of the user in relation to the identified assessment.
- the feedback metric field of feedback item data structure 230 may comprise a percentage grade of the identified assessment for the identified user.
- Feedback which may be provided to, or otherwise accessed by, feedback interface 118 is not limited to assessments.
- feedback may comprise any vehicle for providing a feedback metric.
- the feedback metric may be related to a user's sequence of actions since the last time that feedback was provided (e.g. interacting with a particular set of information resources, interacting with information resource(s) a number of times and/or the like).
- Non-limiting examples of other types of feedback include: user evaluation of information resources or actions taken in relation to information resources; instructor evaluation of user comprehension or mastery of a particular learning objective; completion of a threshold number of actions in relation to information resources which may be ascribed a suitable feedback metric; completion of one or more particular actions in relation to information resources which may be ascribed a suitable feedback metric and/or the like.
- Such other forms of feedback may, but need not necessarily be represented in feedback database 122 by feedback item data structure 230 .
- feedback item data structures having different fields may be used to represent such other forms of feedback.
- an assessment manager 128 may be provided which may perform the task of generating or otherwise obtaining and/or managing the feedback metrics for feedback interface 118 .
- assessment manager 128 can reside on the computing device of user 142 and can interact with different sources of feedback, such as resource interaction activity data 156 (which may include information from the gradebook of a LMS system), browser activity data 152 , user 142 , user interface 154 and external feedback sources (not shown).
- each row (e.g. each record) of user action log 250 represents one action and comprises four fields: user ID, state, action, start time stamp and end time stamp.
- the state field represents actions that the user has done prior to the action the user is currently performing.
- the action field represents the action that the user is currently doing.
- Possible actions for the action field of user action log 250 may correspond to the types of information resources. As discussed above, in one particular embodiment, information resources may have a type of assessment, question, taxonomy item or general resource; in such embodiments, possible actions include various forms of interaction with these types of information resources.
- possible forms of interaction with these resources include reading, viewing, listening to and/or taking a more active part in a general resource, answering a question, taking an assessment and/or interacting with an instructor in relation to any one of these resources.
- the time stamp start and end fields respectively represent the start and end times of a particular action.
- each record of user action log may comprise a single time stamp field (e.g. start time). Comparing action data item 220 ( FIG. 2B ) and user action log 250 , it can be seen that each record of user action log 250 includes information which may be drawn from a corresponding action item of the form of action item 220 ( FIG. 2B ).
- FIG. 3B is a schematic resource-space diagram 255 corresponding to the FIG. 3A action log 250 .
- each square corresponds to a resource item and is labeled with its corresponding resource ID.
- the circles in FIG. 3B represent actions that the user has done and the dashed line represents the path that the user took between actions.
- FIG. 3B shows that the user progressed through interacting with resource items 1 , 3 , 4 , 7 and 6 before taking assessment A4.
- learning system 100 of the illustrated embodiment comprises a ranking engine 126 which may use information from resource classification database 114 , action database 124 , feedback database 122 , user database 158 and/or any other information available to system 100 to maintain relationships between information resource items and their respective values. These relationships may be maintained by ranking engine 126 in a state table 130 .
- FIG. 4A shows an example of a state table 275 which may be used by ranking engine 126 according to a particular embodiment.
- each row (record) in state table 275 comprises a plurality of fields: a state field 277 ; an action field 279 ; and at least one value field 281 .
- state table 275 may optionally comprise a plurality of value fields.
- state table 275 in addition to the at least one value field 281 , state table 275 also comprises optional value fields 2-n indicated by reference numeral 283 .
- optional value fields 283 may comprise customized or personalized value information.
- State field 277 represents a state of a user prior to the action 279 of the current record. State field 277 may comprise references to the resource IDs of particular resource items with which users may interact.
- the third record 285 of the FIG. 4A example state table 275 has a state field 277 which indicates a state where a user has previously interacted with resource items with resource IDs of 1, 4 and 7.
- Action field 279 represents a next action.
- action field 279 of state table 275 includes possible actions which correspond to the information resource types (e.g. general information resources, assessments and questions) being used by resource interface 112 of learning system 100 together with a resource ID reference.
- a particular state and a particular action of a single record of state table 275 may be referred to herein as a state-action pair.
- state table 275 includes at least one value field 281 which may represent the value that system 100 associates with performing a corresponding action, given a corresponding state.
- Value field 281 may comprise a numerical metric, such that value fields 281 of particular state table records may be easily compared to one another. In the case of the FIG.
- state table 275 comprises optional additional value fields 283 .
- additional value fields 283 may be similar to value field 281 , but may be customized or personalized.
- additional value fields 283 may be personalized by filtering or otherwise evaluating values in accordance with one or more suitable evaluation criteria.
- optional additional value field(s) 283 may be filtered or otherwise evaluated in accordance with any suitable evaluation criteria. For example, it will be appreciated that particular state-action pairs may have different values for users who have different learning aptitudes (or learning styles). One user may have a greater aptitude for visual learning and another user may have a greater aptitude for interactive learning.
- state-action pairs may be assigned a first value field for users with a visual learning aptitude and a second value field for users with an interactive learning aptitude.
- value fields 283 of state table 275 may be filtered or otherwise evaluated according to any suitable evaluation criteria.
- suitable evaluation characteristics which may be used as a basis for value fields 283 of state table 275 include: characteristics of resources (for example, evaluating video-based information resources only, evaluating interactive resources only and/or the like), characteristics of assessments (for example, evaluating assessments involving multiple choice questions only, audio assessments only and/or the like), characteristics of actions (for example, evaluating assessment only), characteristics of users (for example, grade level, learning disability, and/or the like).
- evaluation criteria may be procured by resource interface 112 , action interface 120 , feedback interface 118 and/or user interface 154 .
- evaluation criteria may be procured as metadata which may be provided in the form of additional fields to one or more of resource item 200 , action item 220 , feedback item 230 and/or user item 240 (see FIGS. 2A-2D ).
- a state table may comprise a plurality of value fields 283 wherein each value field is filtered or otherwise evaluated according to different evaluation criteria. In such embodiments, it is not necessary that there be an unfiltered or non-evaluated value field 281 .
- Ranking engine 126 may update state table 130 in response to feedback obtained from feedback interface 118 and in response to user actions which may be obtained by action interface 120 .
- FIG. 5A illustrates a method 300 for updating state table 130 according to a particular embodiment.
- Method 300 commences in block 302 which involves monitoring user actions involving interaction with information resources contained in repositories 150 . Whenever such an action is detected (for example by action interface 120 and/or feedback interface 118 ), method 300 proceeds to block 304 which involves updating a user action log.
- updating a user action log may involve adding a record (e.g. a row) to the user action log to correspond with the action that has just been observed in block 302 .
- Method 300 then proceeds to block 306 which involves an inquiry into whether the block 302 action involves a feedback-generating action.
- a user interacting with an assessment represents one non-limiting example of a feedback-generating action.
- there may be additional or alternative feedback-generating actions which may be based on particular types of actions and/or corresponding types of information resources.
- a feedback-generating action may comprise any event which comprises or otherwise provides a feedback metric.
- An action giving rise to any of the various forms or types of feedback described above may comprise feedback-generating actions.
- additional or alternative feedback-generating actions could include: the provision of a feedback metric by an instructor (e.g.
- method 300 loops back to block 302 .
- method 300 proceeds to block 308 which involves updating the values of the state table.
- the block 308 process of updating values of the state table may depend on the feedback metric associated with the feedback-generating action. Assuming, for the sake of example, that the feedback-generating action evaluated in block 306 corresponds to an assessment, then the feedback metric may comprise a grade of the corresponding block 306 assessment and the block 308 process of updating the values of the state table may depend on this grade.
- the records of the state table that may be updated in block 308 may correspond to the state-action pairs in the user action log since the last time that the user performed a feedback-generating action and may be reflective of the path that the user took from the previous feedback-generating action to the current feedback-generating action.
- the state-action pairs in the user action log prior to taking assessment A4 include: (null
- the block 308 process of updating the state table involves the application of reinforcement learning techniques.
- the block 308 process of updating the state table may involve the two step process of: selecting the state table records to update (as shown in optional block 308 A of the illustrated embodiment); and selecting one or more new values for each selected state table record (as shown in optional block 308 B of the illustrated embodiment).
- the block 308 A process of selecting state table records to be updated may involve selecting the state table records with state-action pairs corresponding to the state-action pairs in the user action log since the last time that the user performed a feedback-generating action.
- the block 308 B process of selecting a new value for the corresponding record may involve applying one or more value functions which depend on the previous value of the selected record and which also depend on the feedback metric of the corresponding block 306 feedback-generating action.
- value functions may permit differential weighting of different state-action pairs—for example, by weighting more recent state-action pairs relatively more heavily and by weighting more temporally distant state-action pairs relatively more lightly.
- the block 308 B process of updating the values of the selected state table records may (but need not necessarily) be performed for each value field.
- the value function(s) used for each value field may be the same or may be different.
- updating the values of the selected state table records in block 308 B involves using the state table in a model of a finite Markov Decision Process (MDP) and using a reinforcement learning technique to approximate solutions for updating the values of the selected state table records.
- MDP finite Markov Decision Process
- a suitable approximation technique for the value function of the model involves temporal difference (TD) reinforcement learning which may be formulated to take advantage of the so-called eligibility trace ⁇ .
- the eligibility trace ⁇ may be a number between [0,1] which may be used to weight the relevance of past steps (e.g. past states) to a current outcome (e.g. feedback).
- the block 308 process of updating the values of the selected state table records involves application of a reinforcement learning technique known as Q-Learning.
- the block 308 process of updating the values of the selected state table records involves application of a reinforcement learning process known as State-Action-Reward-State-Action (or SARSA) Learning.
- a Monte Carlo method may be used in the block 308 process of updating the values of the selected state table records.
- block 308 B involves using the feedback metric of the block 306 feedback-generating action to update selected records of the state-table.
- the records updated in block 308 B may be selected (in block 308 A) based on the state-action pairs in the user action log since the last feedback-generating action.
- the updates to the values of the selected records may be determined based on a reinforcement learning technique.
- the updates to the selected records may be individually weighted. In some embodiments, such weighting may involve the application of relatively heavy weights to records corresponding to relatively more recent state-action pairs (determined from the user action log) and relatively lighter weights to records corresponding to relatively more temporally distant state-action pairs (determined from the user action log).
- Method 300 ends after the state table is updated at the conclusion of block 308 .
- learning system 100 of the illustrated embodiment may comprise a recommendation engine 140 which provides information in respect of recommended actions for user 142 (e.g. information resources for user 142 to interact with and/or assessments for user 142 to take).
- Recommendation engine 140 may base the recommended actions on one or more learning objectives.
- Such learning objectives may comprise or be based on one or more taxonomy items.
- Such learning objectives may be provided by user input (e.g. user 142 may specify a learning objective or the like through user interface 154 , such as by selecting a taxonomy item, for example).
- Learning objectives may also be based on information from: resource classification database 114 , action database 124 , feedback database 122 , user database 158 , state table 130 and/or any other information available to system 100 .
- Recommendation engine 140 may generally involve providing information in respect of one or more recommended actions which may help user 142 to achieve his or her learning objectives. Recommendation engine may provide this information about recommended actions via user interface 154 .
- recommendation engine 140 may provide information about an action path which may comprise a structured series of one or more recommended actions in relation to accessible information resources. Such a structured series of recommended action(s) may comprise an ordered series of recommended actions where the action path is considered by recommendation engine 140 to be an optimum path for achieving the learning objective.
- a recommended action path culminates in a feedback-generating action, although this is not strictly necessary.
- recommendation engine 140 may additionally or alternatively provide information in respect of a non-ordered list of one or more recommended actions in relation to accessible information resources which are considered by recommendation engine 140 to be have value in relation to the learning objective.
- the information about recommended actions output by recommendation engine 140 may be refreshed whenever user 142 performs a feedback-generating action (e.g. completes an assessment) and/or the values of state table 130 are otherwise updated (see, for example, method 300 for updating the state table values ( FIG. 5A )).
- the information about recommended actions output by recommendation engine 140 may be refreshed after each user action in relation to the accessible information resources.
- FIG. 5B is a schematic depiction of a method 350 for providing information about recommended actions for user 142 according to a particular embodiment.
- Method 350 may be referred to herein as action recommendation method 350 .
- Method 350 commences in block 352 which involves procuring one or more user learning objectives.
- the block 352 user objectives may be used to limit the extent of information resources from repositories 150 under consideration during the rest of method 350 .
- block 352 may involve an inquiry to user 142 via user interface 154 —e.g. user 142 could be queried as to what he or she wants to learn or user 142 could be asked to select from a list and/or a taxonomy of learning topics or learning objectives.
- a topic may be dictated for user 142 by some third party (e.g. an instructor or a curriculum setting authority) or by some other rule.
- this block 352 learning objective may be used in optional block 353 to narrow the extent of (or otherwise determine a subset of) information resources from repositories 150 under consideration during the rest of method 350 .
- a subset may be determined based on the mapping techniques of mapping engine 116 described above.
- the block 352 learning objective may be associated with a taxonomy item which may be considered by mapping engine 116 to be relatively closely associated with a number of information resources.
- the block 352 learning objective may be semantically associated with a number of key words and mapping engine 116 may compare (e.g.
- suitable semantic analysis algorithm(s) such as cosine similarity and/or the like
- the block 352 learning objective may be similarly used in optional block 353 to determine a subset of state table 130 which may be used for the remainder of method 350 .
- method 350 may involve working with a subset of state table 130 which comprises the set of state-action pairs having either a state or an action that includes one or more of the block 353 subset of information resources determined to be relevant (e.g. by semantic analysis) to the block 352 learning objective.
- a subset of state table 130 which comprises the set of state-action pairs having either a state or an action that includes one or more of the block 353 subset of information resources determined to be relevant (e.g. by semantic analysis) to the block 352 learning objective.
- it is assumed (without explicit statement) that method 350 is working with a subset of the available information resources from repositories 150 and a subset of state value table 130 as dictated by the block 352 learning objective.
- Method 350 then proceeds to block 354 which involves procuring a target state.
- FIG. 5C schematically depicts a method 370 for selecting a target state which may be suitable for use in block 354 .
- Method 370 commences in block 372 which involves filtering the state table to obtain only records whose actions are feedback-generating actions (e.g. actions which result in feedback comprising a corresponding feedback metric which may result from those actions).
- feedback-generating actions e.g. actions which result in feedback comprising a corresponding feedback metric which may result from those actions.
- one example of a feedback-generating action involves taking an assessment.
- Other examples of feedback-generating actions giving rise to feedback are discussed above. For the sake of brevity, this description will assume (without loss of generality) that the records of interest in method 370 are those whose feedback-generating action comprises an action of taking an assessment.
- Block 372 then involves filtering the state table to obtain only records whose actions are “take assessment”.
- FIG. 5C shows a non-limiting example of a filtered state table 372 A which may be the result of the block 372 filtering procedure. It may be seen that the exemplary illustrated filtered state table 372 A only includes records for which the associated actions correspond to assessments.
- Method 370 then proceeds to block 374 which involves selecting the record with the highest value from among the block 372 filtered state table records (e.g. filtered state table 372 A).
- a record of the state table may have more than one value field.
- this description will assume (without loss of generality) that there is only one value field in action recommendation method 350 ( FIG. 5B ).
- the methods described herein could be extended to relate to one or more additional value fields. As described in more detail below, the use of such additional value fields may be used to personalize the recommendations of action recommendation method 350 . It will be appreciated from the discussion of updating the values of the state table records in method 300 ( FIG.
- the block 374 record having the highest value from among the block 372 filtered records corresponds to a state and an action considered by system 100 to have relatively high value based on assessment feedback from the users of learning system 100 (which may include the particular user that is the subject of action recommendation method 350 ( FIG. 5B ) and other users of system 100 ).
- the record returned by the block 374 procedure is shown as record 374 A. It will be observed that record 374 A corresponds to the record having the highest value in exemplary filtered state table 372 A.
- Block 376 involves procuring all of the block 372 filtered state table records which have values within a threshold range of the block 374 highest value state table record.
- method 376 involves procuring all of the records having values within a threshold range of the value of record 374 A.
- the particular threshold used in block 376 may be a configurable (e.g. user configurable or system configurable) parameter of learning system 100 .
- FIG. 5C shows a set of filtered and thresholded records 376 A corresponding to a value threshold of 0.40 from the highest-value record 374 A.
- the block 376 thresholding process may comprise selecting a threshold number of records.
- the filtered state table records may be ranked according to value and then block 376 may involve taking a threshold number (e.g. 10) of the highest value-ranked records.
- the block 376 thresholding process may comprise selecting a threshold fraction of the block 372 filtered state table records.
- the filtered state table records may be ranked according to value and then block 376 may involve taking a threshold fraction (e.g. 1 ⁇ 4) of the highest value-ranked records.
- the block 376 thresholding process may involve the use of values in one or more optional value fields 283 (see FIG. 4A ) of the state table records.
- the filtered state table records may be ranked according to value and then block 376 may involve taking the highest value-ranked records which have at least a threshold value level (e.g. greater than 0.5) in an optional value field.
- criteria other than values may be used as the basis for the block 376 thresholding.
- such other criteria could include statistical criteria (e.g. standard deviation of values, percentiles of values), number of updates (e.g. block 308 updates) to the values and/or the like.
- Block 378 involves selecting a target state or target record (based on the block 376 filtered and thresholded records) to be returned as the output of method 370 (block 354 — FIG. 5B ).
- block 378 involves selecting a target state from among the block 376 records.
- block 378 involves selecting the block 376 record with the smallest state size (i.e. the smallest number of actions in the corresponding state) to be the selected target state. In the case of the exemplary records shown in FIG.
- the set of records 378 A includes a plurality of records that each have a state size of two (i.e. two actions in the corresponding state).
- block 378 may return the state corresponding to the record with the highest value as the block 354 selected target state. Accordingly, in the exemplary case shown in FIG. 5C , block 378 returns the state ⁇ 1,7 ⁇ as the target state of method 370 (block 354 — FIG. 5B ).
- block 378 involves selecting a median state size record from among the block 376 records.
- block 378 involves randomly selecting a record from among the block 376 filtered and value-thresholded records.
- the block 376 records may be used as a basis for selecting the block 378 record without necessarily selecting the target state from among the block 376 records.
- block 378 may involve selecting a target state based on the most frequently occurring actions in the block 376 records. Such a target state may be created if it is not already present among the block 376 records.
- a target state may be created (or selected) to include actions that occur more than a threshold number of times (e.g. 10 times) among the states of the block 376 records.
- a threshold number e.g. 10
- a threshold fraction e.g. 10
- any suitable technique(s) may be used for selecting (or creating) the target state in block 378 and that such a technique may be based on information from the block 376 records.
- method 370 ( FIG. 5C ) returns a target state which may be used in block 354 of action recommendation method 350 ( FIG. 5B ).
- the block 354 procedure for procuring a target state may be personalized by the optional steps of offering a diagnostic assessment in block 360 and/or procuring information in respect of the user's current state. These optional procedures are described in more detail below.
- method 350 has determined a target state.
- the block 354 target state comprises the state field of a record which has a feedback-generating action (e.g. an assessment action).
- the block 354 target state may have a relatively high value field (e.g. indicating that the actions in this state will help to achieve the user's block 352 learning objective).
- the block 354 target state may also have a relatively small state size (i.e. a state with a relatively small number of action entries).
- the block 354 target state may be selected based on its value field which may be in turn based on feedback information obtained from the users of system 100 (for which feedback information is incorporated into the state table).
- Block 356 involves making initial action recommendations for user 142 .
- Block 356 may be performed by recommendation engine 140 ( FIG. 1 ).
- the block 356 initial recommendations may generally comprise one or more recommended actions which may help user 142 to achiever his or her block 352 learning objectives.
- the block 356 recommended actions may, but need not necessarily, be based on the block 354 target state.
- the block 356 initial recommendation may involve recommending some or all of the actions in the block 354 target state. Such actions may be recommended in any order or in some desired order.
- block 356 involves additionally or alternatively recommending a relatively structured set of actions which may be referred to herein as an action path or, more succinctly, a path.
- An action path recommended as a part of block 356 may involve an ordered series of one or more actions in relation to accessible information resources which is considered by system 100 to be an optimum path for achieving the block 352 learning objective.
- a recommended action path culminates in a feedback-generating action, although this is not strictly necessary.
- block 356 may additionally or alternatively recommend a non-ordered list of one or more actions in relation to accessible information resources which are considered by system 100 to be have value in relation to the block 352 learning objective.
- FIG. 5E schematically depicts a method 450 for providing information about an action path comprising an ordered series of recommended actions which may be suitable for use in block 356 according to a particular embodiment.
- Method 450 commences in block 452 which involves initializing a filter criteria for path records.
- system 100 may know information about a user's current state (e.g. about the user's history of actions)—see the example user action log of FIG. 3A .
- the block 452 path-record filter criteria may be personalized to the particular user by incorporating the user's current state information. Such personalization of the block 452 path-record filter criteria may lead to correspondingly personalized recommended action path output from method 450 .
- the block 452 initial path-record filter may be based on the user's current state.
- Method 350 shows that user state information may optionally be acquired in block 362 (see FIG. 5B ).
- block 454 involves filtering the state table to get a filtered set of records corresponding only to path records.
- Method 450 then proceeds to block 458 which involves an inquiry into whether any of the block 454 set of path records have corresponding actions that are in the block 354 target state (see FIG. 5B ).
- the block 354 target state comprises one or more action entries. If any of these action entries of the block 354 target state correspond to the action field of the block 454 set of path records, then the block 458 inquiry is positive. Otherwise, the block 458 inquiry is negative. Taking first the block 458 YES output, method 450 proceeds to block 464 which involves selecting the record with the highest value from among the records having an action entry in the target state and selecting the action of that record to be the next recommended action in the path.
- the averages computed for each action in block 460 may be weighted by the relative number of times that the value of the corresponding record has been updated (e.g. updated in block 308 ( FIG. 5A )). For each action, this weighted average may be determined according to the formula:
- weighted_average ⁇ _action ⁇ SA ⁇ record_value ⁇ # ⁇ ⁇ updates_for ⁇ _record total_ ⁇ # ⁇ _updates ⁇ _for ⁇ _SA
- SA is the set of all records corresponding to the action (from among the block 353 subset);
- record_value is the value in the value field of a corresponding record
- #updates_for_record is the number of times that the value for a particular record has been updated in block 308 ( FIG. 5A );
- total_#_updates_for_SA is the number of times that the values of all records in the set SA have been updated in block 308 ( FIG. 5A ).
- method 450 proceeds to block 462 which involves selecting the action corresponding to the highest block 460 weighted average to be the next recommended action.
- method 450 eventually ends up in block 466 with a selected action to be the next recommended action in the recommended path.
- Block 466 involves adding the selected action (from block 464 or 462 ) to be the next action in the recommended path.
- Method 450 then proceeds to block 468 which involves temporarily removing the selected action (from block 464 or block 462 ) from the target state for the remainder of method 450 .
- Method 450 then proceeds to block 470 which involves an inquiry into whether the target state has been emptied of all actions (e.g. whether the target state is empty after having its last action removed in block 468 ). If the block 470 inquiry is positive, then method 450 ends. Otherwise method 450 loops back to block 454 via block 472 .
- Block 472 involves updating the path record filter criteria to reflect the fact that the selected action has been added to the recommended action path in block 466 . This may involve adding the block 466 action to the path-record filter criteria. For example, as discussed above, in the first iteration the path record filter criteria may be set to ⁇ null ⁇ in block 452 in circumstances where it is desirable to recommend the best action for a user, given the assumption that system the user has no historical state. However, when an action is selected in block 466 , it is then assumed that the user will perform the selected action and that the next recommended action should take into account that the user will have performed the first recommended action. To effect this, block 472 involves adding the block 466 selected action to the path-record filter criteria.
- method 450 has developed an action path comprising an ordered list of actions that may be used as the initial recommended actions in block 356 of action recommendation method 350 ( FIG. 5B ).
- block 356 may additionally or alternatively involve recommending a non-ordered set of one or more actions in relation to accessible information resources which are considered by system 100 to be have value in relation to the block 352 learning objective.
- FIG. 5F schematically depicts a method 500 for providing information about a set of n recommended actions which may be suitable for use in block 356 according to a particular embodiment.
- Method 500 commences in block 502 which involves initializing a filter criteria for set-records.
- the block 502 process of initializing filter criteria for set records may be similar to the block 452 process of initializing filter criteria for path records.
- system 100 may know information about a user's current state (e.g. about the user's history of actions)—see the example user action log of FIG. 3A .
- the block 502 set-record filter criteria may be personalized to the particular user by incorporating the user's current state information. Such personalization of the block 502 path-record filter criteria may lead to a correspondingly personalized set of recommended actions output from method 500 .
- Method 350 shows that user state information may optionally be acquired in block 362 (see FIG. 5B ).
- method 500 proceeds to block 504 which involves filtering the state table to get a filtered list of records corresponding only to set records—i.e. corresponding to records whose states are the same as those of the block 502 set-record filter criteria.
- Method 500 then proceeds to block 508 which involves an inquiry into whether the block 504 filtered list includes the desired number n of records.
- the desired number n of records which are recommended in accordance with method 500 may be a user-configurable or system parameter which may generally comprise any positive integer number (e.g. one or more). In some embodiments, it is desirable that the number n of recommended actions be a plurality of actions, but this is not necessary. If the block 504 filtered list does include the desired number n of records, then method 500 proceeds to block 510 . In the illustrated embodiment, block 510 involves selecting the n highest valued records to be the n recommended actions output from method 500 .
- block 514 involves determining a weighted average of the values of the records for each action regardless of the states of those records, wherein the attributed weight is based on the relative number of times that the value of the corresponding record has been updated (e.g. updated in block 308 ( FIG. 5A )).
- a weighted averaging process could be similar to that described above for block 460 of method 450 .
- method 500 proceeds to block 516 which involves supplementing the block 512 recommended actions with the block 514 actions having the highest weighted-average values.
- block 516 may involve recommending the 2 actions with the highest block 514 weighted averages.
- method 500 generates a set of n recommended actions which may be used for the initial action recommendations in block 356 of action recommendation method 350 ( FIG. 5B ).
- the set of n recommended actions generated in accordance with method 500 could comprise an addition to or an alternative to any other block 356 initial action recommendations (e.g. a path recommended in accordance with method 450 ( FIG. 5E )).
- block 356 ends with the recommendation of an initial set of one or more actions for user 142 to perform which are considered by learning system 100 to be helpful in permitting user 142 to achieve her block 352 learning objective. While user 142 may be forced to perform the block 356 recommended actions in some embodiments, in other embodiments this is not necessary and user 142 may be permitted to perform any action that they want.
- Method 350 may monitor and log user action in block 302 , 304 . The monitoring and logging in block 302 , 304 of method 350 may be similar to that described above in blocks 302 , 304 of method 300 ( FIG. 5A ).
- the block 306 inquiry of method 350 into whether the user's current action is a feedback-generating action may be substantially similar to the above-described block 306 inquiry of method 300 .
- the block 308 procedure of updating state table values of method 350 may be substantially similar to the above-described block 308 procedure of method 300 .
- method 350 differs from method 300 in that method 350 comprises an optional block 358 .
- Block 358 involves recommending the next best one or more actions for user 142 given that the user has just completed an action that was monitored and logged in blocks 302 , 304 .
- FIG. 5D schematically depicts a method 400 which may be suitable for use in the optional block 358 process of determining a next best one or more actions for user 142 given their updated state after just having completed an action according to a particular embodiment.
- Method 400 commences in block 404 which involves an inquiry into the updated state of the user from the user action log and as to whether the user has completed all of the actions in the block 354 target state.
- Block 406 involves recommending one or more next actions based on the updated user state.
- block 406 may involve recommending all of the actions in a target state that are not already part of the user's updated state.
- block 406 may additionally or alternatively involve a re-application of method 450 ( FIG. 5E ) using the user's updated state for the block 452 initial path-record filter criteria so as to obtain an updated path comprising an ordered series of one or more recommended actions.
- block 406 may additionally or alternatively involve a re-application of method 500 ( FIG. 5F ) using the user's updated state as the block 502 initial set-record filter criteria.
- Block 404 inquiry is positive (i.e. the user has performed all of the actions in the block 354 target state)
- method 400 proceeds to block 408 which involves recommending that the user take an assessment.
- Block 410 then involves an inquiry as to whether the user does in fact take the assessment recommended in block 408 . If the user does not take the assessment recommended in block 408 (block 410 NO output), then method 400 proceeds to block 412 .
- Block 412 involves recommending one or more actions that the user has not already performed.
- the block 412 recommended actions may involve performing a weighted averaging technique and then selecting the action or actions with the highest weighted average values in a manner similar to blocks 460 , 462 (and blocks 514 , 516 ) described above, except that rather than performing the weighted average over the actions in the target state, the block 412 weighted averaging technique may be performed over any actions that the user has not already performed. One or more actions may be selected in block 412 on the basis of those actions with the highest weighted average values.
- Block 412 may conclude with the recommendation of one or more action(s) that the user has not already done (i.e. one or more actions that are not part of the user's state). After the determining the block 412 recommended action(s), method 400 ends in block 414 .
- block 358 returns an updated list of action recommendation(s) prior to looping back to blocks 302 , 304 .
- method 400 advances to block 416 which involves updating the values of the state table based on the feedback metric associated with the assessment.
- Block 416 may be substantially similar to block 308 of method 300 ( FIG. 5A ) and block 308 of method 350 ( FIG. 5B ). If the values of the state table are updated in block 416 , then method 400 ends in block 418 , which also involves ending method 350 ( FIG. 5B ).
- method 350 includes optional blocks 360 and 362 which may be used to influence the target state procured in block 354 and/or the initial recommendations generated in block 356 .
- the use of user state information (optional block 362 ) to influence initial path-recording filter criteria (in block 452 ) and/or initial set-record filter criteria (in block 502 ) is discussed above.
- Optional block 360 involves having user 142 perform a pre-test (or diagnostic assessment) to test the user's initial level of understanding of a particular topic or learning objective. Such a block 360 diagnostic assessment may be used in a variety of manners.
- FIG. 5G schematically illustrates one method 550 for using the diagnostic assessment which may be suitable for use in block 360 according to a particular embodiment.
- Method 550 commences in block 552 which involves evaluating or otherwise obtaining an evaluation of the diagnostic assessment performed by the user—e.g. to determine the correct answers and the incorrect answers provided by the user.
- an evaluation is performed by an external entity (e.g. LMS 150 C, an instructor and/or the like), but the results of the evaluation are provided to learning system 100 (e.g. through resource interaction activity records 156 , through assessment manager 128 and/or the like).
- Method 550 then proceeds to block 554 which involves locating supplementary information resources.
- the block 550 supplementary resources may be similar to the incorrectly answered diagnostic assessment questions.
- mapping engine 116 For example, the incorrectly answered questions could be submitted to mapping engine 116 for semantic analysis and the block 550 supplementary content may be determined by mapping engine 116 to be semantically similar to the incorrectly answered questions.
- suitable metrics e.g. cosine similarity and/or the like for assessing semantic proximity.
- method 550 proceeds to block 556 which involves adding actions corresponding to the supplementary content to the target state to obtain an expanded target state.
- the block 556 expanded target state may then be output as the block 354 target state and may be used for the balance of method 350 .
- block 556 may involve adding supplementary actions to the recommended actions in block 356 and/or block 358 .
- Such supplementary actions may comprise actions which involve interaction with the block 554 supplementary resources.
- Learning system 100 can provide personalized recommendations of actions for a particular user 142 or for groups of particular users who share one or more common characteristics.
- One technique for providing personalized action recommendations involves the use of the user's current state, which is reflective of the history of actions of that user in relation to accessible information resources.
- the user's current state may be used by learning system 100 in some embodiments to personalize the recommendation blocks ( 356 and 358 ) of method 350 ( FIG. 5B ) and more particularly in connection with the illustrated embodiments of recommendation procedures 400 ( FIG. 5D ), 450 ( FIG. 5E) and 500 ( FIG. 5F ).
- Each of these exemplary embodiments of methods for recommending actions may personalize recommended actions by taking into account the user's current state (or action history) when making recommendations.
- the illustrated example action path recommendation method 450 of FIG. 5E (which may be used in block 356 and/or block 358 of method 350 ( FIG. 5B ) and in block 406 of method 400 ( FIG. 5D )) may use the user's current state as the initial path-record filter criteria (in block 452 ). In this manner, the action path recommended by action path recommendation method 450 may be personalized for a particular user based on the user's history of interaction with accessible information resources.
- the illustrated example method 500 of FIG. 5F for recommending a set of n actions (which may be used in block 356 and/or block 358 of method 350 ( FIG. 5B ) and in block 406 of method 400 ( FIG.
- the set of n actions recommended by action recommendation method 500 may be personalized for a particular user based on the user's history of interaction with accessible information resources.
- FIG. 5G shows an example of a method 550 which may be suitable for incorporating the results of a diagnostic assessment into the block 354 target state and/or the block 356 recommendations.
- the diagnostic assessment may also be used in a similar manner to make updated recommendations in block 358 . As discussed above, the questions that a particular user answers incorrectly may be used to determine supplemental resources.
- Such supplemental resources may be added to the target state for the particular user (which may indirectly influence the recommended actions for the particular user) and/or such supplemental resources can be incorporated directly into recommended actions for the particular user. In this manner, the recommended actions are personalized for a particular user based on the diagnostic assessment taken by the user.
- additional value fields 283 may be filtered or otherwise evaluated according to a variety of characteristics, including, by way of non-limiting example, according to: a particular user; one or more particular user characteristics (e.g.
- user profile characteristics such as language, country, sex, age, personal preferences, physical abilities, psychological abilities, intellectual abilities); characteristics of information resources (for example, video-based information resources only, interactive resources only, resources available from particular repository only and/or the like), characteristics of assessments (for example, assessments involving multiple choice questions only, audio assessments only, assessments set by certain curriculum setting authorities only and/or the like), characteristics of actions (for example, by assessment only), other characteristics (for example, policies of some schools, policies of some jurisdictions (e.g. school jurisdictions, states and/or countries) and/or the like.
- information resources for example, video-based information resources only, interactive resources only, resources available from particular repository only and/or the like
- characteristics of assessments for example, assessments involving multiple choice questions only, audio assessments only, assessments set by certain curriculum setting authorities only and/or the like
- characteristics of actions for example, by assessment only
- other characteristics for example, policies of some schools, policies of some jurisdictions (e.g. school jurisdictions, states and/or countries) and/or the like.
- state table 130 maintained by system 100 ascribes value to particular state-action pairs whenever feedback (comprising a feedback metric) is obtained by feedback interface 118 .
- the state table maintained by system 100 may comprise a plurality of value fields, which may include additional value fields 283 .
- additional value fields may be used to maintain values filtered or otherwise evaluated according to any of the evaluation criteria mentioned above or according to any other suitable evaluation criteria.
- a user characteristic of being dyslexic An additional value field may be maintained in the state table for users who have the characteristic of dyslexia.
- Feedback events may then be filtered or otherwise evaluated, such that the values of this additional value field are updated only in response to feedback involving users who share this characteristic (e.g. dyslexic users).
- action recommendations can be personalized for users who share this characteristic (e.g. dyslexic users) by using the additional value field (in the place of, or in combination with, the general value field (or any other suitable value field)) in the action recommendation methods described above. In this manner, the actions recommended by system 100 can be personalized for users who share this characteristic.
- any user characteristic may be used as an evaluation criteria for an additional value field.
- user characteristics which may be used as evaluation criteria for additional value fields, include: grade level of the user; age of the user; sex of the user; preferred learning style or aptitude of the user; preferred information resource types (e.g. video, graphic, textual, interactive, etc.) of the user; physical disabilities of the user (e.g. blindness, deafness, ability to move the hands, etc.); learning disabilities of the user; and/or the like.
- Such evaluation criteria may be configurable parameters of system 100 . In some embodiments, such evaluation criteria may be user-configurable.
- criteria which may be used as evaluation criteria for additional value fields are not limited to user characteristics and may include some of the other criteria mentioned above.
- the same state-action pairs may have different values in Spanish as they do in North America and such different values maybe managed using separate value fields for the same state action pairs.
- Embodiments of the present invention include various operations, which are described herein. Unless otherwise specified, these operations may be performed by hardware components, software, firmware, or a combination thereof.
- Certain embodiments may be implemented as a computer program product that may include instructions stored on a machine-readable medium. These instructions may be used to program a general-purpose or special-purpose processor to perform the described operations.
- a machine-readable medium includes any mechanism for storing information in a form (for example, software, processing application) readable by a machine (for example, a computer).
- the machine-readable medium may include, but is not limited to, magnetic storage medium (for example, floppy diskette); optical storage medium (for example, CD-ROM); magneto-optical storage medium; read-only memory (ROM); random-access memory (RAM); erasable programmable memory (for example, EPROM and EEPROM); flash memory; or another type of medium suitable for storing electronic instructions.
- some embodiments may be practiced in distributed computing environments where the machine-readable medium is stored on and/or executed by more than one computer system.
- the information transferred between computer systems may either be pulled or pushed across the communication medium connecting the computer systems.
- Methods indicated as a single series of process steps may have one or more subsets of the steps executed on a corresponding one or more computer processors that then communicate interim results or the availability of results among the computer processors using interprocess communication, for example.
- Data values may be stored in cache memory residing in proximity to the CPU or by addressing computer main memory through memory management hardware or even by such memory management hardware causing data to be stored on an external storage device to be loaded into computer main memory.
- Computer processing components used in implementation of various embodiments of the invention include one or more general-purpose processing devices such as a microprocessor or central processing unit, a controller, graphical processing unit (GPU), cell computer, or the like.
- digital processing components may include one or more special-purpose processing devices such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like.
- DSP digital signal processor
- ASIC application specific integrated circuit
- FPGA field programmable gate array
- the digital processing device may be a network processor having multiple processors including a core unit and multiple microengines.
- the digital processing device may include any combination of general-purpose processing device(s) and special-purpose processing device(s).
- a component for example, a software module, processor, assembly, device, circuit, etc.
- reference to that component should be interpreted as including as equivalents of that component any component which performs the function of the described component (i.e. that is functionally equivalent), including components which are not structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary embodiments of the invention.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Business, Economics & Management (AREA)
- Library & Information Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
where:
State | Action | Value | # updates for record | ||
{2, 3} | 1 | 0.3 | 10 | ||
{4, 6} | 1 | .7 | 15 | ||
{4, 8, 9} | 1 | 0.5 | 1 | ||
and the records having action values=7 are given by:
State | Action | Value | # updates for record | ||
{1, 3} | 7 | 0.4 | 7 | ||
{6, 7} | 7 | 0.5 | 33 | ||
{4, 2, 9} | 7 | 0.8 | 12 | ||
In this exemplary case, the weighted average for
and the weighted average for
-
- In the learning system embodiments described herein, the learning system makes feedback-driven action recommendations for a particular user which may be based on feedback from the users of the learning system (including the particular user and/or other users of the learning system). In other embodiments, the methods described herein can be adapted for other purposes. In some embodiments, the invention comprises methods and systems for feedback-driven provision of information. One particular type of information provision comprises providing information in respect of recommended actions for interacting with information resources (e.g. educational information resources). In other embodiments, however, other types of information could be provided. Non-limiting examples of types of information that could be provided with suitable modification to the methods and systems described herein include: information about history (logs) of online activity; information about physical activity results (athletes program training, series of results, scores, etc.); information in respect of any other types of resources that can be engaged with through an action; and/or the like.
- One particular type of feedback capable of providing a feedback metric is an assessment (e.g. an educational assessment) which can provide a grade. In other embodiments, however, feedback metrics could be provided via other types of feedback. Non-limiting examples of other types of feedback that could be provided include: the provision of a feedback metric by an instructor (e.g. after a user takes a verbal test with the instructor); a user interacting with a gaming subsystem (not shown) which assigns a feedback metric to the user; a user attempting to perform one or more tasks (e.g. physical tasks, assembling a widget, completing a multi-step operation) for which a feedback metric may be assigned; financial results of action(s) which may provide any of a host of suitable financial feedback metrics; audience reaction to an event (which may provide feedback metrics such as ticket sales or loudness of cheering, for example) and/or the like.
- One particular type of user action that could be monitored is user interaction with information resources (e.g. educational information resources). In other embodiments, however, other types of user actions could be monitored. Non-limiting examples of other types of actions which could be monitored include: a series of one or more physical movements (e.g. in the context of sports or other activities involve physical training); operations of a trade/occupation; activities associated with an assembly line; military intelligence; and/or the like.
Claims (55)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/486,923 US9064017B2 (en) | 2011-06-01 | 2012-06-01 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US14/740,733 US10249206B2 (en) | 2011-06-01 | 2015-06-16 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US16/101,934 US11195428B2 (en) | 2011-06-01 | 2018-08-13 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US17/524,575 US20220180763A1 (en) | 2011-06-01 | 2021-11-11 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201161492246P | 2011-06-01 | 2011-06-01 | |
US13/486,923 US9064017B2 (en) | 2011-06-01 | 2012-06-01 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/740,733 Continuation US10249206B2 (en) | 2011-06-01 | 2015-06-16 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
Publications (2)
Publication Number | Publication Date |
---|---|
US20120310961A1 US20120310961A1 (en) | 2012-12-06 |
US9064017B2 true US9064017B2 (en) | 2015-06-23 |
Family
ID=47262478
Family Applications (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/486,923 Active 2034-01-11 US9064017B2 (en) | 2011-06-01 | 2012-06-01 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US14/740,733 Active US10249206B2 (en) | 2011-06-01 | 2015-06-16 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US16/101,934 Active 2033-02-25 US11195428B2 (en) | 2011-06-01 | 2018-08-13 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US17/524,575 Pending US20220180763A1 (en) | 2011-06-01 | 2021-11-11 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
Family Applications After (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/740,733 Active US10249206B2 (en) | 2011-06-01 | 2015-06-16 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US16/101,934 Active 2033-02-25 US11195428B2 (en) | 2011-06-01 | 2018-08-13 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US17/524,575 Pending US20220180763A1 (en) | 2011-06-01 | 2021-11-11 | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
Country Status (1)
Country | Link |
---|---|
US (4) | US9064017B2 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11269901B2 (en) | 2020-01-16 | 2022-03-08 | International Business Machines Corporation | Cognitive test advisor facility for identifying test repair actions |
US11853901B2 (en) | 2019-07-26 | 2023-12-26 | Samsung Electronics Co., Ltd. | Learning method of AI model and electronic apparatus |
Families Citing this family (32)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9064017B2 (en) * | 2011-06-01 | 2015-06-23 | D2L Corporation | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
US10075384B2 (en) | 2013-03-15 | 2018-09-11 | Advanced Elemental Technologies, Inc. | Purposeful computing |
US9721086B2 (en) | 2013-03-15 | 2017-08-01 | Advanced Elemental Technologies, Inc. | Methods and systems for secure and reliable identity-based computing |
US9454622B2 (en) * | 2013-03-15 | 2016-09-27 | Doron Etzioni | Educational hub |
US9378065B2 (en) * | 2013-03-15 | 2016-06-28 | Advanced Elemental Technologies, Inc. | Purposeful computing |
US20150026184A1 (en) * | 2013-07-18 | 2015-01-22 | Catherine Ossikine | Methods and systems for content management |
US20160092085A1 (en) * | 2014-09-26 | 2016-03-31 | Intel Corporation | Selective access to interactive device features |
US10558987B2 (en) * | 2014-03-12 | 2020-02-11 | Adobe Inc. | System identification framework |
US9794355B2 (en) * | 2014-04-08 | 2017-10-17 | Samsung Electronics Co., Ltd. | Systems and methods for adaptive notification networks |
US20160063881A1 (en) * | 2014-08-26 | 2016-03-03 | Zoomi, Inc. | Systems and methods to assist an instructor of a course |
US10733898B2 (en) | 2015-06-03 | 2020-08-04 | D2L Corporation | Methods and systems for modifying a learning path for a user of an electronic learning system |
US12067901B2 (en) | 2015-06-03 | 2024-08-20 | D2L Corporation | Methods and systems for providing a learning path for an electronic learning system |
US10748436B2 (en) | 2015-06-03 | 2020-08-18 | D2L Corporation | Methods and systems for improving resource content mapping for an electronic learning system |
US11997123B1 (en) * | 2015-07-15 | 2024-05-28 | Management Analytics, Inc. | Scaleable cyber security assessment system and method |
US10515562B2 (en) * | 2015-11-04 | 2019-12-24 | EDUCATION4SIGHT GmbH | Systems and methods for instrumentation of education processes |
US10306009B2 (en) * | 2015-12-22 | 2019-05-28 | Pearson Education, Inc. | Systems and methods for decreasing latency in data packet provision |
US10095772B2 (en) * | 2016-01-22 | 2018-10-09 | International Business Machines Corporation | Calculation of a degree of similarity of users |
US11176473B2 (en) * | 2017-01-06 | 2021-11-16 | International Business Machines Corporation | Partially observed Markov decision process model and its use |
US10713317B2 (en) * | 2017-01-30 | 2020-07-14 | Adobe Inc. | Conversational agent for search |
US20180314925A1 (en) * | 2017-04-27 | 2018-11-01 | Facebook, Inc. | Systems and methods for sending notifications based on predicted effectiveness |
CN107291867B (en) * | 2017-06-13 | 2021-07-20 | 北京百度网讯科技有限公司 | Dialog processing method, device and equipment based on artificial intelligence and computer readable storage medium |
CN107463601B (en) * | 2017-06-13 | 2021-02-12 | 北京百度网讯科技有限公司 | Dialog understanding system construction method, device and equipment based on artificial intelligence and computer readable storage medium |
CN107515909B (en) * | 2017-08-11 | 2020-05-19 | 深圳市云网拜特科技有限公司 | Video recommendation method and system |
US10989427B2 (en) | 2017-12-20 | 2021-04-27 | Trane International Inc. | HVAC system including smart diagnostic capabilites |
CN108337569A (en) * | 2018-04-03 | 2018-07-27 | 优视科技有限公司 | A kind of interactive discussion method, apparatus and terminal device based on video |
US11042640B2 (en) * | 2018-08-27 | 2021-06-22 | Vmware, Inc. | Safe-operation-constrained reinforcement-learning-based application manager |
CN109582875B (en) * | 2018-12-17 | 2021-02-02 | 武汉泰乐奇信息科技有限公司 | Personalized recommendation method and system for online medical education resources |
CN110647693B (en) * | 2019-09-23 | 2021-03-05 | 北京京东智能城市大数据研究院 | Path recommendation method and device |
US20220005368A1 (en) * | 2020-07-01 | 2022-01-06 | International Business Machines Corporation | Upskill management |
US20230061206A1 (en) * | 2021-08-25 | 2023-03-02 | Royal Bank Of Canada | Systems and methods for reinforcement learning with local state and reward data |
WO2023225078A1 (en) | 2022-05-20 | 2023-11-23 | Advanced Elemental Technologies, Inc. | Systems and methods for a connected computing resource and event/activity identification information infrastructure using near existential or existential biometric identification of humans |
CN116402391B (en) * | 2023-04-07 | 2023-11-10 | 长沙民政职业技术学院 | Comprehensive capability evaluation method and system based on big data |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6614781B1 (en) * | 1998-11-20 | 2003-09-02 | Level 3 Communications, Inc. | Voice over data telecommunications network architecture |
US7003560B1 (en) * | 1999-11-03 | 2006-02-21 | Accenture Llp | Data warehouse computing system |
US7085814B1 (en) * | 1999-06-11 | 2006-08-01 | Microsoft Corporation | Data driven remote device control model with general programming interface-to-network messaging adapter |
US20080295114A1 (en) * | 2007-05-07 | 2008-11-27 | Pramod Vasant Argade | Method and apparatus for execution control of computer programs |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6014134A (en) * | 1996-08-23 | 2000-01-11 | U S West, Inc. | Network-based intelligent tutoring system |
US6299452B1 (en) * | 1999-07-09 | 2001-10-09 | Cognitive Concepts, Inc. | Diagnostic system and method for phonological awareness, phonological processing, and reading skill testing |
US6688889B2 (en) * | 2001-03-08 | 2004-02-10 | Boostmyscore.Com | Computerized test preparation system employing individually tailored diagnostics and remediation |
US7739141B2 (en) * | 2003-07-10 | 2010-06-15 | International Business Machines Corporation | Consulting assessment environment |
WO2005017688A2 (en) * | 2003-08-11 | 2005-02-24 | George Dale Grayson | Method and apparatus for teaching |
US20060240394A1 (en) * | 2005-04-20 | 2006-10-26 | Management Simulations, Inc. | Examination simulation system and method |
US10740722B2 (en) | 2005-04-25 | 2020-08-11 | Skyword Inc. | User-driven media system in a computer network |
CN1794233A (en) * | 2005-12-28 | 2006-06-28 | 刘文印 | Network user interactive asking answering method and its system |
US20080113327A1 (en) * | 2006-11-10 | 2008-05-15 | Microsoft Corporation | Interactive system for teaching and learning algorithms through discovery |
GB2446427A (en) * | 2007-02-07 | 2008-08-13 | Sharp Kk | Computer-implemented learning method and apparatus |
US8630577B2 (en) * | 2007-08-07 | 2014-01-14 | Assessment Technology Incorporated | Item banking system for standards-based assessment |
US20090106312A1 (en) * | 2007-10-22 | 2009-04-23 | Franklin Charles Breslau | User function feedback method and system |
WO2010124247A2 (en) * | 2009-04-24 | 2010-10-28 | Advanced Brain Monitoring, Inc. | Adaptive performance trainer |
BR112012017226A8 (en) * | 2010-01-15 | 2018-06-26 | Apollo Group Inc | dynamic learning recommendation methods and computer readable non-transient storage media |
US20170262529A1 (en) * | 2010-10-01 | 2017-09-14 | Google Inc. | Sponsor answers and user-approved, system-suggested links in a social search engine |
US9336380B2 (en) * | 2010-12-15 | 2016-05-10 | Microsoft Technology Licensing Llc | Applying activity actions to frequent activities |
US9064017B2 (en) * | 2011-06-01 | 2015-06-23 | D2L Corporation | Systems and methods for providing information incorporating reinforcement-based learning and feedback |
-
2012
- 2012-06-01 US US13/486,923 patent/US9064017B2/en active Active
-
2015
- 2015-06-16 US US14/740,733 patent/US10249206B2/en active Active
-
2018
- 2018-08-13 US US16/101,934 patent/US11195428B2/en active Active
-
2021
- 2021-11-11 US US17/524,575 patent/US20220180763A1/en active Pending
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6614781B1 (en) * | 1998-11-20 | 2003-09-02 | Level 3 Communications, Inc. | Voice over data telecommunications network architecture |
US7085814B1 (en) * | 1999-06-11 | 2006-08-01 | Microsoft Corporation | Data driven remote device control model with general programming interface-to-network messaging adapter |
US7003560B1 (en) * | 1999-11-03 | 2006-02-21 | Accenture Llp | Data warehouse computing system |
US20080295114A1 (en) * | 2007-05-07 | 2008-11-27 | Pramod Vasant Argade | Method and apparatus for execution control of computer programs |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11853901B2 (en) | 2019-07-26 | 2023-12-26 | Samsung Electronics Co., Ltd. | Learning method of AI model and electronic apparatus |
US11269901B2 (en) | 2020-01-16 | 2022-03-08 | International Business Machines Corporation | Cognitive test advisor facility for identifying test repair actions |
Also Published As
Publication number | Publication date |
---|---|
US11195428B2 (en) | 2021-12-07 |
US20150310754A1 (en) | 2015-10-29 |
US10249206B2 (en) | 2019-04-02 |
US20220180763A1 (en) | 2022-06-09 |
US20180366020A1 (en) | 2018-12-20 |
US20120310961A1 (en) | 2012-12-06 |
US20190027053A9 (en) | 2019-01-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11195428B2 (en) | Systems and methods for providing information incorporating reinforcement-based learning and feedback | |
Schmitt | Current methodological considerations in exploratory and confirmatory factor analysis | |
Code et al. | The Mathematics Attitudes and Perceptions Survey: an instrument to assess expert-like views and dispositions among undergraduate mathematics students | |
US10902321B2 (en) | Neural networking system and methods | |
Dorn et al. | Empirical validation and application of the computing attitudes survey | |
Romero et al. | Data mining in education | |
Reise et al. | Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective | |
Chen | Enhancement of student learning performance using personalized diagnosis and remedial learning system | |
US20080176203A1 (en) | Cognitive / Non-cognitive ability analysis Engine | |
Abd Aziz et al. | The mediating effects of student satisfaction on technostress–performance expectancy relationship in university students | |
Morren et al. | The relationship between clinical instructor characteristics and student perceptions of clinical instructor effectiveness | |
Bhardwaj | In search of self-efficacy: development of a new instrument for first year Computer Science students | |
Kerwin et al. | You know what i know: interviewer knowledge effects in subjective expectation elicitation | |
Barbu et al. | Data mining tool for academic data exploitation: literature review and first architecture proposal | |
Kozierkiewicz-Hetmańska | A method for scenario recommendation in intelligent e-learning systems | |
Gallo | The effect of an interdisciplinary career exploration course on college students' career decision-making and career decision-making self-efficacy | |
Molina‐Cabello et al. | Are learning styles useful? A new software to analyze correlations with grades and a case study in engineering | |
Dahl et al. | Combining elo rating and collaborative filtering to improve learner ability estimation in an e-learning context | |
Magliaro | Comparing information literacy needs of graduate students in selected graduate programs through the technology acceptance model and affordance theory | |
Seghroucheni et al. | Exploitation of the recommendation systems in the calculation of the learning path | |
Menezes Cunha et al. | Cross-country comparability of a social-emotional skills assessment designed for youth in low-resource environments | |
Binh et al. | An integrated approach for an academic advising system in adaptive credit-based learning environment | |
Tang et al. | [Retracted] Classification and Analysis of College Students’ Skills Using Hybrid AI Models | |
Oyedokun et al. | Correlation analysis of the relationship between demographic variables, computer self-efficacy, and information-seeking behavior of Nigerian University students | |
Racine | Predicting STEM Employability |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: HYPERCUBE TECHNOLOGIES CORP., CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BILIC, JUGOSLAV;BILIC, IVANA;CALLISON, JUSTIN;AND OTHERS;SIGNING DATES FROM 20120530 TO 20120601;REEL/FRAME:029086/0340 |
|
AS | Assignment |
Owner name: DESIRE2LEARN INCORPORATED, CANADA Free format text: ARTICLES OF AMALGAMATION;ASSIGNOR:HYPERCUBE TECHNOLOGIES CORP.;REEL/FRAME:035655/0585 Effective date: 20131224 |
|
AS | Assignment |
Owner name: D2L INCORPORATED, CANADA Free format text: CHANGE OF NAME;ASSIGNOR:DESIRE2LEARN INCORPORATED;REEL/FRAME:035686/0720 Effective date: 20140914 Owner name: D2L CORPORATION, CANADA Free format text: CHANGE OF NAME;ASSIGNOR:D2L INCORPORATED;REEL/FRAME:035704/0280 Effective date: 20140926 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FEPP | Fee payment procedure |
Free format text: SURCHARGE FOR LATE PAYMENT, LARGE ENTITY (ORIGINAL EVENT CODE: M1554); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |