WO2021066985A1

WO2021066985A1 - Computer-based systems and methods for instructional and behavioral recommendations

Info

Publication number: WO2021066985A1
Application number: PCT/US2020/048931
Authority: WO
Inventors: Todd Glover; Linda A. REDDY; Alexander Kurz; Stephen N. ELLIOTT
Original assignee: Rutgers, The State University Of New Jersey; Arizona State University
Priority date: 2019-10-04
Filing date: 2020-09-01
Publication date: 2021-04-08

Abstract

Computer-based systems and methods for instructional and behavioral recommendations, such as educator coaching recommendations and intervention plans, are provided. The system processes disparate education-related data, processes the data using a computationally-efficient, matrix-based recommendation algorithm, and identifies needs and resources by using student related data to determine which students require support, organizes students into groups, and generates a customized intervention plan for a coach. The system can automatically identify and confirm data file types prior to uploading the data, and it can accommodate student data stored in a variety of file formats. Additionally, the system can execute one or more machine learning algorithms which process data relating to educator interventions in order to more rapidly generate recommendations for coaches, and to improve future recommendations by the system.

Description

COMPUTER-BASED SYSTEMS AND METHODS FOR INSTRUCTIONAL AND BEHAVIORAL RECOMMENDATIONS

SPECIFICATION

BACKGROUND

STATEMENT OF GOVERNMENT INTERESTS

This invention was developed with grant funding from the U.S. Department of Education grants (i.e., Teacher Incentive Fund Competition S374A120060; Office of Special Education State Personnel Development Grant H323A160012; OSEP Stepping-Up Technology Implementation Competition H327S 170020). Accordingly, the government has certain rights to the invention.

RELATED APPLICATIONS

This application claims priority to United States Provisional Patent Application Serial No. 62/910,754 filed on October 4, 2019, the entire disclosure of which is hereby expressly incorporated by reference.

TECHNICAL FIELD

The present disclosure relates generally to the field of computer-based recommendation systems. More specifically, the present disclosure relates to computer- based systems and methods for instructional and behavioral recommendations.

RELATED ART

Educators, such as teachers, are people who help students acquire knowledge and improve academic performance. The term “educators,” as used herein, refers to and includes a wide variety of educational professionals, including, but not limited to, classroom teachers, interventionists, and other support personnel (e.g., school psychologists, paraprofessionals) who are generally assigned to lead or support instruction and manage student behavior, providing student intervention support in multiple classrooms, each averaging between 20-30 students. “Coaches,” as used herein, refers to educational support professionals who focus on supervising, advising, and improving the effectiveness of educators to promote student academic performance and positive behavior. Given the number of students that they support, each with unique learning and behavioral strengths and difficulties, it is difficult for educators and coaches to keep track of every student’s academic performance and behaviors to find appropriate customized intervention approaches, and to tailor intervention plans to support each student’s unique needs. This difficulty is compounded by the prevalence of students with behavioral concerns and/or learning difficulties.

There are existing software-based learning support systems that can provide support for teachers. However, most teacher-support systems are based on group needs or general theories about classroom practices and fail to take into account data relating to each student. Further, there are few systems designed to support coaches in coaching educators using appropriate instructional and behavior practices. As such, these prior art systems fail to provide educators and coaches with the tools and models to improve practices or choose new customized instructional and behavioral practices/interventions to address individual student’s learning and behavioral difficulties and/or disorders. Moreover, existing computer-based systems do not adequately process student data that are often stored in numerous, incompatible data storage formats, nor do such systems rapidly generate customized recommendations for students using reduced processing power and with greater computational speed. Indeed, since existing computer-based systems are incompatible, educational professionals often need to utilize multiple software applications (such as Microsoft Excel, Access, proprietary educational support software, etc.) and engage in time-consuming data translation, data manipulation, and data formatting tasks in order to engage in their daily work. Such drawbacks represent significant technological shortcomings of existing software systems. Moreover, existing systems do not adequately leverage machine learning technology, which would greatly increase the speed and accuracy of machine-based recommendation systems for educational professionals.

Therefore, there is a need for computer-based systems and methods which can identify, select, and implement interventions that support educators and those who coach them in improving student learning and behavior, with greater computational efficiency, speed, and machine learning features not presently available in existing systems. These and other needs are addressed by the systems and methods of the present disclosure. SUMMARY

The present disclosure relates to computer-based systems and methods for instructional and behavioral recommendations, such as educator coaching recommendations and intervention plans. The system processes disparate education- related data from disparate (and, often, incompatible) data sources, processes the data using a computationally-efficient, matrix-based recommendation algorithm, and identifies needs and resources by using student related data to determine which students require support and organizes those students into groups (also referred to herein as cases). The student related data include educator interviews relating to students, academic data (e.g., student screening data or achievement test data, office disciplinary referrals, etc.), direct observation data, behavioral data (e.g., behavioral surveys/rating scale data), and other types of data. The system can automatically identify and confirm data file types prior to uploading the data, and it can accommodate student data stored in a variety of file formats. The system can also receive ancillary data, such as benchmark expectations to which to compare the student related data, antecedent and consequence contextual data for establishing target behaviors, assessment data, etc. Additionally, the system can execute one or more machine learning algorithms which process data relating to educator interventions in order to more rapidly generate recommendations for coaches.

The students are grouped/placed into one or more data-based groups automatically by the system, or manually by the user or a third party (e.g., an educator). Groups can be comprised of single students or multiple students. The system then automatically sets a goal for each of the student(s) using a matrix-based recommendation system that executes a matrix-based recommendation algorithm to generate a recommendation for a group based on the initial student data and/or the ancillary data. The system then designs an intervention plan for achieving the goal. The intervention can be automatically recommended from a toolkit or custom-generated. The system then guides coaches in modeling how the intervention works, and it facilitates an educator’s practice of the intervention. By the term “modeling,” it is meant demonstrating to an individual (e.g., a coach models for an educator) how to effectively implement an intervention. The system further provides coaches a structure to provide ongoing performance feedback to educators and methods to effectively evaluate intervention implementation and goal attainment. BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing features of the invention will be apparent from the following Detailed Description of the Invention, taken in connection with the accompanying drawings, in which:

FIG. 1 is a diagram illustrating overall system of the present disclosure;

FIG. 2 is a flowchart illustrating the overall process steps carried out by the system of the present disclosure;

FIG. 3 is a diagram illustrating the matrix-based recommendation system of the present disclosure;

FIGS. 4A-4D are illustrations showing tables used by a grouping algorithm of the present disclosure.

FIG. 5 is a screenshot showing an example dashboard user interface generated by the present system; and

FIGS. 6-49 are screenshots showing example processes of the system of the present disclosure.

DETAILED DESCRIPTION

The present disclosure relates to computer-based systems and methods for instructional and behavioral recommendations, as described in detail below in connection with FIGS. 1-49. Specifically, the present disclosure relates to improved computer-based tools and mechanisms for providing a user, such as an educator, a coach, a school psychologist, an administrative educator, etc., with a system to support educators and other school personnel (e.g., paraprofessionals, interventionists) in identifying, selecting, and implementing interventions to improve student learning and behavior. The system of the present disclosure includes a matrix-driven recommendation system that executes a matrix- based recommendation algorithm for generating coaching recommendations, which can significantly increase the speed with which the system can generate the recommendations, thereby saving computational resources and processor time. The present disclosure further includes a system capable of automatically identifying and confirming data file types prior to uploading the files into the present system, which speeds up the upload process and streamlines the speed and accuracy with which files can be uploaded by the user into present system. This eliminates the need for the user to manually identify a file type prior to upload, thus also saving computational resources and time. The present system further includes an automatic student heat map graphic user interface (“GUI”) generation component based upon the uploaded data files, which includes user-definable tolerances. This component significantly improves the way in which student data are conveyed to the user. Further, the present system includes automatic recommendations for actions based on information about students’ progress and intervention implementation and it can “fine tune” future instructional and behavioral recommendations by applying one or more machine learning algorithms to leam whether sufficient progress is being made to achieve a particular goal when using a specific intervention.

FIG. 1 is a diagram illustrating the system of the present disclosure, indicated generally at 10. The system includes a user device 12, a network 14, a web server 16, and one or more remote servers 20a-20n. The user device 12 can be any electronic device such as a mobile phone, a tablet computer, a smartphone, a phablet, an embedded device, a personal computer, a desktop computer, a wearable device, a field-programmable gate array (“FPGA”), an application-specific integrated circuit (“ASIC”), etc. The user device 12 includes a processor and a memory. The processor can be configured to execute one or more applications on the user device 12. For example, the applications can include a web browser, a coaching application, etc. The memory can be a hardware component configured to store data related to operations performed by the user device 12. For example, the memory can store data received from a user of the user device 12. The memory can include any suitable, computer-readable storage medium such as a disk, non volatile memory (e.g., read-only memory (“ROM”), erasable programmable ROM (“EPROM”), electrically-erasable programmable ROM (“EEPROM”), flash memory, etc.), volatile memory, (e.g., random access memory (“RAM”), dynamic random-access memory (“DRAM”), etc.) or other types of storage media.

The network 14 can be any type of wired or wireless network, including but not limited to, a legacy radio access network (“RAN”), a Long Term Evolution radio access network (“LTE-RAN”), a wireless local area network (“WLAN”), such as a WiFi network, an Ethernet connection, or any other type network used to support communication. The user device 12, the web server 16, and the remote servers 20a-20n can communicate (e.g., transmit/receive data) to each other via the network 14.

The web server 16 can be any kind of server connected to the internet and capable of providing access to web resources via Uniform Resource Locators (“URLs”). The web server 16 can be accessed by a web browser. The web server 16 includes a coaching engine 18. The coaching engine 18 provides the user with functions associated with the present disclosure. Alternatively, the coaching engine 18 can be stored on the user device 12, or on the remote server 20a-20n. The remote servers 20a-20n can be any type of server used for data storage, such as, for example, a hard drive, a cloud storage repository (e.g., Dropbox, Google Drive, etc.), etc. In an example, the remote server 20a-20n can store student data, such as exam data, standardized testing data, behavioral data, personal data, academic data, etc.

FIG. 2 is a flowchart illustrating the overall process steps being carried out by the system 10, indicated generally at method 30. In step 32, the system 10 identifies needs and resources by using student related data to determine which students require support and organizes those students into groups (cases). The student related data include educator interviews relating to students, student academic data (e.g., screening data or achievement test data, office disciplinary referrals, etc.), observation data, direct behavioral data (e.g., behavior surveys/rating scale data), and other types of data. The student related data can be entered manually by the user (e.g., selecting an assessment, entering student scores, etc.), uploaded by the user (e.g., the user can upload a file, such as student data, into the coaching engine 18), or entered/uploaded automatically by the system 10 (e.g., the coaching engine 18 can import data from the user device 12 or one or more of the remote servers 20a-20n, from a third party application, via an application programming interface (“API”), etc.). Additionally, the system 10 can automatically identify and confirm data file types prior to uploading the data. For example, the system 10 can identify a student’s state achievement test data stored on server 20a, link the achievement test data to existing students or automatically create new student cases, confirm the data are in a proper format, and automatically upload the data. If the achievement test data are in an incompatible or un-preferred format, the system 10 can convert the data into a proper format.

The system 10 can further receive ancillary data, such as benchmark expectations to which to compare the student related data, antecedent (observations before a student’s behavior occurred) and consequence (actions taken in response to a student’s behavior) contextual data for establishing target behaviors, supplementary diagnostic data, etc. The assessment data can include, for example, a functional behavior assessment (“FBA”), which can assess educators’ and students’ behaviors, including antecedents and consequences associated with student behaviors.

The students can be grouped/placed into one or more groups automatically by the system 10, or manually by the user or a third party (e.g., an educator). For example, a first student and a second student can be placed into a first group based on their similar learning needs in the area of “ “letter sounds and digraphs,” a third student and a fourth student can be placed into a group case based on their similar learning needs in the area of “phonemic awareness,” etc. Of course, it is noted that the features disclosed herein (e.g., the intervention recommendations generated by the systems/methods of the present disclosure) could be directed to individual students without requiring the grouping of multiple students.

In step 34, the system 10 automatically sets a goal for each of the student(s) using the student related data and/or ancillary data. Specifically, the system 10 can set goals using a matrix-based recommendation system to generate an instructional recommendation for a group (comprised of one or more students) based on the initial data and/or the ancillary data. The matrix-based recommendation system will be discussed in greater detail below. The system 10 can also automatically set one or more goals for each group of students. The system 10 can further calibrate the goal using observations, color-coded indicators of need generating by comparing scores to benchmarks, etc., and it can recommend and enable users to select methods for assessing and determining sufficient progress towards student goals. This includes establishing target criteria for meeting goals. It should be understood that the system 10 can set more than one goal for each student or group of students, though only one goal is set at a time for each student. The system helps facilitate the development of common goals for students grouped based on common needs.

In step 36, the system 10 structures and supports coaches and educators in designing intervention plans for achieving a goal. The intervention can be preset or custom generated, and can be customized by the user. The system 10 can assign the students to groups (comprised of one or more students) based on the student’s needs and automatically recommend interventions based on the students’ needs. In addition, it can support coaches and educators in developing new interventions and in using existing fidelity checklists or establishing their own fidelity checklist with specific steps for intervention implementation.

In step 38, the system 10 structures and supports coaches in effectively modeling intervention plans for educators. Specifically, the system 10 guides an educator’s coach in walking the educator through how the intervention works, and practicing the intervention. The system 10 can further import notes related to the intervention preparation meetings, import notes during feedback meetings, provide tutorials to the educators relating to the importance of practicing the intervention, how to perform the intervention, etc.

In step 40, the system 10 guides coaches in providing ongoing performance feedback to educators. Specifically, the system 10 keeps track of goal progress and measures the user’s or educator’s success implementing the intervention. The feedback can include areas of strength, areas where improvement is required, and areas where additional modeling and practice are required.

In steps 42-44, the system 10 evaluates intervention implementation and goal attainment. Specifically, the system 10 checks on the progress of students and determines whether they are improving at a rate where they will achieve their goals, whether their educator(s) are implementing interventions with sufficient fidelity, and if their goals or intervention plans need to be adjusted. For example, the system 10 provides the user with a checklist of the intervention implementation and a recording of a student’s progress over time. In step 42, the system determines whether sufficient progress has been made in connection with an intervention. If so, the system returns to step 40 and continues to provide performance feedback. If not, step 44 occurs, wherein the system determines whether the intervention has been implemented with fidelity by the educator(s). If not, step 38 occurs, wherein the user is directed through a series of steps to improve the educator’s implementation of the intervention. If the intervention has been implemented with fidelity, step 36 occurs, wherein a redesign of the intervention plan is initiated. Importantly, feedback is continually provided to the system to allow the system to “fine tune” and learn from any changes made to intervention plans. The system 10 allows the user to modify the intervention, select a new intervention, change a goal attainment date, or end the intervention.

Importantly, the system can apply one or more machine learning algorithms to leam whether sufficient progress is being made to achieve a particular goal when using a specific intervention, and can use this information to “fine tune” future instructional and behavioral recommendations made by the system, including individual interventions and sequencing of interventions. Use of machine learning algorithms enables the system to more efficiently refine intervention plans for groups or individuals on an ongoing basis. Specifically, the system applies supervised learning algorithms that can identify and map out relationships among formative student assessment data, recommended skill development progressions, receipt of specific interventions, attainment of criterion-based benchmarks, student groupings, intervention implementation data, and teacher and student outcomes. Over time, the system can use these data to significantly improve the speed and accuracy of intervention recommendations. The system “learns” what interventions, which are generally viewed as evidence-based and effective, will not produce the desired learning outcome when maintained at an implementation level below a certain threshold. For example, the system efficiently (from a computational speed perspective) and effectively stores data, organizes data and pulls student data to identify accurate interventions for small groups or individual student needs. The system uses the characteristics of finer temporal resolution (timeframe within which analysis is completed, which could be reduced as desired, thereby speeding up processing by the system) and process-level data (including monitored student trajectories) to issue faster and more accurate predictions.

It is noted that the machine learning algorithms capable of being utilized by the systems/methods of the present disclosure include, but are not limited to, neural networks, decision trees, supervised learning algorithms, regression, and other algorithms. Such algorithms, when properly trained on student/intervention data, can reduce the amount of data required to be processed by the system by predicting an intervention outcome when limited data are available to the system. For example, using conventional methods, it may take 6 weeks of data gathering and analysis before recommending an intervention for an educator; by using machine learning, the system can make recommendations in a shorter timeframe (e.g., 1 week) since the machine learning algorithms can accurately predict relevant data, even in the presence of limited data (due to learning of patterns by the machine learning algorithms over time, and adjustment of weights by the system). Still further, the machine learning algorithms implemented by the systems/methods of the present disclosure can be used not only to recommend future interventions, but to also modify and adapt (e.g., dynamically) existing interventions as they are being carried out by educational professionals and monitored by the system.

It is also contemplated that the machine learning algorithms utilized by the systems/methods of the present disclosure could also include natural language processing (NLP) algorithms. For example, the system could include a “chatbot” user interface component (driven by NLP algorithms) which allow an educator to describe, in free form text, a particular educational scenario involving one or more students, whereupon the NLP algorithms can parse the free form text to ascertain relevant concepts helpful in identifying an appropriate intervention, and can generate the recommended intervention using the detected concepts.

FIG. 3 is a diagram illustrating functionality of the matrix-based recommendation system, indicated generally at 50, discussed in step 44 of FIG. 2. The matrix-based recommendation system 50 includes a matrix-based recommendation engine 58 that executes a matrix-based recommendation algorithm which receives as input one or more parameters (a first parameter 52, a second parameter 54, an n^th parameter 56) relating to student performance. The matrix-based recommendation engine 58 processes the parameters 52, 54, 56 using the matrix-based recommendation algorithm and generates a recommendation 60 for the student. The parameters 52, 54, 56 can include the student- related data compared to a benchmark. For instance, using an example of assessed students’ reading skills, the parameters 52, 54, 56 can include parameters such as a letter naming fluency, a phonemic awareness, phonics, oral reading fluency words read correctly, oral reading fluency accuracy, nonsense word fluency-correct letter sounds, etc., or other parameters. In addition, diagnostic student data on specific skill needs at various points in an instructional sequence such as consonant blends, vowel teams, inflectional endings, etc., can be included as parameters.

FIGS. 4A-4D are illustrations showing matrices utilized by the matrix-based recommendation engine 58 of the system 10. The matrices function as algorithms that allow for rapid grouping of students into one bucket or a plurality of buckets depending on the student’s grade and skill level in a category, as well as the rapid generation of recommendations for the groups of students. More specifically, the matrices algorithmically control the grouping of students and generation of recommendations based upon the satisfaction of pre-defmed criteria by the input student data. By way of example, buckets in the area of early reading could include phonemic awareness, vowel teams, or multi-syllabic words. However, the buckets can pertain to any academic, developmental, or behavioral skill or feature of a student (e.g., fluency, comprehension, vocabulary, behavior, mathematics, etc.). Each bucket can include a primary level of the skill/feature, a sublevel of the skill/feature, and a tier number relating to how frequent or intense an intervention is necessary. Accordingly, each bucket can be assigned an intervention plan.

FIG. 4A shows predetermined buckets correlating to a time of the school year 112, and a grade level 113, 114, 115, 116. The time of the school year 112 includes a beginning of the year (BOY), a middle of the year (MOY), and an end of the year (EOY). The grade levels include kindergarten 113, first grade 114, second grade 115, and third grade 116. Tier assignments representing how intense an intervention is necessary are indicated in parenthesis in FIG. 4A, and correspond to the intervention foci shown in FIG. 4B, discussed below. Importantly, the matrix-driven recommendations discussed in connection with FIGS. 4A-4B allow the system to very rapidly group and categorize students, thereby significantly increasing the speed with which recommendations are made by the system as well as reducing computational complexity.

FIG. 4B is a graph showing columns that include four primary levels (level A 118, level B 119, level C 120, and level D 121) which correspond to a phonemic awareness or phonics skill type. Each primary level 118, 119, 120, 121 includes a sublevel (e.g., Al, A2, A3, etc.) that corresponds to the specific elements of the primary level (e.g., elements of the phonemic awareness or phonics skill type). Returning to FIG. 4A, each bucket is assigned a primary level 118, 119, 120, 121, a sublevel, and a tier number. The lowest sublevel bucket in which a student has at least one skill need is used to group the student. For example, during the beginning of the year, if a first grade student is assessed to have problems with segmenting/blending, the first grade student would fall into the bucket “A2(3),” where “A2” indicates the type of intervention required, and “(3)” indicates the type of intensity required.

FIG. 4C relates to a bucket associated with a fluency skill type. As such, students who are assessed with problems with rate or expression when reading are grouped into the bucket associated with fluency skill. It is noted that the fluency skill does not have sublevels, and can be a single tier (e.g., tier 2). FIG. 4D is a table relating to buckets associated with a comprehension skill type, including an understanding main ideas bucket 123, a monitoring and adjusting bucket 124, a making connections bucket 125, and a higher-order skills bucket 126. If a student is assessed to have problems with any bullet points associated with comprehension buckets 123, 124, 125, 126, the system 10 will place that student into the corresponding bucket.

FIG. 5 is a screenshot showing an example dashboard 130 generated for a user 132 by the system 10. The dashboard 130 includes a coaching tools menu 134, a time and scheduling menu 136, an explore and leam menu 138, and a management menu 140. The coaching tools menu 134 includes the user’s assignments, the user’s data and interview, rosters, and Functional Behavioral Analyses (FBAs). The time and scheduling menu 136 include the user’s calendar and reminders. The explore and leam menu 138 includes a coaching platform (sometimes referred to herein as “ReadyCoach”) and an intervention library. The management menu 140 includes the user’s profile, a notification button, and a setting button. As shown, the user 132 has three assignments (e.g., educators). Teacher Mary Smith 142 has three cases (e.g., first case 146, second case 148, etc.) and teacher Devin Wright 144 has one case. Each case has a status button (e.g., button 150 for first case 146, button 152 for second case 148, etc.) which shows the user the current status of each case. For example, for case 146, the user 132 is at step 40 of method 30 (provide performance feedback) and needs to provide support and monitor intervention provision for teacher Mary Smith 142. For each assignment (e.g., educator) the user 132 can add additional cases using an add case button (e.g., add case button 154). FIGS. 6-49 are screenshots showing multiple interfaces generated by the present system. FIG. 6 shows an interface for gathering data related to step 32 of FIG. 2. Specifically, the user can enter data manually using button 172, can upload a file using button 174, or can conduct an interview using button 176.

FIG. 7 shows another interface for gathering data related to step 32 of FIG. 2. The user can use buttons 182, 184, 186 for entering or uploading data including gathering data through an interview process. Table 188 shows a list of students 190, the educators or support staff members assigned to each student 192, a first data parameter relating to each student 194, and a second data parameter relating to each student 196. In the example, the first data parameter includes BOSS observation scores such as Active Engaged Time (AET) and Passive Engaged Time (PET), and the second data parameter includes behavior scores. The user can select button 198 to view educator rosters, or select button 200 when finished adding data.

FIG. 8 shows a “your data and interviews” screen from the coaching tools menu 134 options in connection with FIG. 5. Here, the user selects students for group 224 from table 226. As seen, student Porter Hutchings is assigned to group 224, and students Cordell Delaney and Lara Depue are selected to be added to group 224. Table 226 shows student related data in each student’s row, such as behavior scores and rating scale scores. The student related data are shown as a heat map, where each score is colored depending on the quality of the score. For example, a score well-below benchmark is colored red, a below-benchmark score is colored yellow, and an above-benchmark score is colored green. The heat map allows the user to quickly and accurately assess the presented student related data, and greatly reduces the number of user interface screens that would conventionally be needed to access and display student related data. Heat map tolerances (e.g., a range of score value for each color) can be predetermined or user defined. Importantly, the system includes the ability to automatically generate the heatmap based upon the type of student file uploaded to the system (which can be automatically determined by the system prior to uploading of the file) as well as user-defined parameters/tolerances that control what color corresponds to what range of data.

FIG. 9 shows a created group of four students titled “off-task students,” assigned to teacher Mary Smith. The user can set goals for the group using button 228, or create another group using button 230. Selecting button 228 will prompt the user with a screen to set goals for group “off-task students.” FIG. 10 shows a screen for setting behavioral goals for the created group of four students titled “off-task students.” The user can describe a target behavior for students in the group and indicate how it will be measured (Direct Observation using button 232).

FIG. 11 shows a direct observation screen, where the user is presented with selectable radio buttons 282, 290, 294, and 296 regarding how to observe and record the student’s behavior, including Frequency /Event, Duration, Interval, or Latency. For example, selecting “Frequency/Event” button 290 leads to two measurement options shown in FIG. 12, “Total Count” 302 or “Rate” 304. The user can select one to determine how data on student behaviors will be measured over time.

FIG. 13 shows a set outcome criterion for the group screen, where the user can download an observation or assessment template using button 312, perform an observation to establish a baseline using button 314, or set outcome criterion using button 316. FIG. 14 shows how a user enters the outcome criterion (the value that determines goal attainment) for use when measuring the target behavior for each student in the “off-task students” group. As can be seen, the user can enter a percentage value in field 330. This could also be another type of value (e.g., rate or number). FIG. 15 shows a button for setting a deadline for attaining a goal for this group. FIG. 16 shows a summary of the student goals and evidence used to group students for the group “off-task students.” The user can select “find an intervention” to select an intervention for this group.

FIG. 17 shows an intervention library screen. The user can manually select or the present system can automatically select one or more interventions for a student, or for a group based on their data-identified needs. FIG. 18 shows an example intervention, comprising 8 steps.

FIGS. 19-23 shows screens associated with uploading student data. FIG. 19 shows an introductory screen providing the user with instructions, and allows the user to upload a file using button 350. FIG. 20 shows the upload student data screen, where the user can confirm a file format of a student data file. Alternatively, the present system can automatically confirm the file format. FIG. 21 shows that if the data are not automatically recognized, the user can map fields from the file to be read by the present system. FIG. 22 shows a benchmark setting screen, where the user can set custom benchmark and color criteria for each benchmark. FIG. 23 shows a preview screen for uploading student data; the data are automatically color coded based on the designated benchmarks.

FIG. 24 shows the users’ data, which include a list of students 362, each of their Dynamic Indicator of Basic Early Literacy Skills (“DIBELS”) scores 364, and each of the areas of need for intervention automatically generated based on the scores 366. The user can select button 368 to select specific skills with which a student has difficulties; the skills selected are input as parameters into the matrix-based recommendation system to generate a grouping recommendation based on the buckets in which the skills reside. For example, FIGS. 25-28 show a menu of skill needs that the user can select for individual students. These skills are organized by primary levels (areas) (e.g., phonemic awareness, phonics, fluency, vocabulary, comprehension, etc., each of which have sublevels (skill focus) and specifics. FIG. 29 shows a screen where a plurality of students has identified skill needs corresponding to buckets in the matrix-based recommendation system.

FIG. 30 shows a set of group recommendations automatically generated by the system. The group recommendations include a list of students, a recommended bucket, and a recommended tier. The user can select a recommendation using save to a dashboard button 392, or discard the recommendation using a discard group button 394. Users can also drag and drop students between buckets for further customization. If the user saves a recommendation, the user can then select an intervention from a toolkit automatically generated based on the needs of a student group using the select intervention button 398, as seen in FIG. 31. FIG. 32 shows an example of the intervention toolkit that was automatically generated from the systems’ large library of interventions based on the specific needs of a student group, where the user can select criteria (such as the group size targeted, specific areas of focus, or who created the intervention) and an intervention (such as syllable type, blends and digraphs). FIG. 33 shows an example of a syllable type intervention description. The user can select interventions that are automatically recommended or can add their own interventions into their local library. It is noted that more than one intervention can be added to each group, as shown by reference number 400 in FIG. 34.

FIG. 35 shows an example of an intervention tools dashboard. The dashboard shows three groups 412, 414, 416, the interventions assigned to the groups 412, 414, 416, the students in the groups 412, 414, 416, the tier level of the groups 412, 414, 416, which days of the week the groups 412, 414, 416 meet, and the focus of the groups 412, 414, 416. FIG. 36 shows another example of the dashboard generated by the present system. As shown, teacher Mary Smith’s off-task students group 422 is the “support and monitor intervention” step, which the user can initiate by selecting button 424. Additionally, teacher Mary Smith’s physically aggressive students group 423 is the “prepare for intervention” step, which the user can initiate by selecting button 426.

FIG. 37 illustrates how the system automatically recommends actions based on students’ progress and the fidelity of intervention implementation. In this example, the fidelity of intervention was low, but the students were not progressing adequately. The system prompted the user to “consider whether intervention adjustments are needed” to improve students’ progress. When clicking on the prompt, the user is then directed to modify the intervention as shown in FIG. 43. Had intervention not been implemented with fidelity, the system would have prompted the user through a series of steps to promote better implementation (e.g., to prepare for coaching feedback that includes identifying a educator’s strengths, areas for improvement, modeling needs, and practice needs); this is shown in FIG. 41. Automatic recommendations can be generated by the system in response to predetermined thresholds being exceeded, or as a result of a machine learning algorithm which leams from past intervention plans generated and monitored by the system and automatically generates recommendations for “fine-tuning” future recommendations.

Machine learning based on previous rates of student progress with specific interventions is used to improve the efficiency of the machine’s data-based decision making. Such machine learning therefore allows the system to generate future recommendations more rapidly, saving computing time and increasing the accuracy of future recommendations. Various inputs could be provided to the machine learning algorithm, including, but not limited to, student related data (including progress data), recommendations made for interventions, progress and fidelity of such interventions, alterations or revisions made to interventions, and other data. All of this information (or part of it) can fine tune recommendations made by the system using machine learning. As noted above, machine learning can reduce the amount of data required to be processed by the system by predicting an intervention outcome when limited data are available to the system; modify and adapt (e.g., dynamically) existing interventions as they are being carried out by educational professionals and monitored by the system; and parse free form text to ascertain relevant concepts helpful in identifying an appropriate intervention and to generate the recommended intervention using the detected concepts. Further, as also noted above, the system can apply one or more machine learning algorithms to learn whether sufficient progress is being made to achieve a particular goal when using a specific intervention, and can use this information to “fine tune” future instructional and behavioral recommendations made by the system, including individual interventions and sequencing of interventions (e.g., for groups or individuals on an ongoing basis). Supervised learning algorithms can identify and map out relationships among formative student assessment data, recommended skill development progressions, receipt of specific interventions, attainment of criterion-based benchmarks, student groupings, intervention implementation data, and teacher and student outcomes. The system can use these data to significantly improve the speed and accuracy of intervention recommendations. The system “learns” what interventions, which are generally viewed as evidence-based and effective, will not produce the desired learning outcome when maintained at an implementation level below a certain threshold.

FIG. 38 shows a meeting setup screen initiated by button 426 from FIG. 36. Here, the user can enter details relating to a meeting with an educator, such as modeling and review notes, and educator practice notes. FIG. 39 shows a support/monitor intervention screen initiated by button 424 in FIG. 36. Here, the user can rate the intervention implementation using button 432, provide implementation feedback/input using button 434, or log the students’ progress towards goals using button 436. FIG. 40 is initiated by button 432 in FIG. 39. FIG. 40 shows where intervention fidelity is logged and graphed in real time in relation to students’ progress in attaining a goal.

FIG. 41 is initiated by button 434 in FIG. 39. FIG 41 shows a preparing for feedback screen, where the user can provide feedback to the educator (e.g., Mary Smith). FIG. 42 shows a goal progress logging screen initiated by button 436 in FIG. 39, where the user can enter data for each of the students, as indicated by reference number 442, and can select a file to upload using button 444. FIG. 43 shows a modify intervention screen where the student is not improving at a rate required to attain the goal. Here, in FIG. 43 the user can modify the intervention using button 452, use a different intervention using button 454, or end the case using button 456. FIG. 44 shows a continue or fade intervention screen where the student is improving at a rate required to attain the goal. Here, the user can continue the intervention using button 462, or change the date for attaining the goal using button 464.

FIGS. 45-49 show screenshots for using a set a reminder button 464 in FIG. 45. The user can select which educator to set the reminder for, and then select what to remind them of, such as prioritize needs/set goals, assign intervention (and for which group), prepare for intervention, rate implementation, rate student progress, provide feedback, relationship check-in, etc.).

Having thus described the system and method in detail, it is to be understood that the foregoing description is not intended to limit the spirit or scope thereof. It will be understood that the embodiments of the present disclosure described herein are merely exemplary and that a person skilled in the art can make any variations and modification without departing from the spirit and scope of the disclosure. All such variations and modifications, including those discussed above, are intended to be included within the scope of the disclosure. What is desired to be protected by Letters Patent is set forth in the following claims.

Claims

1. A computer-based method for generating instructional and behavioral recommendations for a coach, comprising the steps of: receiving student data at a computer system; processing the student data to identify needs for the student; processing the student data using a matrix-based recommendation algorithm to (i) identify an intervention plan tailored to the needs of the student; and (ii) generate one or more recommendations relating to modeling by the coach of the intervention plan with an educator; electronically evaluating implementation of the intervention plan; providing performance feedback to the educator based on evaluation of the implementation of the intervention plan; processing data relating to implementation of the intervention plan using a machine learning algorithm; and adjusting the intervention plan using information generated by the machine learning algorithm.

2. The method of Claim 1, wherein the student related data comprises at least one of an educator interview relating to the student, academic data, assessment data, and observation data.

3. The method of Claim 1, wherein the step of processing the student data using the recommendation matrix comprises receiving one or more parameters relating to a performance of the student, processing the one or more parameters, and generating the recommendation, wherein the one or more parameters comprises student-related data compared to a benchmark and/or diagnostic student data on specific skill needs at various points in an instructional sequence.

4. The method of Claim 1, further comprising automatically identifying a data file type of a data file related to the student data, automatically confirming a format of the data file, and automatically uploading the data file.

5. The method of Claim 4, further comprising generating and displaying a heat map indicating student-related performance data.

6. The method of Claim 1, further comprising monitoring implementation and progress toward goal attainment of the intervention plan using the machine learning algorithm.

7. The method of Claim 6, further comprising processing data output by the machine learning algorithm to improve speed and accuracy of future intervention plans and recommendations .

8. The system of Claim 1, further comprising adjusting a future intervention plan using information generated by the machine learning algorithm.

9. A system for generating instructional and behavioral recommendations for a coach, comprising: a computer system receiving student data; a coaching engine executed by the computer system, the coaching engine: processing the student data to identify needs for the student; processing the student data using a matrix-based recommendation algorithm to (i) identify an intervention plan tailored to the needs of the student; and (ii) generate one or more recommendations relating to modeling by the coach of the intervention plan with an educator; electronically evaluating implementation of the intervention plan; providing performance feedback to the educator based on evaluation of the implementation of the intervention plan; processing data relating to implementation of the intervention plan using a machine learning algorithm; and adjusting the intervention plan using information generated by the machine learning algorithm.

10. The system of Claim 9, wherein the student related data comprises at least one of an educator interview relating to the student, academic data, assessment data, and observation data.

11. The system of Claim 9, wherein the coaching engine receives one or more parameters relating to a performance of the student, processes the one or more parameters, and generates the recommendation, wherein the one or more parameters comprises student- related data compared to a benchmark and/or diagnostic student data on specific skill needs at various points in an instructional sequence.

12. The system of Claim 9, wherein the coaching engine automatically identifies a data file type of a data file related to the student data, automatically confirms a format of the data file, and automatically uploads the data file.

13. The system of Claim 12, wherein the coaching engine generates and displays a heat map indicating student-related performance data.

14. The system of Claim 9, wherein the coaching engine monitors implementation and progress toward goal attainment of the intervention plan using the machine learning algorithm.

15. The system of Claim 14, wherein the coaching engine processes data output by the machine learning algorithm to improve speed and accuracy of future intervention plans and recommendations .

16. The system of Claim 9, wherein the coaching engine adjusts a future intervention plan using information generated by the machine learning algorithm.