AU2011200233A1

AU2011200233A1 - Method and system for an assessment within a multi-level organization

Info

Publication number: AU2011200233A1
Application number: AU2011200233A
Authority: AU
Inventors: Greg Ritter; David Yaskin
Original assignee: Blackboard Inc
Current assignee: Blackboard Inc
Priority date: 2005-04-12
Filing date: 2011-01-20
Publication date: 2011-02-10
Also published as: NZ582001A; CA2604472A1; NZ581997A; NZ581998A; NZ593779A; NZ593777A; NZ593778A; AU2011200236A1; AU2011200234A1; WO2006110892A2; AU2011200238A1; IL205483A0; IL205480A0; NZ582002A; WO2006110892A3; NZ562435A; AU2006235568B2; IL186532A0; IL205484A0; EP1875420A4

Description

Australian Patents Act 1990- Regulation 3.2 ORIGINAL COMPLETE SPECIFICATION STANDARD PATENT Invention Title "Method and system for an assessment within a multi-level organization" The following statement is a full description of this invention, including the best method of performing it known to me/us: P/00/01 I (- %NRP-nr((fR IC\1417A00 i IWflC -QI 101/ C:\NRPonbr\DCCKXM\34I6841 1 DOC]8/)11211I METHOD AND SYSTEM FOR AN ASSESSMENT WITHIN A MULTI-LEVEL ORGANIZATION Cross Reference to Related Applications 5 The disclosure of the complete specification of Australian Patent Application No. 2006235568, as originally filed, is incorporated herein by reference. Field of Invention 10 The present disclosure relates generally to computer software and hardware systems, and more particularly, to such a system for providing assessment of multiple levels of effectiveness. Background of the Invention 15 As part of providing quality educational opportunities, many academic institutions are interested in assessing and evaluating their performance and efforts as well as the performance of units and individuals within the institution. These institutions can include, for example, on line institutions, K-12 schools, colleges, universities and even multi-campus university 20 systems. One purpose of assessment involves collecting and documenting the necessary evidence needed for satisfying the requirements of regional or national accreditation bodies. Another purpose is to allow the institution to make informed and objective decisions about, for example, curriculum content, course offerings, and educational effectiveness. 25 In the past, assessment efforts have been disjointed and performed in relative isolation at the different levels of the academic institution. For example, the assessment process at the classroom level was performed separate from any assessment (if ever performed) at a department level or higher (e.g., institutional level). This approach lead to redundancy of information and efforts, as well as disparate results that would need to be combined in some 30 manner and evaluated if an overall assessment result is desired. Furthermore, identifying and collecting information needed to meet accreditation requirements may be performed on an ad-hoc basis when needed. These processes are then repeated the next time accreditation is reviewed. Such an approach relies heavily on repeatedly reviewing the 35 same information and generating the necessary documentation each time accreditation is reviewed.

WO 2006/110892 PCT/US2006/014043 Accordingly, there exists a need for an assessment system for institutions that is able to consider and connect multiple organizational tiers of the institution and to provide systematic and systemic data collection, organization, and retrieval components to support institutional assessment efforts. 5 SUMMARY A system of hardware and software provides a systematic and systemic method for performing assessment of the effectiveness of different levels within an institution. The system includes template driven definitions that span multiple tiers of hierarchically arranged operating units within the institution thereby providing flexibility 10 and customization to handle a user-definable model of the institution's organization. The system includes import/export functionality, robust institutional modeling, program initiatives, curriculum mapping techniques, customizable assignment binders, and selective dissemination of documents. It is understood that other embodiments of the present invention will 15 become readily apparent to those skilled in the art from the following detailed description, wherein it is shown and described only various embodiments of the invention by way of illustration. As will be realized, the invention is capable of other and different embodiments and its several details are capable of modification in various other respects, all without departing from the spirit and scope of the present invention. Accordingly, the 20 drawings and detailed description are to be regarded as illustrative in nature and not as restrictive. BRIEF DESCRIPTION OF THE DRAWINGS Various aspects of an institutional assessment system are illustrated by way of example, and not by way of limitation, in the accompanying drawings, wherein: 25 FIG. 1 illustrates a block-level diagram of an institutional environment in which an assessment system is implemented in accordance with the principles of the present invention. FIG. 2A depicts a flowchart of an exemplary method for using an assessment plan for a multi-tier organization using a template driven model. 2 WO 2006/110892 PCT/US2006/014043 FIG. 2B illustrates an exemplary template having hierarchically arranged forms. FIG. 2C depicts an exemplary assessment process.. FIG. 2D depicts an exemplary model for a higher education institution 5 within which assessments can be performed in accordance with the principles of the present invention. FIG. 3A depicts a screen shot of a home page from which a user may elect to perform functions within an assessment system. FIGS. 3B -31 depict a series of screen shots of a user interface that allows a 10 user to perform Goal and Planning Assessment within an assessment system in accordance with the principles of the present invention. FIG. 4A - 4C depict a series of screen shots of a user interface that allows a user to perform curriculum planning in accordance with the principles of the present invention. 15 FIG. 4D illustrates an exemplary curriculum map indicating the alignment between courses and objectives. FIG. 5A depicts a flowchart of an exemplary method for selective deployment of instruments in an assessment management system in accordance with the principles of the present invention. 20 FIGS. 5B - 5J depict a series of screen shots of a user interface that allows a user to manage and create instruments within an assessment system in accordance with the principles of the present invention. FIG. 6A depicts a screen shot of a user interface that relates to displaying user outcomes in accordance with the principles of the present invention. 25 FIG. 6B depicts a flowchart of an exemplary method for generating the contents of the screen shot of FIG. 6A. FIGS. 7A - 7K depict a series of screen shots of a user interface that allows selective reporting of data and content from within an assessment system. 3 WO 2006/110892 PCT/US2006/014043 FIGS. 8A - 8L depict a series of screen shots of a user interface of an assessment system that provides a number of auxiliary tools. FIGS. 9A - 9D depict a series of screen shots of a user interface of an on line learning/academic system with which an assessment system can be integrated in 5 accordance with the principles of the present invention. The figures include a number of different screen shots of an exemplary user interface. One of ordinary skill will appreciate that the depicted interface is provided by way of example and that the layout, options and items on each screen shot may be altered or augmented without departing from the scope of the present invention. 10 DETAILED DESCRIPTION The detailed description set forth below in connection with the appended drawings is intended as a description of various embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced. 15 The detailed description includes specific details for the purpose of providing a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without these specific details. In some instances, well known structures and components are shown in block diagram form in order to avoid obscuring the concepts of the invention. Additionally, the term "automatic" may be used 20 herein to describe one or more process steps that may be performed in an automated manner using various types of programmable processors or computers. However, one of ordinary skill will recognize that the performance of these steps may also be accomplished manually or via a combination of manual and automated processes. 25 Terminology In general, as used herein, "assessment" means a repeating or cyclic improvement process that involves measuring outcomes versus stated objectives or goals and identifying corrective action. The term "evaluation" has more of a connotation of a point in time measurement of one or more outcomes versus stated objectives without 30 necessarily having a built-in improvement step. For example, a program may have the 4 WO 2006/110892 PCT/US2006/014043 goal of developing students' critical thinking abilities and might annually determine whether, as a program, that goal has been achieved. Therefore, one might wish to annually assess this by looking at the aggregate performance of the program's students on a senior capstone project.. 5 Other terms that are used consistently herein include: Mission: A broad statement that describes the over-arching purpose of an organization. Mission statements typically are not measurable because of the scope that they encompass and because they are not time-constrained. Goal: Missions are frequently broken down further into a series of goals. 10 Though more specific than a mission, goals are still broad statements and may not be easily measurable. Goals provide guidance on areas that should be addressed through specific, measurable objectives. Objective: An objective is an expression of the intended result or consequence of some activity (e.g. instruction or some other performance). It differs from 15 an outcome, which is the achieved result, although the two are sometimes used interchangeably. The activities described by an objective should be measurable within a defined timeframe. Frequently, the term is used with a modifier to clarify the activity, e.g. a learning objective or a performance objective. Standard: In educational assessment, for example, a standard is a 20 statement of what a learner should know or be able to do. A standard is more specific than a goal and should be measurable. In that sense, a standard is much closer to an objective than a goal. However, unlike an objective, a standard may not be related to a specific activity, instruction, performance, etc. or timeframe. Also, standards are frequently generated by organizations external to the academic institution (e.g. state governments, 25 accrediting agencies) as a means to standardize measurement of performance or learning across multiple organizations. Outcome: An outcome is the achieved result or consequence of some activity (e.g. instruction or some other performance). It differs from an objective, which is the intended result, although the two are sometimes used interchangeably. Frequently, the 30 term is used with a modifier to clarify the activity, e.g. a leading outcome or a performance outcome. 5 WO 2006/110892 PCT/US2006/014043 A "template" is a means of defining the structure of various data in the system and the instruments used to collect that data. A template may define one or more customizable forms and the attributes that are present on each form. A "form" is a customizable page within a template consisting of one or more attributes. An "attribute" is 5 a piece of data that defines an entity (such as the name of a user) which may be represented on a form as a field. Certain attribute fields may be required by the system. The level of customization (e.g. the ability to change labels on an attribute versus the ability to add new attributes to a form) may be dependent on the type of template that the form is a part of. 10 For example, a template for collecting data about a user might consist of one form. On that form are specific attributes, such as "name" "social security number" and "type." The "name" and "social security number" attributes might allow for entry of text information, while the "type" attribute might offer a selection of 'faculty,' 'staff,' or 'student.' The "name" and "social security number" attributes might be required, meaning 15 that any person completing the form must enter data for those attributes, while the "type" attribute might be optional. One of ordinary skill will recognize that there are numerous different and varying attributes that can be defined for each form using standard available web-based interfaces. For example, the information related to an operating unit will vary for a "College" operating unit and a "Department" operating unit. A College operating 20 unit may have fields to designate Dean, while a Department would have fields to identify the current Chair of the department. Beyond such labeling distinctions, different operating units may even have different types of data. For example, an operating unit representing a Center for Media Students may need to store, as part of the operating unit, information about the grant that funds the Center, the grant's initiation and expiration 25 dates, and who manages the Grant, while a Department, funded out of the university budget, may only need to store university budget codes assigned to the Department. Based on their purpose and level within the organization, different operating units will have different informational needs. Templates may also consist of more than one form; this is particularly 30 useful for the representation of a data collection process that consists of multiple steps. For example, an Assessment Project template might consist of a "properties form" to collect data about the properties of the project itself, one or more "objective forms" to 6 WO 2006/110892 PCT/US2006/014043 collect data about each objective being measured, and four "activity forms" per objective to collect data about the different steps of assessing the performance of the objective. Each form would have specific attributes defined. This exemplar template is illustrated in Fig. 2B. One of ordinary skill will recognize that there could be many different ways to 5 configure a multiple-form template to represent a variety of different processes for different purposes. While many examples are provided herein that specifically include a higher-education institution, the principles of the present invention contemplate other types of institutions as well. For example, corporations, governmental entities, and K-12 10 institutions are all considered within the scope of the present invention. An institution may also be a consortium of schools and/or campuses. In general terms, an institution is an operating unit and is, itself, made up of different operating units that may correspond to campuses, colleges, departments, sub-departments, etc. The systems and methods described herein do not require any particular arrangement of operating units but, instead, 15 allow the institution to model its organization into a hierarchy of operating units for purposes of management, planning, and reporting of assessment efforts. The terms "unit", "operating unit", and "operational unit" are often used interchangeably herein. However, there are unique aspects of operating units related to their respective level in the organizational hierarchy that allow customization and flexibility based on these 20 unique aspects. In the higher education example, operating units offer programs of study, or simply "programs", with different courses and sections. Thus, assessment can occur, for example, at or across an institutional level, a departmental level, a program level, an individual level, a course level and at the section level. Programs may also include 25 courses offered by different operating units. For example, a particular program of study may require both an introductory Engineering course and a computer science course. Within the institutional level assessment, different varieties of assessment may also be performed on different levels of operating units as well. For example, the institution may be organized according to operating units that represent department levels, college levels, 30 etc. Thus, a framework is described herein, in which a hierarchically arranged group of organizational units and activities are defined and assessed against various criteria and objectives. 7 WO 2006/110892 PCT/US2006/014043 FIG. 1 depicts a functional block diagram of an exemplary environment for an assessment system 102 in accordance with the principles of the present invention. As described in more detail herein, the assessment system 102 provides a framework for performing various types of assessments of an institution such as, for example, a higher 5 education institution. A user 104 of the system 102 typically uses a web browser or similar interface to communicate with an appropriately configured front-end 106 of the system 102. For example, the front-end 106 may be a web server hosting a number of applications 108 that the user 104 may access. The applications 108 are one or more software components or programs that execute on a programmable computer platform to 10 provide functionality related to performing institution-related assessment activities. Such applications 108 may include components for modeling the institution or organization, defining assessment projects, identifying users (e.g., students), defining objectives and goals, and collecting and reporting data. The applications 108 may also access data storage facilities 112 and other 15 computer systems 114. For example, the data facilities 112 may be one or more databases having assessment project data stored and arranged in a convenient and appropriate manner for easy manipulation and retrieval. The other computer systems 114 may be a variety of third-party systems that contain data or resources that are useful for the assessment system 102. In the exemplary higher education environment, the systems 20 114 may include a student information system (SIS) that maintains student demographic information as would be appreciated by one of ordinary skill. The systems 114 may also include an electronically maintained class, or course, schedule for the institution that includes information about the courses such as section numbers, professors, class size, department, college, the students enrolled, etc. Other campus-related systems such as 25 financial aid and the bursar's office may be included in the systems 114 of FIG. 1. The back-end 110 is appropriately configured software and hardware that interface between the applications 108 and the various resources 112, 114. Another resource to which the back end 110 may provide connectivity is a campus (or institutional) academic system 116. An example of such a system is provided 30 by the present Assignee under the name Academic SuiteTM with many features thereof described in U.S. Patent No. 6,998,138 entitled "Internet-Based Education Support System and Methods", the disclosure of which is incorporated herein in its entirety. As 8 WO 2006/110892 PCT/US2006/014043 described in that patent, the campus academic system 116, in an academic environment, provides a platform that allows students and teachers to interact in a virtual environment based on the courses for which the student is enrolled. This system may be logically separated into different components such as a learning system, a content system, a 5 community system, and a transaction system. An example of such a student is the user 118 that can access the academic system 116 via a web browser or similar interface. The user 118 may also be faculty, staff or an administrative officer. An example of these separate components of the system 116 are described in detail in pending patent applications "Method and System for Conducting Online Transactions (Serial No. 10 10,373,924 filed 2/25/03), "Content and Portal Systems and Associated Methods" (Serial No. 11/142,965 filed 6/2/05), and "Content System and Associated Methods" (Serial No. 10/918,016 filed 8/13/04), all of which are incorporated herein by reference in their entirety. Of particular usefulness to the assessment system 102, the academic 15 system 116 provides a virtual space that the user 118 may visit to receive information and to provide information. One exemplary arrangement provides the user 118 with a home page where general information may be located and that has links to access course specific pages where course-specific information is located. As explained in the incorporated patent and patent applications, electronic messaging, electronic drop boxes, 20 and executable modules may be provided within the user's virtual space on the academic system 116. Thus, with respect to the assessment system 102, one of the applications 108 may be used to generate information that is to be deployed to one or more users of the academic system 116. Via the back-end 110, the information may be sent to the academic system 116 where it is made available to the user 118 just as any other 25 information is made available. Similarly, from within the academic system 116, the user may enter and submit data that is routed through the back end 110 to one of the applications 108. One of ordinary skill will recognize that the academic system 116 and the assessment system 102 may be more closely integrated so that the connectivity between the applications 108 and the system 116 is achieved without a network 30 connection or special back end software 110. Although the front end 106, applications 108, and back end 110 of the assessment system 102 are each depicted as a single block in FIG. 1, one of ordinary skill 9 WO 2006/110892 PCT/US2006/014043 will appreciate that each may also be implemented using a number of discrete, interconnected components. As for the communication pathways between the various blocks of FIG. 1, a variety of functionally equivalent arrangements may be utilized. For example, some pathways may be via the Internet or other wide-area network, while other 5 pathways may be via a local-area network or even a wireless interface. Also, although only a single user 104 of the assessment system 102 is explicitly shown, multiple users are not only contemplated but are very likely within the environment of FIG. 1. The structure of FIG. 1 is logical in nature and does not necessarily reflect the physical structure of such a system. For example, the assessment system 102 may be distributed 10 across multiple computer platforms as can the data storage 108. Furthermore, the three components 106, 108, 110 are separate in the figure to simplify explanation of their respective operation. However, these functions may be performed by a number of different, individual components, or a more monolithically arranged component. Additionally, any of the three logical components 106, 108, 110 may directly 15 communicate with the academic system 116 without an intermediary. Also, although the users 104, 118 are depicted as separate entities in FIG. 1, they may, in fact, be the same user or a single web browser instance concurrently accessing both the assessment system 102 and the academic system 116. Assessment within an institution such as a higher-education academic 20 institution is a complex undertaking that encompasses many different levels of evaluation, data collection, and correction. For example, at the institutional level, a university may be focused on assessing accreditation requirements and strategic planning initiatives. At the program level, the relevant managers may be focused on assessing program effectiveness and discipline-specific accreditation requirements. At the classroom level, 25 the instructors may be interested in assessing course design and curriculum coverage. Thus, some of the diverse questions that one or more assessment processes may try to address are decisions on faculty promotion and tenure, determining an effective mix of full-time and adjunct faculty, determining an effective mix of traditional and remote courses, and assessing whether the needs of specific student populations (e.g., commuters, 30 transferees, minorities) are being met. 10 WO 2006/110892 PCT/US2006/014043 Assessment Projects and Initiatives In general the phrase "assessment project" is used herein to describe efforts of an institution, an operating unit, a program or a course to assess and evaluate different aspects of its operation and effectiveness. One or more assessment projects may 5 be ongoing at any time for the same operating unit or different operating units. An "assessment initiative" may encompass a number of different assessment projects that are related to a common theme. For example, an assessment initiative for Accreditation Board of Engineering and Technology (ABET) accreditation may be defined by the College of Engineering such that the ABET assessment project of each of the individual 10 departments within the College (e.g. Department of Electrical Engineering, Department of Civil Engineering, Department of Mechanical Engineering, etc.) may connect to the College-level assessment initiative to enable College-level reporting on status and outcomes.. By explicitly connecting, or associating, a number of assessment projects with an assessment initiative, a user of the present system is able to aggregate and 15 organize data from among the different projects so that assessment and analysis can occur from both an individual project perspective as well as from the assessment initiative's cross-project perspective. Exemplary assessment projects might be: a) internal to the program, such as a curriculum planning project or a program review; 20 b) internal to the institution, such as fulfilling institutional strategic planning requirements; or c) for external review or accreditation, such as completing an institutional accreditation report or a discipline-specific accreditation report. Assessment projects may be initiated at different times and will span 25 different time frames. For example, while a project supporting the unit's responsibilities for institutional accreditation may actually take place over several years, strategic planning assessment projects may only last eight months and happen with more regularity than accreditation. Even though the assessment process can be highly complex, there are 30 some general provisions of the assessment system 102 of FIG. I that allow a user 104 to systematically and automatically initiate assessment projects. The general method of 11 WO 2006/110892 PCT/US2006/014043 FIG. 2A is utilized, regardless of whether the user is defining an assessment project for an institution operating unit, some lower-level operating unit, or a program or course. From a starting page, the user makes a selection, in step 202, via a user interface to enter a goal and planning component of the assessment system 102. After 5 activating such a component, the user is prompted to define an assessment project. To simplify this process, the system 102 may provide the user with a default template or form that has some predefined fields and information. Alternatively, the user may be presented with a number of different types of templates and asked to make a selection. For example, the user may be asked whether the assessment project involves a program, a 10 department, a college, an institution, or a course. Based on the user's response, an appropriate template may be provided to the user. For example, at the course level, the user would likely not be interested in defining strategic planning goals. Similarly, at the institutional level, the user would likely not be interested in specifying a course objective. Thus, the templates provided to the user may have their initial content dependent on what 15 type of assessment project is being defined. In one particular embodiment, the assessment system 102 may be initially provided with a number of predefined templates. As part of the initialization process (before assessment projects are defined) a user or other administrator may customize these templates in accordance with the institution and its programs, departments, courses, etc. The customization of templates is not a 20 necessary step before an assessment project is begun but it advantageously provides flexible customization geared towards each respective institution and their hierarchical organization. As a result, in step 204, the user selects a template for the type of assessment project being generated. Then, in step 206, the user collects and enters data 25 required by the forms as defined in the selected template. For example, the collection and entering of data may include metadata about the assessment project, the specific objectives of the assessment project, and activities related to each objective. Once the form is completed, then it can be presented to other users for review and approval in step 208. For example, a department chair may have responsibility for creating an assessment 30 project but there may be a committee that may modify and approve the assessment project before it can be implemented. 12 WO 2006/110892 PCTIUS2006/014043 In the above description, it is assumed that the user has adequate access rights within the assessment system 102 to perform the desired tasks. As known to one of ordinary skill, user identities and user roles may be employed to restrict user activities so that certain users have more access privileges to data resources than other users. These 5 access privileges may be relatively static or may be more dynamic in nature as various users are placed in different roles or committees. Certain users may be allowed no access to an assessment project, read-only access to the project, edit-access to the assessment project, or create-access to the assessment project. Additionally, within an assessment project, the individual components of the project may have controlled access based on 10 user roles as well. Furthermore, an assessment project manager role may be employed, and assigned to a user, to help with administering and managing the project and its implementation. Thus, different resources within the present assessment system may be protected through the application of access privileges that allow each resource to be available to one or more identified users or shared by all users. 15 FIG. 2B depicts an exemplary template structure that may be used to implement the default templates that are presented to the user. In assessment projects, templates are used to define the set of forms that are employed to collect the information that makes up the assessment project. An assessment project template 250 may include one or more of several types of forms: 20 a) a main form 252 is used to collect information about the project itself. Additional fields can be added to a project form in new templates by the user, but certain fields may be required on an assessment project template's project form in order to standardize data collection. While not absolutely necessary, it is advantageous to permit only one project form per assessment project template 250. 25 b) An objective form 254 is used to collect information about the objectives of the assessment project. An assessment project typically has one or more objectives, so at least one objective form is present. Many assessment projects have multiple objectives, however. c) Objective forms can have one or more activity forms 256 - 262 30 associated with them in the template 250. These activity forms 256 - 262 collect information about various activities related to conducting the assessment of the objective. Each activity form may have a different set of fields on it. Addition of an objective (e.g. 13 WO 2006/110892 PCTIUS2006/014043 addition of an objective form 254) to a template 250, also generates the associated activity forms 256 - 262. The template 250 of FIG. 2B may be fully customizable and definable by a user. However, in some instances, an institution may enforce requirements that all 5 assessment projects include certain base information. Such a requirement may be helpful, for example, in ensuring that institution-wide reporting can be performed at least on the base set of assessment project attributes. In one example, the template 250 of FIG. 2B includes: Summary Statement: This provides an area to enter a general description 10 or summary of the whole assessment project. This summary can also be used as part of a report generated for the assessment project. For example, the summary statement could include an overview of the project, names and titles of the university faculty and staff involved, and a summary of the findings. This might appear as the first page of a report. Start/End Dates: Projects to evaluate learning outcomes or reviews of 15 programs at an institution may have specific time periods which they span (e.g. an accreditation project may last several years, but a program review may happen annually.) A user initiating an assessment project can specify start and end dates for the assessment project. The default start date would be the project's date of creation. The default end date would not be set (i.e. no end date) as some assessment projects could be continuous 20 and ongoing. Phase Summaries: For each phase that is defined for the assessment project by the template, the user (or manager) is able to enter summary information for that phase. Recurrence: The assessment process is cyclical, so frequently the same (or 25 similar) assessment project may be run on a regular basis. For example, a departmental program review may occur every year, or the department's discipline-specific accreditation may need to be re-confirmed every three years. It is therefore useful to track if an assessment project is a re-occurrence of some previous assessment project. Workflow: Workflow status can be used to control the state of the 30 template and to determine what action or activity is next in the process flow. 14 WO 2006/110892 PCT/US2006/014043 The flowchart of FIG. 2C provides one example of one way to define a process of initiating the assessment process that is also further described later with reference to FIGS. 3A -31. More particularly, the flowchart depicts a specific sequence of events that do not necessarily have to occur in that order. For example, the step of 5 modeling (step 270) does not necessarily have to occur before an assessment project (step 272). Thus, the steps of the flowchart may occur in an order other than depicted in FIG. 2C. There are events that may advantageously be performed in a particular order for logical reasons. For example, aligning course content (step 280) will usually be performed after defining unit objectives (step 274). Additionally, one or more steps of 10 the flowchart of FIG. 2C may be added or omitted without departing from the scope of the present invention. Before beginning the definition of an assessment project, the institution itself may be modeled. FIG. 2D is a sample of only a part of a model that may apply to an institution such as , for example, a university with multiple campuses. As illustrated, 15 the model captures details about various hierarchically arranged operating units 290-293, as well as programs 296, courses 294, and sections 295. Other equivalent terminology that may also be used herein to describe the organization of the institution may specify "nodes" within different "organizational levels" of the institution. For example, the institution itself may be a node at the organizational level of "institution". Additionally, 20 at the organizational level of "campus", there may be multiple nodes with each node representing a different campus (e.g., Campus1 291 of FIG. 2D). Other organizational levels, as already mentioned, may include many different types of operating units and entities within the institution such as, for example, colleges, departments, programs, colleges, sections, courses, etc. all of which can be considered a node. The model of FIG. 25 2D is provided merely by way of example to show the flexibility and customization provided by the present assessment system. The modeling of the institution may be approached in a variety of ways. For example, an academic institution may be comprised of a number of hierarchically arranged organizational levels. The institution may have a number of different colleges that each have different departments that offer various 30 courses having individual sections. Also, within this organizational structure, there may be programs of study such as , for example, different degree programs (B.S., M.S., PhD) or concentrations of study. One of ordinary skill will recognize that the institution may 15 WO 2006/110892 PCTIUS2006/014043 be organized in a variety of different ways and that inIplementations of the present invention are able to model those various structures as different operating units that are used within an assessment project or initiative. Because of the hierarchical nature of the model, operating units may have lower-level operating units that are sometimes referred 5 to herein as sub-operating units or lower-level operating units. The actual mechanics of modeling the institution may be accomplished in a variety of ways without departing from the scope of the present invention. For example, as part of the initial set-up of the assessment system described herein, an administrator may manually define the hierarchical arrangement of the institution in a manual step-by-step process. 10 Alternatively, the model definition may take place such that a list of the organizational levels is processed as a batch to create the model. Still yet another alternative may involve integration with the campus academic system 116 (or other existing system) through an automated process to define the model. In one particular example, the common attributes for each node within an organizational level may be captured in the 15 form of a editable template. When an instance of a particular node is to be modeled within that organizational level, a user can select the appropriate template and then modify it in accordance with the particular characteristics of that node. Thus, in step 270, initiation of an assessment project may begin by modeling the hierarchical structure of the institution. Once this is accomplished, 20 definition of an assessment project may begin, in step 272. The assessment project within an operating unit (e.g., a college, a department) may be related to a higher-level operating unit for which goals and objectives have already been defined. In an educational institution, for example, the sub-units may be academic departments, administrative or other departments. Underneath these units may be courses or course sections. When 25 creating other assessment projects, the previous modeling of the institution (step 270) does not need to be re-accomplished but may be re-utilized. For the sub-operating units, their own objectives are defined in step 274. These objectives may be aligned with the top-level goals and objectives to create a link between the multiple hierarchical levels of operating units. Within a sub-operating unit, 30 one or more courses are defined in step 276. As part of defining a course, the course objectives are aligned with higher-level goals and objectives, in step 278, so that a link between courses and the meeting of certain goals and objectives can be established. In 16 WO 2006/110892 PCT/US2006/014043 addition to the course having its own objectives, the course can also include a definition of its content (e.g., assignments, tests, reading material, collaboration exercises, etc.). A description of the various content for a course may include which course or unit objective it is aligned with. Thus, in step 280, course content is aligned with course objectives and, 5 possible, higher-level goals and objectives. As a result, top-level managers can establish which specific resources satisfy strategic goals and objectives, while low-level managers can identify deficiencies in course offerings and content when compared to desired curriculum objectives. One advantageous benefit of the present system, described below, is that at each level of the process an appropriate form can be presented to the user to 10 direct the definition of the assessment project. Goals, Objectives, Standards, and Assessment Planning FIGs. 3A - 31 provide a series of screenshots of one exemplary user interface that allows a user or project manager to create an assessment project. One of 15 ordinary skill will appreciate that not all steps depicted in the screen shots are necessary nor are all the features within the user interface. FIG. 3A depicts a screen shot of an opening window 300 for an English department of a university. This window would typically be presented to a user once that user logs into the assessment system and would be based on the user's identity (e.g., an 20 Engineering professor or department head would be presented with a different, appropriate window). The title bar 302 shows the example hierarchy of: University College of Liberal Arts 25 English Department that has been originally defined for this institution or changed through subsequent modification. Such a hierarchy can, for example, be defined by the institution when initially configuring the assessment system or modified during a later update of that 17 WO 2006/110892 PCT/US2006/014043 definition. From this "home page" a user may start a new assessment project or interact within a current assessment project. A number of sub-windows may be provided within the opening window 300. Some exemplary windows, as shown in FIG. 3A, include a mission statement 5 window 306, a list of available assessment projects 312, and a list of courses 308 within the English Department. As shown, the available courses 308 can be filtered according'to the term in which they are offered. Another possible sub-window lists the different programs 304 offered by the English Department. From the Home window 300, the user can opt to manage the assessment process by selecting to open the "Goals & Assessment 10 Planning" tab 310. The tabs located in the left of the window 300 allow a user to select the different functions available within the system. The "Goal Manager" window of FIG. 3B is identified by its title bar 302 and the tab 314. (This exemplary window is for a hypothetical Business & Finance Department unlike the previous window which related to an English Department). From 15 within this interface, a user is presented with a list of analysis processes 318 that are available to support strategic planning of a particular unit or program (e.g., institution, campus, department, Graduate program, etc.) These processes can include, for example, "Goals" or "Mission Statement", "Vision Statement", and other widely used analysis processes that are known within the area of strategic planning. As used herein, a mission 20 statement generally relates to what a institution plans to do while a vision statement relates to the institution's principles or what it believes in. This list of processes may also include an identification of an appropriate term 316 to which each process relates. For example, this allows historical mission/vision statements to be maintained even as new ones are developed. The current 25 user is also provided with a respective icon 317 that is associated with each process that indicates that user's access privileges for the particular process. From the list 318, a user may choose to define or edit a mission/vision statement for the operating unit or program. As seen in FIG. 3C, the user is presented in the area labeled "1 Properties" with a mechanism for specifying whether the statement 30 applies to the entire academic department or some specific context (e.g., program, resource center, etc.) within the department. The text box 323 allows the user to enter the specific context if appropriate. This box 323 may also be populated with predetermined 18 WO 2006/110892 PCT/US2006/014043 contexts from which the user can select as well. The term "department" is used generically and can apply in general to whatever level operating unit the user has selected. In this example, the title bar 302 indicates that the "department" is the "Division of Business & Finance". 5 The text box 322 allows the user to enter the language of the mission statement. In this instance, the vision statement is something different than the mission statement and so a separate text box 320 allows the user to enter the vision statement for the department. Once the mission and vision statements are created, the goals for the department can be defined using an interface such as the one shown in FIG. 3D. 10 This interface screen provides a list of the department goals that includes a label 324 for each goal and a portion of the text 326 of each goal. To add a new goal, an "Add" button 325 is provided. Although not shown, the add process would typically include defining a label for the new goal, a textual description of the goal, and some additional meta-data such as its priority, a category identifier, a keyword (to assist with 15 searching or customized reports), and an identifier of related goal or objectives. These additional meta-data may also be included in the list via a user-controlled selection process 327. In many instances, entering the mission and vision statements for the department, as well as the goals, may already have been performed and the user may bypass such steps and move directly to defining a new assessment project. 20 The screen of FIG. 3E includes a title bar 328 indicating to the user that the screen pertains to assessment projects of the English Department. The list of currently defined assessment projects can include a identification of the project 330, its description 332, the name of the initiative to which it relates 334, as well as other meta data. However, some of these columns may be eliminated as well. By selecting a project 25 name 330, the user can open that assessment project and read or edit the particular data pertaining to that assessment project (assuming they have the proper access privileges). Also, using the "Add" button 336, a user can begin the process of defining a new assessment project. The defining of a new assessment project may take place as described above with the use of templates to assist in the process. 30 FIG. 3F depicts a partially defined assessment project related to "undergraduate Degree Evaluation". In this window, the user is presented with the already defined objectives for the project. For example, a list of objective labels 344 and 19 WO 2006/110892 PCT/US2006/014043 associated descriptive text 346 are shown. Furthermore, the selection window 347 allows a user to add additional columns to the depicted list. Particular objectives may be added or removed from the list by use of the "Add" button 340 and the "Remove" button 342. There are many possible customizable attributes or meta-data that can be 5 associated with an objective within an assessment project; these attributes are defined in the various templates for an Objective from (see FIG. 2B). The interface screen of FIG. 3G depicts some exemplary attributes for an objective. This particular objective is labeled "Non-Print Media" and relates to the study and comparison of non-print media. In the "Properties" area 348, a label and a description of the objective is entered by the 10 user. The description explains the purpose of the objective. The "options" area 350 allows the user to specify other attributes about the objective. Such as whether it is an academic objective rather than an administrative objective. This particular interface screen is related to the template 250 of FIG. 2B in that the screen, or web page, is presented to a user with a number of forms to be completed in which each form is defined 15 within the template Thus, the web page displays the objective form 254 and by selecting different tabs, the user can select different forms 256-262 to complete or edit. In other words, an Assessment Project template may be thought of as a complete process (e.g., summary, objective, measurement, findings, follow-up, review) accomplished with a set of interrelated forms. 20 The alignment of objectives with other goals and objectives is one advantageous benefit of the present system. The selection box 352 allows the user to select various available objectives and goals that have been defined. For example, departmental goals, institutional goals, and other operating unit goals and objectives can all be selected using the drop down box 352. Once the particular level of goals is 25 selected, the specific goals can be selected using the arrow button 354. The selected goals and objectives are shown in the box 349. This process allows the user to identify other goals, objectives, or standards are aligned with the present objective being defined. Thus, an association can be created between the goals of any of the different hierarchical levels of operating units defined within the assessment project and a newly defined 30 objective. One result is that a user at the institutional level (or some other level) can identify which lower level objectives relate to which of the strategic goals and objectives of the institution. In general, the result is that associations or alignments may be 20 WO 2006/110892 PCT/US2006/014043 accomplished between goals and objectives of higher, lower, and parallel hierarchical levels within the organization or institution. Within the window 349, an external standard is shown as one of the selected goals. External standards are not necessarily defined by the institution but may 5 be defined by a regulatory body or other third party such as an accreditation council. In such a case, the drop down selection window 352 would also allow the user to select from among available external standards and have the standard's goals listed for possible selection. These standards may change over time and, therefore, can be imported into predetermined storage locations so that the interface screen of FIG. 3G can populate the 10 selection windows with the latest information as it is provided by the standard's authors. Other areas 353, 355 of the interface screen allow the user to provide comments or attachments that are related to the objective. Such areas 353, 355 may be particularly helpful when the objective is being reviewed by a committee or other party. 15 Operating Units The above-described defining of goals, assessment projects, and objectives relies on the defining of operating units. The present system and methods provide techniques for defining operating units in a flexible manner that allows an institution to model whatever its organizational hierarchy structure looks like. The interface screen of 20 FIG. 3H depicts one exemplary method for a user to define operating units, programs, and courses within the "Division of Art". Using such an interface allows campuses, colleges, departments, and sub-departments to define their own hierarchy. The screen of FIG. 3H presents a list of available, or defined, units, programs, courses, and sections 360 that exist for the Division of Art. Optionally, visual 25 cues such as icons 364, 366, 368, may be provided so that the user can determine whether an item in the list is a course, a program or a unit. From the tool bar 362, a user can elect to add a "unit", a "program" or a "course" at the particular hierarchical position (e.g., Division of Art). To move to a different level, the user would select an appropriate choice from the lists of units that are presented. Further details of these items are 30 described later. 21 WO 2006/110892 PCT/US2006/014043 The identification and assignment of roles to different users is important to determining the access privileges to various data resources previously described. The screen of FIG. 31 depicts one exemplary way to manage users within a department. For example, the "user" button 372 allows a user to elect to add a new user to the department. 5 In this process, information about the user would be provided. Such information can include, for example, name, email address, user ID, role, access privileges, etc. Additional columns can be used as well via a user-selection process such as window 376. The screen of FIG. 31 shows a list 374 of existing users and their associated meta-data. This list 374 can be manipulated so that all users can be listed or 10 only certain, selected users can be listed. Thus, using the drop-down window 370, a user can elect to display within the list 374 only those users satisfying a particular role. Curriculum Planning and Mapping The previously described interface screens relate to the general category of 15 goals and assessment planning for an operating unit. Such planning usually takes place at the relatively higher layers of the institution and relate to the strategic goals and objectives of the institution. As shown, however, lower-level operating units can define objectives and goals as well. Finally, at the lowest hierarchical levels, the objectives which are defined relate back, or are aligned to, the previously defined goals. At a higher 20 education institution, for example, the levels can be categorized as institutional level, program level, and classroom level. For example, within a program, a number of courses or sections may defined that are designed to satisfy the objectives of the program, the department and the institution. The following figures in FIGs. 4A-4C depict exemplary techniques for curriculum planning within a program. 25 Retuning quickly, to FIG. 3H, a user is presented a list with a number of courses. From this list, a particular course can be selected for further definition. FIG. 4A depicts a user interface screen that allows a user to define course-specific objectives for a course or course section within a program. Examples of a course objective, may include such items as critical thinking, on-line research, portrait drawing, etc. The left set of tabs 30 400 identify for the user that the general functionality relates to curriculum planning and, more particularly, to defining section objectives. 22 WO 2006/110892 PCTIUS2006/014043 The interface screen of FIG. 4A includes a table that lists the already defined objectives of this course 405. The attributes which are defined for a course may include, but are not limited to, an objective name 402, the location of its definition 404, the category of the objective 406, the rubric 408, by which the objective is measured and 5 user access privileges for each objective. There is also a column 410 that indicates how many content items have been aligned to the particular objective of that row. Thus, when the user elects to add an objective, using the button 403, a definition screen is presented to the user that allows the user to enter information relating to these attributes. For example, a form can be presented to the user with these fields 402, 404, 406, 408, 410, 412 (or 10 other fields as defined by the associated template) and the user can enter data or select from pre-populated data in order to define the objective. Other attributes (not shown) may include a textual description of the objective, a summary of the objective, and key words related to the concept of the objective. The "definition" attribute 404 allows a course objective to be defined in an 15 efficient manner. For example, a particular objective may be defined for a course (e.g., CS101) that will also be an objective for every section of that course irrespective of the specific instructor for the section. When the section-specific objectives are defined for a section, there is no need to re-define the particular course objective but rather simply select it from all the available objectives presented on a template during the definition 20 process. In addition, a default, or pre-selected, objective may be presented to the user as well. By selecting an already defined objective, all the attributes can be re-used and, optionally, changed by the user if desired. This functionality also allows objectives defined for one course to be selected for inclusion in an entirely different course without repeating the entire objective definition process. 25 An objective may be aligned with any previously defined objective or goal regardless of the operating unit in which that objective or goal was defined (for example, see FIG. 3G and the description accompanying that figure). For example, a goal at one operating unit may be aligned with an objective at an immediately lower-level operating unit which, itself, is aligned with an objective of an even lower-level operating unit. 30 When defining a course objective, the goals and objectives to which it is aligned can be identified in a number of ways. A course objective may, for example, be aligned with a program objective which itself is aligned with an institutional or department-level 23 WO 2006/110892 PCTIUS2006/014043 objective or goal. In this way, there is a linking between goal an objectives from different hierarchical levels of operating units. In one example, course content may be aligned with a course objective which itself is aligned with a program objective. In this way, the course content is indirectly aligned with program objectives (and possibly even higher 5 level goals and objectives). Because goals and objectives may change over time, the "align" button 401 is provided to allow a user to update the alignment information of an objective even after it has been initially defined. The rubric attribute 408 identifies the rubric used to measure the findings related to the objective. The present system includes functionality for building rubrics 10 related to course objectives and for other uses within an assessment project. One of ordinary skill will recognize that a variety of techniques and methods can be used to identify the subject matter of the rubric and arrange and define its structure. Once created, these rubrics are managed and stored by the present system to be available for various uses in their respective operating units. Also, assuming proper access privileges 15 are defined, rubrics may be shared between different operating units and assessment projects as well. The rubrics may be general in nature such as a "writing assignment" rubric that can be applied to any writing assignments. The objectives for a particular writing assignment may include, for example, "utilize research material" and "critical thinking". The rubric used will have a result that measures attainment of these objectives. 20 Another level of assessment planning that can occur is at the lesson level. In other words, the lessons are identified within a course and aligned with the course or section objectives. Thus, FIG. 4B depicts an interface screen that relates to lesson planning within a course. Within a course (e.g., Intro to Graphic design) 416, a user can define a number of lesson plans that are meant to achieve the course objectives identified 25 earlier. The table of FIG. 4B depicts a number of already-defined lesson plans and includes attributes of each lesson plan such as its name 418, where it is defined 420, the number of student resources, 422, number of faculty resources 424, the number of associated assessments 426, and access privileges 428. Additional attributes may be added and some of the attributes deleted from the table of FIG. 4B without departing from 30 the scope of the present invention. Similar to methods described earlier, the user can elect to add a new lesson plan (using the button 417) and be provided a template appropriate for its creation. In particular, the user can be offered a template that allows 24 WO 2006/110892 PCT/US2006/014043 the user to identify the attributes shown in the table of FIG. 4B as well as the course objectives to which the lesson plan aligns. During creation of a lesson plan, the user can specify such attributes as where, if at all, the lesson plan may already be defined, the number of course resources (e.g., books, papers, digital files, etc.) that are used within the 5 lesson plan and the number of faculty resources used. Within the definition of a lesson plan, there can also be the identification of the number, names, and types of assessments that are included. These assessments can include tests, quizzes, papers, collaborative assignments, on-line surveys, etc. Once the course objectives and the lesson plans have been defined, a 10 variety of curriculum planning and curriculum mapping investigations can be performed. Returning briefly to FIG. 4A, the column 410 identifies a number of curriculum items or content for a course have been aligned to a particular objective of that course. Turning now to FIG. 4C, one example of a curriculum-related report about a operating unit is depicted. In this report, a list of the courses 430 offered by an operating unit (e.g., 15 English Department) is shown. This list also includes an indication 432 of the type of curriculum activities within the course aligned with various objectives and goals. Thus, the columns of the list identify the activities within a course such as tests, collaboration, discussion threads, assignments, collections, etc. For each course in the list 430, the corresponding number of aligned items of each type are identified under each activity 20 type. Thus, the report summarizes what types of activities in each course are aligned with various objectives. By selecting one of the displayed numbers 433, the user can be presented with a detailed list of exactly which objectives with which that particular activity is aligned. The identified objectives may be limited to merely course objectives but may also include objectives and goals from all levels (e.g., higher, lower, and 25 parallel.) More advanced curriculum planning and mapping may be performed as well using the information that the user has entered about goals, objectives, courses and lessons. For example, FIG. 4D depicts an exemplary curriculum map 440 that graphically displays a mapping of program objectives to course offerings. Each column 444 of the 30 table 440 relates to a particular program objective and each row 442 relates to a course within the program. The cells of the table include an indicator 446 of whether or not a particular course is aligned with a particular program objective. The indicator 446 may 25 WO 2006/110892 PCT/US2006/014043 be a binary indicator (e.g., "yes" or "no") or may be multi-valued. Alternative measures can be the level at which the objective is met (e.g., introduction, re-enforcement, mastery, etc). Thus, a user can quickly realize which objectives may not have many courses aligned therewith or where there is a high-degree of redundancy. As a result, curriculum 5 planning decision can be made about what new courses might be added and what their content should be. The objectives just described have been identified as "program" objectives merely as a practical example. However, one of ordinary skill will recognize that these objectives may also be goals and objectives from higher, lower, and parallel levels in the institution hierarchy. 10 Another possible curriculum mapping function can include a different view of courses and aligned objectives. For example, an operating unit that "owns" a course (e.g., the Computer Science Department) can use the information described above to identify all other units that have aligned program objectives to that course. Thus, the curriculum plan can be viewed from both a top-down approach as well as a bottom-up 15 approach. Potential for new courses or changes to existing courses within a program can be determined by identifying which goals and objectives are being ignored by the offerings within a program. Furthermore, within the courses themselves, lesson plans can be modified or augmented to more closely align with the course objectives (which indirectly align to the other levels of the hierarchy). From the top level of the institution, 20 evidence can be automatically gathered from across the various operating units of how the institution's goals and objectives are being met. This evidence is collected automatically, systematically, and objectively during the performance of the assessment projects and is readily available for internal evaluation by the institution as well as by external accreditation boards. 25 Instrument Creation and Management FIG. 5A depicts a flowchart of an exemplary method for deploying instruments within the assessment system 102 described herein in accordance with the principles of the present invention. In general, an instrument is a data collection tool that 30 can include, for example, surveys, quizzes, portfolios, assignments, assignment binders, papers, tests. These instruments are used to collect data that is used for various purposes within an assessment project. Some data may have rubrics applied to it, other data may 26 WO 2006/110892 PCT/US2006/014043 be used to measure achievement of a goal or objective, and other data may be used to quantify values being assessed within the project. For purposes of explaining FIG. 5A, the specific example of a test as the instrument will be used without an intent to limit instruments to only tests. 5 In step 502, a user identifies a new test instrument through the user interface. This is typically performed by making a selection to create a new test or to edit an existing test. The user continues, in step 504, by defining the contents of the test, or instrument, itself. Because of the hierarchical arrangement of some of the operating units within the present system, it is advantageous that a test can incorporate default 10 information required by a department or the institution. For example, every test may require the student to sign a "honor" clause or to answer certain demographic information. Thus, when a new test is defined, it will automatically include such information. In the example where the instrument is a survey, the institution may have certain required questions and each department may have its own auxiliary questions. 15 Thus, the starting "canvas" for a survey instrument may initially have a variety of default questions depending on the unit under which the instrument is created. In step 506, the recipient list for the instrument is selected. This list may be generated based on data about user, units or courses. The list might be generated dynamically at the time of deployment or have been generated previously and saved as a 20 static list. For example, the recipient list may be defined as all 400-level sections taught by a tenured professor. If generated dynamically, that list would show all 400-level sections taught by a tenured professor at the time of deployment, therefore the dynamically generated list might be different when used the Fall of 2006 and the Fall of 2007, as the sections taught by tenured faculty will have changed. However, if generated 25 statically in the Fall of 2006, it will store all 400-level sections taught by a tenured professor at the time of the list creation. That list would be the same in Fall of 2007 as it was in Fall of 2006, because it represents a static view of the data as it was at the time of list-generation. Furthermore, an instrument may be copied and then modified such that the modified copy is deployed to an earlier population or a different population 30 altogether. Accordingly, both the original instrument and the changed copy are available for deployment in the future. 27 WO 2006/110892 PCT/US2006/014043 Deployment of an instrument can occur in a variety of ways, some of which rely on the academic system 116 of FIG. 1, in particular, the community system portion of the academic system 116. As mentioned briefly, and described in more detail in the incorporated patent application, the academic system and community system 5 include a virtual space that each student may visit to send and receive information and deployment of an instrument is often accomplished within this environment. In one particular implementation, the academic system may include logical components such as a learning system, a content system, and a transaction system such as those offered commercially by the present Assignee. Typically, there is a home page that a user is 10 presented after logging in to the academic system 116. From the home page, the user can navigate to a "My Courses" area, an "E-Mail" area, a "Financial" area, etc. and from the "My Courses" area, the user can open respective windows specific to each course in which the student is enrolled. Thus, deployment of an instrument can entail sending an executable web module that displays on the student's home page or within the appropriate 15 course area. The deployment could also be accomplished through a simple HTML link, or an e-mail message. The deployment (for example, in the case of employer surveys of recent graduates) may also include sending information to parties external to both the assessment system 102 and the academic system 116. Advantageously, the instrument can be deployed in more than a single media (e.g., off-line via bubble sheets) as well 20 depending on the recipients preferences or some other setting within the assessment system 102. FIG. 5B depicts a screenshot of a user interface for defining test instruments for an English department. The exemplary attributes of the instrument that are explicitly shown include the instrument name 510, the number of available versions 25 512, information about previous deployments 514, and access privileges 516. Other attributes may include the deployed date, the due date, and a textual description of the instrument. Each instrument has a selection link that allows the user to select the instrument for further action (e.g., remove) and a "Manage" button 520 for editing and management. Additionally, there is a "deploy" button for when the user wants to actually 30 deploy a previously defined instrument. If multiple versions of an instrument are available, the "deploy" button may cause an intermediary window to inquire of the user 28 WO 2006/110892 PCT/US2006/014043 as to which version to deploy. As part of the deployment, the user is presented with an interface screen (not shown) that allows selection of the desired recipients. Adding a new test instrument is initiated by selecting the "Add" button 519. Doing so causes a new test creation page to be presented to the user for modification 5 and editing. From this from a user can define such items as the questions of the test, how it is scored, the instructions for completing the test, and similar test characteristics. FIG. SC is similar to FIG. 5B except that the items in column 530 refer to survey instruments rather than test instruments. FIG. 5D depicts an exemplary form defining a survey instrument that might be offered to a user when that user elects to 10 define a new survey. For example, the form identifies the survey name 532 and includes a brief summary area 534 about the survey itself. The particular example of FIG. 5D includes a technique 536 for breaking the survey into sections in which different questions 538 can be entered. As shown, the questions can be relatively complex with multiple scoring criteria. The depicted survey form is exemplary in nature and one of 15 ordinary skill will recognize that many other types and forms of questions may be used without departing from the scope of the present invention. The present assessment system 102 also provides, via its user interface, methods for managing course evaluation instruments for a particular operating unit. For example, the interface screen 540 of FIG. 5E provides a list 542 of course evaluations 20 available to the English Department. The displayed list of available evaluations can be modified by selecting the appropriate level of detail from the window 544. Thus, a user may select (if access privileges allow) evaluations from other units and even from higher level operating units. From within the management screen 540, the user can select options to manage 550 the evaluation or to deploy 548 the evaluation. Because multiple 25 deployments of the same evaluation are possible, the screen 540 includes deployment statistics 546 about this particular evaluation. The user can elect to create a new course evaluation by selecting the "Add" button 552. Similar to techniques already described, an appropriate authoring form is provided to the user in order to define the new evaluation. An exemplary 30 authoring form is depicted in the interface screen 560 of FIG. 5F. The authorship of an evaluation can be distributed among various authors because some questions may be from the institution, others may be from the department, still others may be from the program, 29 WO 2006/110892 PCT/US2006/014043 some may be course specific and still others may be instructor specific. As explained earlier, the selection of recipients of an instrument (e.g., an evaluation) can be based on user demographic information, course attribute information, or a combination of both. Additionally, course evaluations may be delivered such that a recipient is presented with 5 an aggregation of questions that is determined dynamically by the recipient's context (e.g., institution, college, department, course, section, sex, ethnicity, residence status, etc.). Because the assessment system 102 already includes a variety of information and data that is typically used to perform questionnaire functions within the 10 institution, a questionnaire manager may be provided that defines standard questionnaires and their contents. Thus, when a user desires to generate a questionnaire, it can be initially created within the assessment system 102 using the templates maintained by the questionnaire manager. The benefits of such a system include the advantageous result that a department, for example, will receive similarly formatted reports and content from 15 each of its separate programs. The exemplary interface screen of FIG. 5G depicts a questionnaire generator that provides a selection window 562 that permits a user to define new fields of the questionnaire. When a new field is added, then its label, instructions and other attributes can be defined as well. The screen view of FIG. 5G shows a list 564 of all the defined fields for the current questionnaire. A preview mode (not shown) is 20 available to show how the questionnaire layout looks for presentation to a user. One of ordinary skill will recognize that there are similarities between a survey and other instruments for collecting data. Unlike a survey, some data collection instruments are not delivered to a specific set of users and may be open for anyone to complete. For example, a satisfaction survey would be sent to a specific set of users 25 typically all at the same time and users would only be able to submit once. However, a curriculum change request form would be available for a user to complete at any time, as determined by the user's need to request a change in the curriculum (e.g., a faculty requesting a new course be added to the curriculum), and the user may submit it multiple times as the need to provide the data may arises on multiple occasions. The data 30 collection capabilities of the survey and course evaluation tools can be used for such general purpose data collection instruments as well, with a change to the deployment model. Instead of being deployed to a specific set of users, such a general purpose data 30 WO 2006/110892 PCT/US2006/014043 collection instrument could be made accessible to a set of users for submission at the discretion of the user based on the user's needs. The exemplary interface screen of FIG. 5G depicts a generator of such a general purpose data collection instrument that provides a selection window 562 that permits a user to define new fields of the instrument. When 5 a new field is added, then its label, instructions and other attributes can be defined as well. The screen view of FIG. 5G shows a list 564 of all the defined fields for the current instrument. A preview mode (not shown) is available to show how the layout looks for presentation to a user. The availability of data across a wide variety of units, courses, and users 10 raises concerns of uniform consistent and efficient evaluation of the data. Thus, an evaluation portfolio manager is provided for an operating unit that lists evaluation templates available to users within the operating unit. Typically, within a higher education institution, for example, the evaluation template manager would be at a relatively high level such as at the college or department level. An evaluation portfolio 15 relates to a particular portion of an assessment project. For example, a college may have to satisfy three different accreditation boards to be "accredited". A different portfolio could be created for each of the different accreditation boards. Each portfolio would have identified, therein, a series of objectives (as outlined by the accreditation board). Drilling down into each objective would be an identification of how "satisfaction" of each 20 objective is measured (a survey, a rubric for writing samples, and a test). The next step then is to collect the artifacts that pertain to each of these measures. Thus, a portfolio includes a label, a number of objectives, one or more ways to measure instruments related to each objective, and the artifacts themselves. At least three different types of assessment capabilities are provided in conjunction with evaluation portfolios. For 25 example, a student or faculty may perform self-assessment. Additionally, a faculty member, for example, may use an evaluation portfolio to perform assessment of an individual (e.g., a student) with respect to a course objective, one or more program objectives, an institutional goal etc. Furthermore, the evaluation portfolios may aggregate results from many entities within the institution to provide assessment of institutional or 30 program-level objectives such as, for example, accreditation criteria. The interface screen of FIG. 5H depicts an exemplary evaluation portfolio manager that lists the available evaluation portfolios. In this example, one evaluation 31 WO 2006/110892 PCTIUS2006/014043 portfolio 566 is listed and is related to NCATE accreditation. The screen further displays attributes of this portfolio such as where it is defined 568, and its deployments 570. From this screen, the user can manage 573 a portfolio, add 571 a new portfolio, or deploy 570 an existing portfolio. 5 Selecting the "Add" button 571 of the portfolio manager interface will reveal the interface screen of FIG. 51 that shows a template of a portfolio that includes a listing of the areas that are the subject of the evaluation portfolio. This template can then be modified by the user. A portfolio is intended to include artifacts from students that exhibit certain outcomes. For example, artifacts may be papers, tests, surveys, quizzes, 10 collaborative exercise, assignments, etc. These artifacts are submitted by a student (or collected by the faculty) in order to be applied towards one or more of the areas identified within the evaluation portfolio template. In the example portfolio 566, there are four areas of interest for evaluation. A list of these areas 580 and their respective descriptions 582 are shown. By selecting one of the links 580, a more detailed view of the specific 15 properties for that portfolio area can be provided to the user. This evaluation portfolio is deployed by a user to selected recipients using the "deploy" button 572 shown in FIG. 5H. As described earlier, a list of recipients may be defined or selected for deployment that varies according a number of criteria. Once the evaluation portfolios are deployed to students within the academic system 116 (see 20 FIG. 1), these students can submit artifacts related to the different areas. Thus, the students to which the evaluation portfolio is deployed each complete the evaluation portfolio. Completing the portfolio is accomplished by adding artifacts to the portfolio. As a result, the assessment system 102 collects artifacts from a number of students related to the evaluation areas 580 of the evaluation portfolio. Because, the artifacts can include 25 identifying and other meta-data, robust filtering and searching of the submitted and stored artifacts are possible. For example, random samples of submitted artifacts may be selected for an area or, alternatively, only artifacts from female students that live off campus may be selected. When defining the evaluation portfolio, the user may also be given the 30 opportunity to define the artifacts that should be collected for each of the areas. Thus, by selecting one of the areas 580, the user may be provided with an artifact definition template that specifies attributes of the artifact (e.g., the scoring system, the fields, and 32 WO 2006/110892 PCT/US2006/014043 the date). One example area in a portfolio may be to show that a student nurse can perform an intramuscular delivery of medicine. The artifact for documenting this area may require the name of the managing clinical nurse, a statement that the student was observed performing the injection, the name of the hospital, and the date of the 5 observation. Another area in a portfolio may relate to satisfying a creative writing requirement. The artifact to apply to this area may include a paper that was scored according to a particular rubric. Thus, the artifacts themselves may vary greatly depending on the area 580 within an evaluation portfolio. Accordingly, the present assessment system 102 allows a user to define templates for the different types of artifacts 10 and then, subsequently, when creating an evaluation portfolio, allows the selection of these templates to identify an artifact when describing an area within the evaluation portfolio, such as that in FIG. 51. A concept that is somewhat similar to a portfolio is that of an assignment binder. An assignment binder typically pertains to a particular accreditation program and 15 collects artifacts related to that purpose over the course of one or more terms. The contents of the assignment binder can be used to support a review of a particular program or curriculum. However, unlike a portfolio, an assignment binder simply collects the artifacts without necessarily assigning them to particular areas or measuring them against predetermined rubrics. FIG. 5J depicts an exemplary interface screen for managing 20 assignment binders. As shown, a binder is created within the College of Engineering and includes a name 590 and a description 592. Other attributes of a binder include, for example, the term for which it applies and user access privileges. Through the use of user lists and course lists, the selection of the artifacts to include in the binder can be controlled to provide desired cross-sections of available artifacts. Although an 25 assignment binder may include artifacts from a wide range of students, selective filtering based on user lists, unit lists, or course lists can be used to extract a limited subset of artifacts from students matching particular criteria. Additionally, artifacts can be selectively returned that match a filter, or search criteria, related to the grade of an assignment, or other assignment specific attributes. Thus, different binders may be built, 30 from the same pool of artifacts, by automatically retrieving artifacts matching one or more defined filter criteria. Within the interface of FIG. 5J, more information and the artifacts themselves may be presented to a user through selection of one of the links 590. 33 WO 2006/110892 PCT/US2006/014043 The definition of goals and objectives, the creation of rubrics, and the collection of artifacts as described above allow automatic evaluation of a student's performance against program or course objectives. The assessment system 102 and its integration with the other campus systems 114 and the campus academic system 116 5 simplify sharing of information among the various systems. As a result, the integration of these systems results in the storage of rubrics, artifacts, various objectives, different goals, and a definition of the relationship or association between these different data entities. Using techniques known to one of ordinary skill, this integrated information can be organized and filtered to identify and report various aspects related to the stored data that 10 may be pertinent to assessment within an institution. For example, a "score-sheet" such as that depicted in FIG. 6A may be built as the result of the integration and flexibility provided by the present assessment system. This table of FIG. 6A includes a list of students 602 and an indication of how they satisfy each objective 604, 606. The indication 608 may be a numerical score or some other 15 qualitative-type score. For example, in the table of FIG. 6A, the indicators include "below", "meets", and "exceeds". These indicators are based on the definition of the rubrics and the scores that are assigned to the artifacts which the students supplied in response to the rubrics. Because the rubrics are aligned with previously defined goals and objectives, these indicators can be automatically determined from these objectives. The 20 table of FIG. 6A also includes a drop down window 610 that allows a user to select other objectives to display (e.g., critical thinking) as columns in the table. FIG. 6B depicts a flowchart of an exemplary method by which the table of FIG. 6A may be generated. In step 620, instruments, rubrics, and goals and objectives are defined as described earlier. As part of the definition process, the instruments are 25 associated with different rubrics and these rubrics are aligned with various goals and objectives. Next, the instruments (e.g., test and quizzes) are deployed, in step 622, to students. Other activities that are aligned with various rubrics may be performed as well. In response to the deployment of instruments and other activities, results are received from the students, in step 624, by the assessment system. The receipt of 30 results may be through automatic submission by the students through a campus academic system 116 as described earlier or through an instructor who receives the results directly from the students. The rubrics may then be applied to the results, in step 626. Once the 34 WO 2006/110892 PCTIUS2006/014043 rubric is applied, scores can be assigned to the results for each of the defined goals and objectives so that a report can be generated (see FIG. 6A), in step 628. The results and assigned scores along with other data may then be stored, in step 630, so that records of the assessment may be maintained for future reference and analysis. 5 Reporting The following figures depict a variety of different reports that may be generated using the assessment tools described herein and the data that is collected as a result of using these assessment tools. One of ordinary skill will recognize that the 10 specific format and contents of each report may vary greatly without departing from the scope of the present invention. However, these reports show the great breadth and depth of the types of data collection, some of which may be automatic, and analysis provided by the assessment techniques and methods described herein. One advantageous benefit of the present system and techniques evidenced within the exemplary reports is the sharing 15 of data from among different level operating units. For example, the data from a number of different course may be "rolled-up" into a report that is generated at a department level and this report may include data that is collected at the course or section level. Thus, a report related to a department goal or objective may include results from artifacts that were submitted at the course level even though there are intervening operating units 20 between the two levels within the model of the institution. This sharing of data between different operating units and different levels of operating units occurs in the other direction as well. For example, when defining and identifying artifacts, instruments, surveys, portfolios, curriculum content, etc. at a lower-level operating unit attributes from higher-level operating units may be automatically inherited without the need to be 25 explicitly defined FIG. 7A depicts a list 702 of report templates that have been pre-defined for use by a user. As new reports are defined, they can be added to the list. In operation, a user would select one of the available templates and then fill out the fields of the template with the particular data for their report. To help in the selection of the report, 30 the list can also include other identifying features such as a description 704 associated with each report template. 35 WO 2006/110892 PCT/US2006/014043 One example report is shown in FIG. 7B (which is a more detailed view FIG. 4C); this report displays a table of "Course vs. Objective". In generating this table, the English Department, for example, may use the template to select some or all of its courses and then select all or some of its department objectives. Because the definition of 5 the courses also included aligning it with certain objectives, this report can be generated to show how each course satisfies each objective. The table includes an indicator 706 at each intersection of an objective and course. This indicator could be quantitative such as a ranked numerical score or it can be qualitative as shown in FIG. 7B. Exemplary qualitative categories are shown in the legend 708 of the table. 10 FIG. 7C depicts a table that might be useful to the institution which shows, by department, the status of course evaluations. Because the generation and return of course evaluations are automatically and systematically handled by the present assessment system, these statistics are readily available without manual collection and tallying of results by each department. Furthermore, not only is the status of completion 15 revealed in the table but the overall scores 710 of the evaluations can be calculated and displayed as well. In addition to tabular reports, more graphical report templates are provided as well. FIGs. 7D and 7E depict a dashboard-type report that graphically indicates certain indicators that an institution or other operating unit may be interested in. In a dashboard, 20 certain indicators are defined along with the attributes for that indicator. For example, the "traffic light" indicator 716 of FIG. 7D has three possible states (e.g., read, yellow, green) which are mapped to a particular graduation rate defined by the institution. The data collection described herein provides the raw data for this indicator and the definition of the indicator is used to map the raw data into the "color" of the traffic light. Similarly, a 25 pie chart 718 can be created from raw data to show the percentage of students who passed and failed a course. Similar raw data may be displayed as a bar chart 720 as well. The template definition screen of FIG. 7E depicts one way in which such a dashboard may be created. For example, a user can create a new metric 714 using the "Add" button 713. In the creation process, the user is provided with a selection window 712 that selects the 30 type of indicator associated with the metric. FIG. 7F depicts another type of report that resembles a scorecard. This type of report may be advantageously used to compare results of the same metric over 36 WO 2006/110892 PCT/US2006/014043 different instances. For example, the scorecard of FIG. 7F shows such statistics as student SAT scores, student satisfaction, the number of minority faculty, etc. However, from an institution perspective, the scorecard provides the values for these metrics for the four most recent terms 720. Thus comparison and tracking of different metrics may be 5 accomplished using a scorecard. FIG. 7G shows the details of using a scorecard template to generate a scorecard such as the one in FIG. 7F. Using this template, a user can define the metric 722 that is going to be applied and displayed along with the class of individuals 724 to whom it applies. A description 726 is useful for identifying the metric and a selection window 728 allows the 10 user to define the type of data that results from applying the metric. For example, the data may be "dollars", "a percentage", "raw numbers", "an average", etc. Near the bottom of the template of FIG. 7G is a window 730 that allows the user to select the different time frames for which these metrics will be included in the scorecard. It is useful, once a report has been defined and run, to be able to save the 15 report for future reference. As shown in FIG. 7H, the assessment system 102 described herein provides that capability. When storing the reports, it may be advantageous to allow the user to set permissions 732 for the report such that certain classes of users may be prevented from accessing the report or, alternatively, to specify a wide group of user roles that may access the report so as to ensure its widest dissemination. 20 FIG. 71 depicts a relatively simple bar chart; however, its contents reveal the great breadth of data collection and reporting capability provided by the assessment system 102 described herein. At the institution, one level of operating units may be "the college". The different colleges 736 may include the business college, the engineering college, etc. Within each college are courses and each of these courses have course 25 evaluations that are deployed and collected and given a rating 738. Each course also has a particular number of seats or students and this data may be stored in a third-party campus information system maintained by the institution or it may be stored within the present assessment system. As a result, the bar graphs of FIG. 71 allow a reporting of data such that the institution can discover the average course evaluation ratings within 30 each different college as a function of class size. The legend 734 depicts an exemplary breakdown of class sizes. 37 WO 2006/110892 PCT/US2006/014043 FIG. 7J depicts another type of graph 741 that can be used to display data. In this instance the data is from an off-campus survey that collected ratings 740 from employers of recent graduates. Additionally, the graphs are broken down by different programs 742 within the college of engineering. Thus, as an operating unit, the college of 5 engineering can analyze data according to sub-units defined previously. For purposes of auditing and other monitoring functions, the assessment system 102 can track activities of users within the system in order to know what documents they have accessed and modified. Thus, the table of FIG. 7K can be created that shows recent events 744 within the assessment system 102 that are associated with 10 one or more users 746. This data may be filtered so as to create a report that is based on time, based on a particular user, based on a class of users, etc. Other Tools There are a number of additional administrative type functions that 15 simplify the management of various assessment projects and initiatives. These functions are not required by the assessment system 102 described herein but facilitate its operation and provide advantages to automating the processes described herein. For example, users can be grouped into committees in order to simplify assigning roles and access privileges. Templates and other documents, when created, 20 may have access privileges assigned based on which committee may access them, approve them, modify them, etc. By associating staff and faculty into different defined committees, the access privileges remain consistent even though faculty and staff can change. Additionally, upon creation, a committee can be provided with its own discussion board area, its own shared calendar, its own e-mail contact list, as well as its 25 own storage repository for the documents it needs and generates. The interface screen of FIG. 8A depicts a tool useful for listing existing committees 804 and adding 802 new committees. FIG. 8B depicts a task manager function which allows an operating unit to identify tasks 804 and track their status. FIG. 8C depicts some of the detailed information 30 that may be collected about a student or faculty. This information may be manually entered or populated from another system such as an SIS of the institution. Information 38 WO 2006/110892 PCTIUS2006/014043 such as gender, ethnicity, financial aid, native language, SAT scores, etc. may be collected. As a result, the user lists described earlier can be finely defined to include only a desired subset of students when deploying instruments or reporting results and other data. Some of the fields shown in FIG. 8C may be static while others are dynamic. Thus, 5 as data changes (e.g., the students GPA), this data may be automatically pulled from the SIS when needed. The assessment system 102 can access the SIS system in at least two different ways. A snapshot of the SIS can be made that the assessment system 102 stores and uses for its purposes or the SIS system can be dynamically accessed each time data from it is needed. 10 Because of the automatic and systematic collections of data described herein, there will be a large number of files generated and stored. Organization of these files can be dependent on operating units, assessment projects, programs, courses, instrument type, etc. FIG. 8D depicts an exemplary file manager that may be used to utilize the files within the system for the English Department 808. Within the manager 15 window, a list of file collections 810 can be provided along with associated attributes 811. To help locate file collections, a drop down window 812 may be provided to allow the user to apply certain filters so that only desired file collections are displayed. FIGs. 8E - 8H depict a template manager that allows operating units to define the templates that will be used in assessment projects and to list existing templates 20 814. Using this manager, a user can modify 815 an existing template or add 816 a new template. One attribute of a template is the owner 818 of the template which reflects the operating unit that created the template. As shown in FIG. 8E, an owner may be an operating unit hierarchically above the present operating unit. Accordingly, the user of the "college of Liberal Arts", for example, can start with a template provided by "the 25 Institution" and add college-specific revisions. In this way, the institution can have complete or flexible control over the contents of assessment project templates to ensure some level of consistency across the different sub-operating units but also allow customization. FIG. 8F depicts a interface screen that allows creation of an assessment 30 project template. Using this interface screen, the user can specify what forms 820 are to be included in a template and view a description 822 of each form. For example, returning briefly to FIG. 3G, the names in the column 820 correspond to the form names 39 WO 2006/110892 PCTIUS2006/014043 that are along the top tabs 252-262. As shown, however, the exact form names may vary as well as the number of forms defined within a template. Exemplary forms shown in FIG. 8F include: Objectives: The objectives for the assessment project 5 Measurements: The measurements that will be used to measure the objective (e.g., test, survey, portfolio) Action Plan: The ordered steps to be taken within the Assessment Project Findings: The findings resulting from the measurements 10 Follow-up: The steps for a follow-up plan, if any. The user is presented with a way to define these forms to handle each data collection process. Thus, when a user initiates the start of a new assessment project, the templates that are provided to the user are those templates and forms that are defined in this process. One of ordinary skill will recognize that other or additional forms may be 15 used as well to define an assessment project template without departing from the scope of the present invention. FIG. 8G depicts an exemplary form definition interface for the "Objectives" form. A similar form definition interface is also provided for the other forms identified as well. The interface screen of FIG. 8G allows a user to add fields 824 20 to the form which define the way the form is presented when displayed for a user to use when defining a new assessment project. The interface screen of FIG. 8G shows how the details of the form's fields are defined and specified. For example, field 7 is provided with a user configurable "type" 826, and "label" 828. Additionally, the user defines (for this particular field), the "drop-down items" 830 from which the user can select. The 25 specific fields and their attributes are not critical to an understanding of the present assessment system 102 as one of ordinary skill will recognize there are many functionally equivalent ways to allow users to organize and define templates and the forms and fields that make up a template. However, the results are advantageously depicted in FIG. 3G where the particular fields of FIG. 8G result in the template screen that was seen earlier in 30 FIG. 3G. 40 WO 2006/110892 PCTIUS2006/014043 Within an institution, there may be many different operating units, some of which have different terms. For example, the medical school, the law school, and the engineering department may all have different term lengths and days. Thus, instruments or portfolios that must be completed by the end of a term or the middle of a term may 5 have different due dates for different operating units. A term manager is therefore provided that allows a user to define different terms. In the interface screen of FIG. 8H, the user is presented with a way to specify a term name 832 and assign to it a start date 834 and an end date 836. One advantageous function described previously related to user lists and 10 their use when deploying instruments and portfolios. In particular, a user list is a multi faceted concept that refers to both its definition and its members. Thus, a user list may be defined as "all male juniors in the College of Engineering" and the members which satisfy that definition are also referred to as the "user list". Referring back to FIG. 8C, a variety of user demographic information is collected and maintained by the assessment 15 system. This information can be employed to define very specific subsets of students by presenting the fields of FIG. 8C to a user as selectable filters which the user can configure to define the desired members for a particular user list. Once the user list is defined, the user has the option of making it a static list or a dynamic list. If it is a static list, then the user can apply the definition at a 20 particular date to select the members of the user list. When some other process references this user list, then only those members within the original list will apply (e.g., only those members will receive a particular course evaluation). If it is a dynamic list, then the selected members are determined each time the user list is referenced. In other words, a survey deployed to a particular dynamic user list may be sent to different members if sent 25 on two different dates. The user manager interface screen of FIG. 81 provides a way for a user to maintain, define, and select different user lists. Using an "Add" button 844, a user is presented a new definition screen which allows them to configure the filters for selecting the desired subset of users. Once defined, this user list can be named and stored so that it 30 is available in the list of FIG. 81. This list provides an identification 840 of available user lists along with a number 842 of the members within the user list. 41 WO 2006/110892 PCT/US20061014043 Similar to a user list, the assessment system 102 also provides the user with a way to selectively define course lists. A course list includes those courses that match a specified set of criteria. The attributes of a course (to which the criteria are applied) may be specified and stored within the assessment system 102 or be imported 5 from another campus computer system such as an online catalog or similar database. The breadth of different course lists that can be created is limited only by the possible number of permutations allowed by the different attributes used to describe a course. Thus, very finely defined course lists may be created to allow selective deployment of portfolios, course evaluations, and other surveys. The course list manager interface screen of FIG. 10 8J depicts an exemplary way for a user to be presented with existing course lists and to define a new course list. Although not shown, creating a new course list involves a user being presented with an interface that allows defining the attributes and their values which must be satisfied to be a member of that particular course list. Once defined, then a dynamic or static course list can be created and stored. 15 Because operating units may be hierarchically arranged, it is possible to take advantage of that organization to simplify the deployment of certain instruments/survey and to perform certain reporting. For example, in an institution made of multiple campuses, each with multiple colleges, it is possible to ensure the same survey is deployed to all colleges of a campus simply by specifying the campuses without the 20 need to specify each individual college. This functionality is provided by allowing the user to define unit lists. A unit list is simply those operating units that meet particular criteria. The interface screen of FIG. 8K depicts a unit list manager tool that displays defined unit lists 860. Selecting one of those lists 860 will display the names of the operating units within that unit list. Using the screen of FIG. 8K, the user may also elect 25 to define a new unit list as well. By automating the process of defining and collecting information related to assessment projects, a vast data warehouse can be generated that provides many different views of the assessment process. Searching through that data warehouse using selectable criteria allows a user to create a view of the data that may be useful for a 30 variety of different purposes. One of ordinary skill will recognize that there are numerous ways to categorize and store data to aid in searching and locating pertinent information. The search interface screen of FIG. 8L is merely one exemplary way in which a view of 42 WO 2006/110892 PCT/US2006/014043 the data can be defined and retrieved from within the assessment system 102 described herein. Using this screen, the user can select from a multitude of search criteria 862. This search criteria can include, for example, unit names 864, keywords, 866, author 868, dates 869, objectives 870, aligned goals 872, etc. 5 Assessment System and Academic System The previously-described functionality of the assessment system 102 has been focused on the assessment system in isolation from many of the other systems available within the institution or organization. However, as briefly mentioned with 10 respect to FIG. 1, a number of other systems may exist within the institution that may advantageously be integrated with the present assessment system. As mentioned earlier, the assessment system 102 described herein can operate in conjunction with an academic system as described in the previously incorporated patent and patent applications. This Internet-based learning system is also called the academic system 116 (See FIG. 1) in that 15 it provides a virtual, on-line community for students. FIG. 9A depicts an exemplary screen that a user may encounter when logged into the academic system 116. In FIG. 9A the user is provided with electronic tools 902 such as e-mail, calendar, and a directory. The academic system 116 also allows the user to connect to external campus computer systems using links 904 such as the Registrar's office or the 20 Advising office. Of particular use to a student, the "My Courses" tab 906 allows the user to drill down into the courses for which they are enrolled to see course assignments, lecture notes, external resources, discussion boards etc. Additionally, system wide announcements can be provided in a window 908. Specific to the assessment system functionality, two modules 910 and 912 25 are displayed on the user's interface page 900. The module 910 describes an assessment exercise that a student must complete by a certain date. It includes a link 911 that the user can select to be taken to the assessment exercise. The implementation of the exercise may be on some other computer system that receives and records the user's responses. These responses and scores are automatically provided to the assessment system 102 and 30 associated with that particular user for that particular assessment exercise. The other module 912 relates to one or more surveys that the user is to complete. Similar to the 43 WO 2006/110892 PCT/US2006/014043 assessment exercise, the student selects and then completes the survey with the results automatically returned to the assessment system. The modules 910, 912 refer to instruments that are defined by a user of the assessment system 102 using the techniques previously described. The user of the 5 assessment system 102 also employs the user list or course list (or both) to select the group of students to receive the respective instruments and then deploys the instruments. Once deployed, the assessment system 102 sends notification to the academic system to locate those students selected to receive the instrument. Within the academic system, a notation is made within the records associated with the implicated students regarding the 10 newly available instrument. When the students next log in to the system, their respective home pages are rendered and the included notation results in a display of the appropriate modules 910, 912. An alternative deployment method would be to add a new instrument within the course area of a particular course. Thus when a student drills down to that course, the course page will be rendered and it will display a link to any new instruments. 15 Faculty members also use the academic system 116 and may be interested in assessment projects within their department or involving the student population that they interact with. For example, window 920 of FIG. 9B provides, for the faculty member, links to information on unit forms, surveys, course evaluations, results, tests, etc. Window 922 relates to a particular assessment project (e.g., "2006 Undergraduate 20 Degree Review") and provides links to information such as the objectives and findings collected so far with respect to this project. Access to this information is provided through appropriate communications interfaces between the assessment system 102 and the academic system. Furthermore, user roles and identities can be used to ensure information is presented to only those users having the required access privileges. 25 As mentioned, within the course-specific areas that a student may access from their virtual space within the academic system, a student has access to a variety of information pertaining to that course such as assignments, reading lists, discussion boards, lecture notes, etc. With the connection to the assessment system 102 in place, the student may now also be provided with a listing of the goals and objectives aligned to 30 each course. Additionally, by drilling down into each lesson within the course, the student can see what are the objectives of today's or some other day's lesson. Additionally, objectives and goals can be aligned to individual content items within the 44 WO 2006/110892 PCT/US2006/014043 course. For example, a particular discussion board topic may be presented to satisfy a particular course or department objective. This alignment can be specified by the instructor and displayed within the course area visible to the student. The academic system 116 includes both a content management system and 5 an administrative system. The assessment system 102 described previously can advantageously utilize this functionality already provided by the academic system. For example, the assessment-related content may be organized by operating unit and managed and accessed using the shared content window of the academic system. FIG. 9C shows the shared content window 929 that includes a unit content tab 930. The unit content tab 10 930 can include hierarchically arranged content within the unit as shown. Selection of specific content from the content tab 930 will display the associated file collections within a sub-window 932. From there, the file hierarchy can be traversed. The display of content in FIG. 9C may be logical in nature and does not necessarily reflect the actual hierarchical arrangement of files as stored within the underlying file system. The window 15 933 of FIG. 9D depicts an administrative screen of the academic system 116. Rather than having a separate administrative function for the assessment system, it can be provided through a section 934 of the window 933. As shown, an administrator can manage functions related to assessment projects, curriculum planning, lesson planning, testing, surveys, reporting etc. While not necessary, the integration of the assessment system 102 20 with the academic system 116 simplifies sharing of user roles, content, course attributes and also simplifies deploying instruments and collecting responses. All these activities can still be accomplished with diverse, separate systems but are advantageously simplified the tighter the two systems are integrated. 25 Template Driven In the above-described assessment system 102, the use of templates has been highlighted as an advantageous technique for defining and collecting the data to be used in assessment projects. For example, at the institutional level a template is provided which guides the user into providing strategic goals and mission and vision statements. 30 When an assessment project is begun, the user is presented with templates that allow objectives to be identified, measurements to be defined, and action plans to be specified. When the identified objectives are to be defined, a template is provided to guide the user 45 WO 2006/110892 PCT/US2006/014043 through this process and to align them with higher level goals and objectives. At the unit level, templates are provided to define portfolios, instruments, programs and program objectives. While at the course level, other appropriate templates are presented to a user to help develop lesson plans, instruments and course and content objectives. In each 5 instance, the lower level objectives and outcomes can be aligned with higher-level objectives and goals. As a result, a template driven system is provided that defines objectives at least at three different hierarchical levels and allows automatic alignment of the objectives throughout the three levels. Of course, more levels are possible as well, because the present assessment system provides operating units with the freedom to 10 model their organization an a flexible manner. Workflow The different phases of an assessment project can be considered as events within the assessment system 102. As such, a workflow system may be included to 15 monitor and facilitate an assessment project. For example, certain re-occurring assessment projects may be tied to the start or end of a term. Thus, a calendar function within a workflow process can initiate a creation of an appropriate assessment project template and forwarding it to the initial user responsible for managing the project. For such a project, as well as those manually initiated, the subsequent steps of the assessment 20 project may be handled by a workflow process. The task plan described earlier can include with each task the identity of the responsible parties and the disposition of the project after completion of each task. For example, once an exemplary assessment project reaches the stage where all of a department's objectives have been defined, then a subsequent step, for example, may be to create a curriculum map for that department. 25 Through use of a workflow process this automatic stepping through the task plan may be accomplished. Some workflow steps may merely include sending e-mails to committee members regarding the status of some phase of an assessment project. Another benefit of implementing a workflow system is that parts of the assessment project may have its access privileges automatically modified at each step of the task plan. For example, once 30 a set of program objectives are agreed upon by the committee, then these objectives become read-only and no one has permission to modify them Thus, because of the automatic and systematic collection of assessment-related information and tasks available 46 WO 2006/110892 PCTIUS2006/014043 within the assessment system 102 described herein, an automatic workflow capability may be included to further automate and monitor an institution's assessment initiatives. Export and Import 5 Assessment projects, and portions thereof, are data entities in and of themselves. For example, an English Department can create an assessment project that relates to how its courses align with its objectives. The creation of such a project includes a definition of objectives and goals, an organization of operating units, a definition of courses, and an identification of course objectives. Inherent in this data is how all the 10 objectives from the different hierarchical levels align with one another. If further detail is desired, each course can include information on course contents and instruments deployed within each course. Another assessment project may be related to how a department prepared for a particular accreditation review. This might include evaluation portfolios, surveys, and other components. In each of these cases, other institutions may benefit 15 from copying those assessment plans. Thus, portions of an assessment project (or the entire project) may be copied within the institution for use by various programs and departments while such functionality may also occur across different institutions as well. Thus, the presently described assessment system 102 can export an assessment project or only portions of an assessment project as a data entity that can be 20 transferred to another institution having its own assessment system. One example of a portion of an assessment project that may be exported, or imported, may be a series of templates related to National Council for Accreditation of Teacher Education (NCATE) accreditation. At that second institution, then, the assessment system can import the data entity and have a pre-populated assessment plan to use or to modify. At a smaller level, a 25 publisher of a course or program of courses can define the objectives met by its courses and their respective content as well as the instruments and, possibly, portfolios that might accompany such a program of courses. This information may be imported into the assessment system at a department or program level at the institution to avoid developing the same material on their own from scratch. 30 Within such imported or exported data there may be references to third party standards. If a user imports a project that includes a reference to, for example, an 47 WO 2006/110892 PCTIUS2006/014043 objective of the standard (e.g., analytical reasoning), that user will most likely desire to know exactly what that standard says about that objective. Thus, importing and exporting data within the assessment system 102 provides the additional functionality to include external standards. This functionality may also extend to exporting and importing 5 instruments related to measuring or evaluating these standards as well. One result of having such import and export functionality included within the present assessment system is that repositories of assessment project-related data entities may be created. Thus, assessment projects, templates, instruments and other data may be exported to and stored in a repository that is accessible by other users via a 10 network or other computer system. Such a repository of information may include different levels of anonymity. For example, an institution may want to export to the repository information about the incoming freshman class' SAT scores, high-school GPA, and student age that was collected during a particular assessment project. This information can then be used by other institutions for comparison and analysis. In such 15 an example, the first institution may want to have their identity associated with the data but also insure that the individual student information is anonymous. In a second example, the institution itself may want to remain anonymous in interacting with the repository. In this case, the institution may provide one or more aspects of an assessment project to the repository which can then provide a comparison against a benchmark or 20 other standard. In this way, the repository provides the institution with feedback regarding the assessment project (e.g., whether or not the students performance on a battery of tests were below, at, or above a nationwide average). Alternatively, the institution can access the repository and import different benchmarks and other standards to perform their own comparison and analysis. Because virtually any aspect of an 25 assessment project may be exported to the repository, institutions can exchange and compare various instruments, rubrics, surveys, templates, reports, as well as results collected in performing the assessment project. Furthermore, this repository may also include non-assessment related information and data as well such as, for example, course content and individual academic portfolios. 30 48 WO 2006/110892 PCT/US2006/014043 Multi-Institution Not only can the assessment plans and portions of the assessment plans (e.g., the objectives, the courses) be exported and shared, the results and findings can be shared as well. Accordingly, a multi-college consortium, for example, can request and 5 receive findings from its members and then report on the findings from the overall perspective. If the members of the consortium utilize substantially similar assessment projects and initiatives, then the collected data and findings will be consistent in nature and format and will allow them to be simply amalgamated and analyzed. A number of variations to the specific behaviors and steps described in the 10 above examples may be made without departing from the scope of the present invention. The various illustrative logical blocks, modules, circuits, elements, and/or components described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic 15 component, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The methods or algorithms described in connection with the embodiments 20 disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to the processor such that the processor can 25 read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The previous description is provided to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles 30 defined herein may be applied to other embodiments. Thus, the claims are not intended to be limited to the embodiments shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not 49 WO 2006/110892 PCTIUS2006/014043 intended to mean "one and only one" unless specifically so stated, but rather "one or more." All structural and functional equivalents to the elements of the various embodiments described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference 5 and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims. No claim element is to be construed under the provisions of 35 U.S.C. § 112, sixth paragraph, unless the element is expressly recited using the phrase "means for" or, in the case of a method claim, the element is recited using the phrase 10 "step for." 50

Claims

1. A method for performing assessment within a multi-level institution, 5 the method comprising the steps of: storing a model of a plurality of organizational levels of the institution, wherein each organizational level includes at least one respective, associated node; for any node, creating for and presenting to a user at least one 10 customizable assessment project associated with that node; and completing the at least one customizable assessment project based, at least in part, on first input received from the user.

2. The method of claim 1, wherein the at least one customizable 15 assessment project is based on an editable template.

3. The method of claim 2, further comprising the steps of: receiving second input from another user; and generating the editable template based on the received second 20 input.

4. The method of claim 3, wherein contents of the editable template are based on a particular organizational level associated with the node associated with the at least one customizable assessment project. 25

5. The method of claim 3, wherein the editable template includes at least one objective form.

6. The method of claim 6, wherein a set of activity forms is associated 30 with the at least one objective form. 51 WO 2006/110892 PCT/US2006/014043

7. The method of claim 6, wherein membership in the set of activity forms is based on the particular organizational level associated with the node associated with the at least one customizable assessment project. 5 8. The method of claim 6, wherein contents of each activity form within the set of activity forms are based on the particular organizational level associated with the node associated with the at least one customizable assessment project.

9. The method of claim 1, further comprising the step of: 10 presenting to the user a list of one or more available editable templates; and receiving second input from the user selecting one of the available editable templates. 15 10. The method of claim 9, wherein the at least one customizable assessment project is based on the selected one editable template.

11. The method of claim 10, wherein the at least one customizable assessment project includes at least one objective, said at least one objective being based, 20 at least in part, on the input received from the user.

13. The method of claim 11, wherein the at least one customizable assessment project includes one or more activities related to the at least one objective, said one or more activities being based, at least in part, on the input received from the 25 user.

14. The method of claim 13, wherein the selected editable template includes a respective set of activity forms associated with each of the respective objective forms. 30

15. The method of claim 11, wherein the editable template includes at least one respective objective form corresponding to each of the at least one objective related to the at least one customizable assessment project. 52 WO 2006/110892 PCTIUS2006/014043

16. The method of claim 15, further comprising the step of: creating each of the at least one objective within its corresponding objective form. 5 17. The method of claim 15, further comprising the step of: selecting for any of the at least one objective form its corresponding objective from among a plurality of available, existing objectives.

18. The method of claim 1, wherein the at least one customizable 10 assessment project comprises a plurality of different customizable assessment projects for that node concurrently available to the user.

19. The method of claim 1, wherein the institution comprises at least three organizational levels including multiple levels of operating units. 15

20. The method of claim 19, wherein the at least three organizational levels further comprise a programs level, a course level, and a section level.

21. The method of claim 1, further comprising the steps of: 20 receiving other input from another user identifying one or more respective attributes associated with each of a plurality of nodes to be modeled; and creating a respective, editable template for each of the plurality of nodes, based on the other input. 25 22. The method of claim 21, wherein the one or more respective attributes for a particular node are based on one of an organizational level of that node, a type of operating unit of that node, a type of operating unit hierarchically above that node; and whether that particular node is a program, course or section. 30 23. The method of claim 21, further comprising the steps of: receiving additional input from a third user selecting one of the plurality of nodes; 53 WO 2006/110892 PCT/US2006/014043 presenting the user with the respective, editable template for the selected one node; and defining a model element for the selected one node by completing the respective editable, template based, at least in part, on the additional input from the 5 third user.

24. A system for performing assessment within a multi-level institution, the system comprising: a computer-based model of a plurality of organizational levels of 10 the institution, wherein each organizational level includes at least one respective, associated node; and a programmable computer, having access to the computer-based model, and configured to: a) receive first input indicative of a particular node within the model; 15 b) present to a user at least one customizable assessment project associated with that particular node; and c) complete the at least one customizable assessment project based, at least in part, on second input received from the user. 20 25. The system of claim 24, wherein the at least one customizable assessment project is based on an editable template.

26. The system of claim 25, wherein the programmable computer is further configured to: 25 receive other input from another user; and generate the editable template based on the received other input.

27. The system of claim 26, wherein contents of the editable template are based on a particular organizational level associated with the node associated with the at 30 least one customizable assessment project.

28. The system of claim 26, wherein the another user and the user are one and the same. 54 WO 2006/110892 PCT/US2006/014043

29. The system of claim 26, wherein the editable template includes at least one objective form.

30. The system of claim 29, wherein a set of activity forms is associated 5 with the at least one objective form.

31. The system of claim 30, wherein membership in the set of activity forms is based on the particular organizational level associated with the node associated with the at least one customizable assessment project. 10

32. The system of claim 31, wherein contents of each activity form within the set of activity forms are based on the particular organizational level associated with the node associated with the at least one customizable assessment project. 15 33. The system of claim 24, wherein the programmable computer is further configured to: present to the user a list of one or more available editable templates; and receive responsive input from the user selecting one of the 20 available editable templates.

34. The system of claim 33, wherein the customizable assessment project is based on the selected one editable template. 25 35. The system of claim 34, wherein the at least one customizable assessment project includes at least one objective, said at least one objective being based, at least in part, on the second input received from the user.

36. The system of claim 35, wherein the at least one customizable 30 assessment project includes one or more activities related to the at least one objective, said one or more activities being based, at least in part, on the second input received from the user. 55 WO 2006/110892 PCT/US2006/014043

37. The system of claim 36, wherein the selected editable template includes a respective set of activity forms associated with each of the respective objective forms. 5 38. The system of claim 37, wherein the programmable computer is further configured to: present to the user the respective sets of activity forms.

39. The system of claim 38, wherein the programmable computer is 10 further configured to: modify contents of each activity form within the respective sets of activity forms, based, at least in part, on the second input of the user.

40. The system of claim 35, wherein the editable template includes at least 15 one respective objective form corresponding to each of the at least one objective related to the at least one customizable assessment project.

41. The system of claim 40, wherein the programmable computer is further configured to: 20 create each of the at least one objective within its corresponding objective form.

42. The system of claim 40, wherein the programmable computer is further configured to: 25 select for any of the at least one objective form its corresponding objective from among a plurality of available, existing objectives.

43. The system of claim 24, wherein the at least one customizable assessment project comprises a plurality of different customizable assessment projects for 30 that node concurrently available to the user.

44. The system of claim 24, wherein the institution comprises at least three organizational levels including multiple levels of operating units. 56 WO 2006/110892 PCT/US2006/014043

45. The system of claim 29, wherein the at least three organizational levels further comprise a programs level, a course level, and a section level.

46. The system of claim 24, wherein the programmable computer is 5 further configured to: receive other input from another user identifying one or more respective attributes associated with each of a plurality of nodes to be modeled; and create a respective, editable template for each of the plurality of nodes, based on the other input. 10

47. The system of claim 46, wherein the one or more respective attributes for a particular node are based on one of an organizational level of that node, a type of operating unit of that node, a type of operating unit hierarchically above that node; and whether that particular node is a program, course or section. 15

48. The system of claim 46, wherein the programmable computer is further configured to: receive additional input from a third user selecting one of the plurality of nodes; 20 present the user with the respective, editable template for the selected one node; and define a model element for the selected one node by completing the respective editable, template based, at least in part, on the additional input from the third user. 25

49. Computer readable media containing programming instructions for performing assessment within a multi-level institution that upon execution thereof, causes one or more processors to perform the steps of: storing a model of a plurality of organizational levels of the 30 institution, wherein each organizational level includes at least one respective, associated node; for any node, creating for and presenting to a user at least one customizable assessment project associated with that node; and 57 WO 20061110892 PCT/US2006/014043 completing the at least one customizable assessment project based, at least in part, on first input received from the user.

50. A method of modeling a plurality of organizational levels in support 5 of conducting an assessment within a multi-level institution, the method comprising the steps of: receiving first input from a user selecting a node associated with one of the plurality of organizational levels; presenting the user with an editable template for the node; and 10 modifying the node by completing at least a portion of the editable template based, at least in part, on second input received from the user.

51. The method of claim 50, further comprising the steps of: for the node, creating for and presenting to another user at least one 15 customizable assessment project associated with the node; and completing the at least one customizable assessment project based, at least in part, on other input received from the another user.

52. The method of claim 51, wherein the user and the another user are the 20 same.

53. The method of claim 50, further comprising the step of: storing the modified node along with any previously defined nodes to create a model of the multi-level institution. 25

54. The method of claim 50, wherein the multi-level institution comprises at least three organizational levels including multiple operating units.

55. The method of claim 54, wherein the at least three organizational 30 levels further comprise a programs level, a course level, and a section level. 58 WO 2006/110892 PCT/US2006/014043

56. The method of claim 50, further comprising the step of: selecting the editable template from a plurality of editable templates. 5 57. The method of claim 56, wherein the editable template is selected based on one or more criteria, said criteria including: the one of the plurality of organizational levels associated with the node; a type of operating unit associated with the node; an identity of the node; a type of operating unit hierarchically above the node; whether the node is a program, a course, or section, and an identity of an operating unit 10 hierarchically above the node.

58. The method of claim 56, wherein respective contents of each of the plurality of editable templates are based on at least one of: its associated organization level; 15 a type of operating unit associated with the node; an identity of the node; a type of operating unit hierarchically above the node; an identity of an operating unit hierarchically above the node; and whether the node is a program, a course, or section. 20

59. The method of claim 50, wherein each of the organization levels have associated therewith, respectively, at least one node.

60. A system for modeling a plurality of organizational levels in support 25 of conducting an assessment within a multi-level institution, the system comprising: a computer-based storage including a plurality of editable templates; and a programmable computer, having access to the computer-based storage, and configured to: receive first input from a user selecting a node associated with one 30 of the plurality of organizational levels; present the user with one of the plurality of editable templates for the node; and 59 WO 2006/110892 PCT/US2006/014043 modify the node by completing at least a portion of the editable template based, at least in part, on second input received from the user.

61. The system of claim 60, wherein the programmable computer is 5 further configured to: create and present to another user at least one customizable assessment project associated with the node; and complete the at least one customizable assessment project based, at least in part, on other input received from the another user. 10

62. The system of claim 61, wherein the user and the another user are the same.

63. The system of claim 60, wherein the programmable computer is 15 further configured to:: store the modified node along with any previously defined nodes to create a model of the multi-level institution.

64. The system of claim 60, wherein the multi-level institution comprises 20 at least three organizational levels including multiple operating units.

65. The system of claim 65, wherein the at least three organizational levels further comprise a programs level, a course level, and a section level. 25 66. The system of claim 60, wherein the programmable computer is further configured to: select the one editable template from the plurality of editable templates. 30 67. The system of claim 66, wherein the programmable computer is further configured to select the one editable template based on one or more criteria, said criteria including: the one of the plurality of organizational levels associated with the node; a type of operating unit associated with the node; an identity of the node; a type of 60 WO 2006/110892 PCT/US2006/014043 operating unit hierarchically above the node; whether the node is a program, a course, or section, and an identity of an operating unit hierarchically above the node.

68. The system of claim 66, wherein respective contents of each of the 5 plurality of editable template are based on at least one of: its associated organization level; a type of operating unit associated with the node; an identity of the node; a type of operating unit hierarchically above the node; 10 an identity of an operating unit hierarchically above the node; and whether the node is a program, a course, or section.

69. The system of claim 60, wherein each of the organization levels have associated therewith, respectively, at least one node. 15

70. A method for creating customizable templates for modeling organizational levels related to an assessment within a multi-level institution, the method comprising the steps of: receiving first input from a user identifying one of the 20 organizational levels; receiving second input from the user identifying one or more attributes associated with each of a plurality of nodes associated with the one organizational level; and creating an editable template, based on the second input, to be used 25 to model one of the plurality of nodes of the assessment.

71. The method of claim 70, wherein the one or more attributes are based on the associated one organizational level. 30 61 WO 2006/110892 PCT/US2006/014043

72. The method of claim 70, wherein the one or more attributes are based on one of: the one organizational level; a type of operating unit associated with the one of the plurality of 5 nodes; an identity of the one of the plurality of nodes; a type of operating unit hierarchically above the one of the plurality of nodes; an identity of an operating unit hierarchically above the one of the 10 plurality of nodes; and whether the one of the plurality of nodes is a program, course, or section.

73. A system for creating customizable templates for modeling 15 organizational levels related to an assessment within a multi-level institution, the system comprising: a programmable computer configured to receive first input from a user identifying one of the organizational levels; 20 receive second input from the user identifying one or more attributes associated with each of a plurality of nodes associated with the one organizational level; and create an editable template, based on the second input, to be used to model one of the plurality of nodes when performing an assessment project. 25

74. A method of modeling a plurality of organizational levels in support of conducting an assessment within a multi-level institution, the method comprising the steps of: receiving first input from a user selecting a first node associated 30 with one of the plurality of organizational levels; presenting the user with an editable template for a second node; and 62 WO 2006/110892 PCT/US2006/014043 defining the second node by completing at least a portion of the editable template based, at least in part, on second input received from the user.

75. The method of claim 74, further comprising the step of: 5 storing the second node along with any previously defined nodes to create a model of the multi-level institution.

76. The method of claim 74, wherein the multi-level institution comprises at least three organizational levels including multiple operating units. 10

77. The method of claim 76, wherein the at least three organizational levels further comprise a programs level, a course level, and a section level.

78. The method of claim 74, further comprising the step of: 15 selecting the editable template from a plurality of editable templates.

79. A system for modeling a plurality of organizational levels in support of conducting an assessment within a multi-level institution, the system comprising: 20 a computer-based storage including a plurality of editable templates; and a programmable computer, having access to the computer-based storage, and configured to: receive first input from a user selecting a first node associated with one of the plurality of organizational levels; 25 present the user with one of the plurality of editable templates for a second node; and create the second node by completing at least a portion of the editable template based, at least in part, on second input received from the user. 30 80. The system of claim 79, wherein the programmable computer is further configured to: store the second node along with any previously defined nodes to create a model of the multi-level institution. 63 WO 2006/110892 PCT/US2006/014043

81. The system of claim 79, wherein the multi-level institution comprises at least three organizational levels including multiple operating units.

82. The system of claim 81, wherein the at least three organizational 5 levels further comprise a programs level, a course level, and a section level.

83. The system of claim 79, wherein the programmable computer is further configured to: select the one editable template from the plurality of editable 10 templates.

84. Computer readable media containing programming instructions for modeling a plurality of organizational levels in support of conducting an assessment within a multi-level institution, that upon execution thereof, causes one or more 15 processors to perform the steps of: receiving first input from a user selecting a node associated with one of the plurality of organizational levels; presenting the user with an editable template for the node; and modifying the node by completing at least a portion of the editable 20 template based, at least in part, on second input received from the user.

85. Computer readable media containing programming instructions for modeling a plurality of organizational levels in support of conducting an assessment within a multi-level institution, that upon execution thereof, causes one or more 25 processors to perform the steps of: receiving first input from a user selecting a first node associated with one of the plurality of organizational levels; presenting the user with an editable template for a second node; and 30 defining the second node by completing at least a portion of the editable template based, at least in part, on second input received from the user. 64 WO 2006/110892 PCT/US2006/014043

86. Computer readable media containing programming instructions for creating customizable templates for modeling organizational levels related to an assessment within a multi-level institution, that upon execution thereof, causes one or more processors to perform the steps of: s receiving first input from a user identifying one of the organizational levels; receiving second input from the user identifying one or more attributes associated with each of a plurality of nodes associated with the one organizational level; and 10 creating an editable template, based on the second input, to be used to model one of the plurality of nodes of the assessment.

87. A method for performing an assessment initiative within a multi-level institution, comprising the steps of: 15 receiving first input identifying a first assessment project; receiving second input identifying a second assessment project, the second assessment project being active concurrent with the first assessment project; and associating the first assessment project and the second assessment project with the assessment initiative. 20

88. The method of claim 87, further comprising the steps of: receiving initial input identifying the assessment initiative; and creating the assessment initiative. 25 89. The method of claim 87, further comprising the step of: storing a model of a plurality of organizational levels of the multi level institution, wherein each organizational level includes at least one respective, associated node. 30 90. The method of claim 89, wherein the first and second assessment projects are related to one of the at least one respective associated node. 65 WO 2006/110892 PCT/US2006/014043

91. The method of claim 89, wherein the first assessment project relates to a first node from among the at least one respective , associated node and the second assessment project relates to a second node from among the at least one respective, associated node; the first node and second node being different. 5

92. The method of claim 89, wherein the first assessment project and the second assessment project relate to one of the plurality of organizational levels.

93. The method of claim 89, wherein the first assessment project relates to 10 a first level of the plurality of organizational levels and the second assessment project relates to a second level of the plurality of organizational levels; the first level being different than the second level.

94. The method of claim 89, further comprising the steps of: 15 identifying a particular node associated with the first assessment project; creating for and presenting to the user at least one customizable assessment project template associated with that node; and completing the at least one customizable assessment project 20 template based, at least in part, on third input received from the user.

95. The method of claim 89, further comprising the steps of: identifying a particular node associated with the second assessment project; 25 creating for and presenting to the user at least one customizable assessment project template associated with that node; and completing the at least one customizable assessment project template based, at least in part, on third input received from the user. 30 96. The method of claim 87, further comprising the steps of: collecting first information related to performance of the first assessment project; 66 WO 2006/110892 PCTIUS2006/014043 collecting second information related to performance of the second assessment project; and combining a portion of the first information and a portion of the second information in a report related to the assessment initiative. 5

97. The method of claim 87, wherein the first assessment project relates to a first multi-level institution and the second assessment project relates to a second multi level institution. 10 98. The method of claim 87, wherein data collected as part of the first assessment project is shared with the second assessment project.

99. The method of claim 87, wherein data collected as part of the second assessment project is shared with the first assessment project. 15

100. The method of claim 87, further comprising: receiving other input identifying at least one other assessment project; and associating the at least one other assessment project with the 20 assessment initiative.

101. A system for performing an assessment initiative within a multi-level institution, comprising: a programmable computer configured to: 25 receive first input identifying a first assessment project; receive second input identifying a second assessment project, the second assessment project being active concurrent with the first assessment project; and associate the first assessment project and the second 30 assessment project with the assessment initiative. 67 WO 2006/110892 PCT/US2006/014043

102. The system of claim 101, further comprising: memory accessible by the programmable computer and storing a model of a plurality of organizational levels of the multi-level institution, wherein each organizational level includes at least one respective, associated node. 5

103. The system of claim 102, wherein the first and second assessment projects are related to one of the at least one respective associated node.

104. The system of claim 102, wherein the first assessment project relates 10 to a first node from among the at least one respective , associated node and the second assessment project relates to a second node from among the at least one respective, associated node; the first node and second node being different.

105. The system of claim 102, wherein the first assessment project and the 15 second assessment project relate to one of the plurality of organizational levels.

106. The system of claim 102, wherein the first assessment project relates to a first level of the plurality of organizational levels and the second assessment project relates to a second level of the plurality of organizational levels; the first level being 20 different than the second level.

107. The system of claim 102, wherein the programmable computer is further configured to: identify a particular node associated with the first assessment 25 project; present to the user at least one customizable assessment project template associated with that node; and complete the at least one customizable assessment project template based, at least in part, on third input received from the user. 30 68 WO 2006/110892 PCT/US2006/014043

108. The system of claim 102, wherein the programmable computer is further configured to: identify a particular node associated with the second assessment project; 5 present to the user at least one customizable assessment project template associated with that node; and complete the at least one customizable assessment project template based, at least in part, on third input received from the user. 10 109. The system of claim 101, wherein the programmable computer is further configured to: collect first information related to performance of the first assessment project; collect second information related to performance of the second 15 assessment project; and combine a portion of the first information and a portion of the second information in a report related to the assessment initiative.

110. The system of claim 101, wherein the first assessment project relates 20 to a first multi-level institution and the second assessment project relates to a second multi-level institution.

111. The system of claim 101, wherein data collected as part of the first assessment project is shared with the second assessment project. 25

112. The system of claim 101, wherein data collected as part of the second assessment project is shared with the first assessment project.

113. The system of claim 101, wherein the programmable computer is 30 further configured to: receive other input identifying at least one other assessment project; and 69 WO 2006/110892 PCT/US2006/014043 associate the at least one other assessment project with the assessment initiative.

114. A computer-based assessment system for a multi-level institution 5 comprising: a first assessment project stored in computer-accessible memory; a second assessment project concurrently stored in the computer accessible memory; and an assessment initiative associated with both the first and second 10 assessment projects stored in the computer-accessible memory.

115. The system of claim 114 wherein the first assessment project and the second assessment project reference at least a portion of common data within the computer-accessible memory. 15

116. A method for creating an assessment initiative within a multi-level institution, comprising the steps of: receiving first input identifying the assessment initiative; creating the assessment initiative; 20 creating a plurality of assessment projects; and associating the plurality of assessment projects with the assessment initiative.

117. The method of claim 116, wherein the step of creating a plurality of 25 assessment projects further includes the step of: receiving identification of at least some of the plurality of assessment projects from a user.

118. The method of claim 116, wherein the step of creating a plurality of 30 assessment projects further includes the steps of: automatically generating at least some of the plurality of assessment projects based on the assessment initiative. 70 WO 2006/110892 PCT/US2006/014043

119. A system for creating an assessment initiative within a multi-level institution, comprising: a programmable computer configured to: receive first input identifying the assessment initiative; 5 create the assessment initiative; create a plurality of assessment projects; and associate the plurality of assessment projects with the assessment initiative. 10 120. The system of claim 119, wherein the programmable computer is further configured to: receive identification of at least some of the plurality of assessment projects from the user. 15 121. The system of claim 119, wherein the programmable computer is further configured to: automatically generate at least some of the plurality of assessment projects based on the assessment initiative. 20 122. Computer readable media containing programming instructions for performing an assessment initiative within a multi-level institution, that upon execution thereof, causes one or more processors to perform the steps of: receiving first input identifying a first assessment project; receiving second input identifying a second assessment project, the 25 second assessment project being active concurrent with the first assessment project; and associating the first assessment project and the second assessment project with the assessment initiative.

123. Computer readable media containing programming instructions for 30 creating an assessment initiative within a multi-level institution, that upon execution thereof, causes one or more processors to perform the steps of: receiving first input identifying the assessment initiative; creating the assessment initiative; 71 WO 2006/110892 PCT/US2006/014043 creating a plurality of assessment projects; and associating the plurality of assessment projects with the assessment initiative. 5 124. A method for performing curriculum planning involving one or more organizational levels within a multi-level institution, the method comprising the steps of: receiving first goals relating to a first organization level of the institution; receiving second goals relating to a second organizational level of 10 the institution; determining a first respective alignment that indicates which, if any, of a plurality of courses are aligned to at least one of the first and second goals; and generating a mapping of the first respective alignment. 15 125. The method of claim 124, further comprising the step of: graphically presenting the mapping to a user.

126. The method of claim 125, wherein the mapping is graphically presented concurrently with one or more other related mappings. 20

127. The method of claim 126, wherein each of the one or more other related mappings is associated with another respective alignment that indicates which, if any, of the plurality of courses are aligned to one or more goals other than the at least one of the first and second goals. 25

128. The method of claim 126, wherein each of the one or more other related mappings is associated with another respective alignment that indicates which, if any, of the first or second goals are aligned to one or more courses other than the plurality of courses. 30 72 WO 2006/110892 PCT/US2006/014043

129. The method of claim 124, further comprising the step of: determining a second respective alignment that indicates which, if any, of a plurality of educational experiences are aligned to at least one of the first and second goals; and 5 generating a mapping of the second respective alignment.

130. The method of claim 124, wherein a particular course is aligned with a particular goal if the particular course includes at least some material designed to satisfy the particular goal. 10

131. The method of claim 124, wherein the first organizational level is one of an operating unit, a department, a college, a program, and an institution.

132. The method of claim 124, wherein the second organizational level is 15 one of an operating unit, a department, a college, a program, and an institution.

133. The method of claim 124, further comprising the step of: determining a second respective alignment that indicates which, if any of the second goals are aligned to at least one of the first goals. 20

134. The method of claim 133, further comprising the step of: determining if a particular course is aligned with a particular first goal based on the respective first alignment of that particular course with a particular second goal and the respective second alignment of that particular second goal with the 25 particular first goal.

135. The method of claim 124, further comprising the step of: receiving user input related to a level of alignment, on a multi-level scale, between a particular course and a particular goal. 30

136. The method of claim 135, wherein the scale reflects a qualitative measurement of alignment. 73 WO 2006/110892 PCT/US2006/014043

137. The method of claim 135, wherein the scale is customizable.

138. The method of claim 124, wherein the first organizational level includes an operating unit of the institution and the second organizational level includes a 5 program.

139. The method of claim 138, wherein the mapping includes one or both of: a) a first mapping of which of the plurality of courses are aligned to with a particular goal of the operating unit; and b) a second mapping of which of the plurality of courses are aligned with a particular goal of the program.

140. The method of claim 124, further comprising the step of: 15 based on the mapping, identifying a curriculum gap.

141. The method of claim 140 wherein the curriculum map comprises a subset of the first and second goals which have fewer of the plurality of courses aligned therewith, as compared to others of the first and second goals. 20

142. The method of claim 141, further comprising the step of: identifying additional material based on the subset of first and second goals. 25 143. The method of claim 141, further comprising the step of: identifying one of the plurality of courses to associate with the additional material to allow the one course to be aligned with at least a portion of the subset. 30 144. The method of claim 141, further comprising the step of: identifying a future course to associate with the additional material to allow the future course to be aligned with at least a portion of the subset. 74 WO 2006/110892 PCTIUS2006/014043

145. The method of claim 124, further comprising the steps of: receiving input modifying at least one of the first goals, the second goals, and the plurality of courses; determining a second respective alignment that indicates which, if 5 any, of a plurality of courses are aligned to at least one of the first and second goals, taking in consideration the input; and generating a mapping of the second respective alignment.

146. The method of claim 145, further comprising the steps of: 10 determining a difference between the mapping of the first respective alignment and the mapping of the second alignment; and providing output indicative of the difference.

147. The method of claim 124, further comprising the step of: 15 determining a second respective alignment that indicates which, if any, of a plurality of lesson plans are aligned to at least one of the first and second goals; and generating a mapping of the second respective alignment. 20 148. A system for performing curriculum planning involving multiple organizational levels within an institution, comprising: a programmable computer configured to: receive first goals relating to a first organization level of the institution; 25 receive second goals relating to a second organizational level of the institution; determine a first respective alignment that indicates which, if any, of a plurality of courses are aligned to at least one of the first and second goals; and generate a mapping of the first respective alignment. 30

149. The system of claim 148, wherein the programmable computer is further configured to display the mapping to a user. 75 WO 2006/110892 PCT/US2006/014043

150. The system of claim 149, wherein the mapping is displayed concurrently with one or more other related mappings.

151. The system of claim 150, wherein each of the one or more other 5 related mappings is associated with another respective alignment that indicates which, if any, of the plurality of courses are aligned to one or more goals other than the at least one of the first and second goals.

152. The system of claim 148, wherein the programmable computer is 10 further configured to: transmit the mapping to a remotely located computer system.

153. The system of claim 148, wherein the programmable computer is further configured to: 15 determine a second respective alignment that indicates which, if any, of a plurality of educational experiences are aligned to at least one of the first and second goals; and generate a mapping of the second respective alignment. 20 154. The system of claim 148, wherein a particular course is aligned with a particular goal if the particular course includes at least some material designed to satisfy the particular goal.

155. The system of claim 148, wherein the programmable computer is 25 further configured to: determine a second respective alignment that indicates which, if any of the second goals are aligned to at least one of the first goals.

156. The system of claim 155, wherein the programmable computer is 30 further configured to: determine if a particular course is aligned with a particular first goal based on the respective first alignment of that particular course with a particular 76 WO 2006/110892 PCTIUS2006/014043 second goal and the respective second alignment of that particular second goal with the particular first goal.

157. The system of claim 156, wherein the programmable computer is 5 further configured to: receive user input related to a level of alignment, on a multi-level scale, between a particular course and a particular goal.

158. The system of claim 157, wherein the scale reflects a qualitative 10 measurement of alignment.

159. The system of claim 157, wherein the scale is customizable.

160. The system of claim 148, wherein the first organizational level 15 includes an operating unit of the institution and the second organizational level includes a program.

161. The system of claim 160, wherein the mapping includes one or both of: 20 a) a first mapping of which of the plurality of courses are aligned with a particular goal of the operating unit; and b) a second mapping of which of the plurality of courses are aligned with a particular goal of the program. 25 162. The system of claim 148, wherein the programmable computer is further configured to: identify a curriculum gap based on the mapping.

163. The system of claim 162, wherein the curriculum gap comprises a 30 subset of the first and second goals which have fewer of the plurality of courses aligned therewith, as compared to others of the first and second goals. 77 WO 2006/110892 PCT/US2006/014043

164. The system of claim 163, wherein the programmable computer is further configured to: identify additional material based on the subset of first and second goals. 5

165. The method of claim 164, wherein the programmable computer is further configured to:: identify one of the plurality of courses to associate with the additional material to allow the one course to be aligned with at least a portion of the 10 subset.

166. The system of claim 164, wherein the programmable computer is further configured to: identify a future course to associate with the additional material to 15 allow the future course to be aligned with at least a portion of the subset.

167. The system of claim 148, the programmable computer further configured to: receive input modifying at least one of the first goals, the second 20 goals, and the plurality of courses; determine a second respective alignment that indicates which, if any, of a plurality of courses are aligned to at least one of the first and second goals, taking in consideration the input; and generate a mapping of the second respective alignment. 25

168. The system of claim 167, wherein the programmable computer is further configured to: determine a difference between the mapping of the first respective alignment and the mapping of the second alignment; and 30 provide output indicative of the difference.

169. The system of claim 148, wherein at least some of the plurality of courses are located outside of the second organizational level of the institution. 78 WO 2006/110892 PCT/US2006/014043

170. The system of claim 148, wherein the programmable computer is further configured to: determine a second respective alignment that indicates which, if any, of a plurality of lesson plans are aligned to at least one of the first and second goals; 5 and generate a mapping of the second respective alignment.

171. The system of claim 170, wherein the second level of the organization includes one of a course or a course section. 10

172. The system of claim 150, wherein each of the one or more other related mappings is associated with another respective alignment that indicates which, if any, of the first or second goals are aligned to one or more courses other than the plurality of courses. 15

173. Computer readable media containing programming instructions for performing curriculum planning involving one or more organizational levels within a multi-level institution, that upon execution thereof, causes one or more processors to perform the steps of: 20 receiving first goals relating to a first organization level of the institution; receiving second goals relating to a second organizational level of the institution; determining a first respective alignment that indicates which, if 25 any, of a plurality of courses are aligned to at least one of the first and second goals; and generating a mapping of the first respective alignment.

174. A method for selectively deploying an assessment instrument to a selected subset of a plurality of recipients, the method comprising the steps of: 30 receiving first input defining the assessment instrument; receiving second input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of recipients; 79 WO 2006/110892 PCT/US2006/014043 identifying the selected subset based on those of the plurality of recipients having associated identifying attributes matching the criteria; and deploying the assessment instrument to the selected subset. 5 175. The method of claim 174, wherein the criteria relates to demographic attributes about the plurality of recipients.

176. The method of claim 174, wherein the criteria relates to course related attributes associated with one or more courses associated with the recipients. 10 177. The method of claim 174, wherein the criteria relates to section related attributes associated with sections associated with the recipients.

178. The method of claim 174, wherein the criteria relates to educational experience-related attributes associated with one or more educational experiences 15 associated with the recipients.

179. The method of claim 174, wherein the criteria relates to program related attributes associated with one or more programs associated with the recipients. 20 180. The method of claim 174, wherein the criteria relates to operating unit-related attributes associated with one or more units with which the plurality of recipients may be associated.

181. The method of claim 180, wherein an operating unit comprises one 25 of an institution, a college, a department, a program, and a course.

182. The method of claim 174, wherein the assessment instrument is one of a test, a survey, a course evaluation, and an evaluation portfolio. 30 183. The method of claim 174, wherein the criteria relates to a combination of at least two of demographic attributes, program-related attributes, course related attributes, educational experience related attributes, section-related attributes, and operating unit-related attributes. 80 WO 2006/110892 PCT/US2006/014043

184. The method of claim 174, wherein the step of deploying further comprises: forwarding the assessment instrument to a separate computer-based system configured to disseminate information to the plurality of recipients. 5

185. The method of claim 184, wherein the separate computer-based system comprises a campus academic system.

186. A method for selectively deploying an assessment instrument to a 10 selected subset of a plurality of recipients of an institution, the method comprising the steps of: receiving first input defining the assessment instrument; receiving second input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of 15 recipients; and forwarding the criteria and the assessment instrument to a separate computer-based system of the institution, wherein the separate computer-based system is configured to distribute the assessment instrument to those of the plurality of recipients having associated identifying attributes matching the criteria. 20

187. The method of claim 186, wherein the separate computer-based system comprises a campus academic system.

188. A method for selectively deploying an assessment instrument to a 25 selected subset of a plurality of recipients of an institution, the method comprising the steps of: receiving first input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of recipients; 30 forwarding the first input to a first separate computer-based system of the institution; 81 WO 2006/110892 PCT/US2006/014043 in response, receiving identification of members of the selected subset; and deploying the assessment instrument to the selected subset. 5 189. The method of claim 188, wherein the criteria relates to a combination of at least two of demographic attributes, program-related attributes, course related attributes, section-related attributes, educational experience-related attributes; and operating unit-related attributes. 10 190. The method of claim 188, wherein the step of deploying further comprises: forwarding the assessment instrument to a second separate computer-based system configured to disseminate information to the plurality of recipients. 15

191. The method of claim 190, wherein the second separate computer based system comprises a campus academic system.

192. The method of claim 188, further comprising the step of: 20 receiving second input defining the assessment instrument.

193. The method of claim 188, wherein the first separate computer-based system comprises a student information system of the institution. 25 194. The method of claim 188, further comprising the step of: storing the selected subset as a particular recipient list.

195. The method of claim 194, further comprising the steps of: receiving second input identifying a subsequent assessment 30 instrument; receiving third input identifying a stored recipient list; and deploying the subsequent assessment instrument to members of the stored recipient list. 82 WO 2006/110892 PCTIUS2006/014043

196. The method of claim 188, further comprising the step of: storing the criteria as a particular criteria list;

197. The method of claim 196, further comprising the steps of: 5 receiving second input identifying a subsequent assessment instrument; receiving third input identifying a stored criteria list; determining a new selected subset based on the stored criteria list; and 10 deploying the subsequent assessment instrument to essentially only members of the new selected subset.

198. The method of claim 197, wherein the step of determining further comprises the step of: 15 identifying the new selected subset based on those of a different plurality of recipients having associated identifying attributes matching the stored criteria list, wherein at least one member of the different plurality of recipients is not a member of the plurality of recipients. 20 199. The method of claim 197, wherein the step of determining further comprises the step of: identifying the new selected subset based on those of a different plurality of recipients having associated identifying attributes matching the stored criteria list, wherein at least one member of the plurality of recipients is not a member of the 25 different plurality of recipients.

200. A system for selectively deploying an assessment instrument to a selected subset of a plurality of recipients comprising: a programmable computer configured to: 30 receive first input defining the assessment instrument; receive second input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of recipients; 83 WO 2006/110892 PCTfUS2006/014043 identify the selected subset based on those of the plurality of recipients having associated identifying attributes matching the criteria; and deploy the assessment instrument to the selected subset.

201. The system of claim 200, wherein the criteria relates to a 5 combination of at least two of demographic attributes, program-related attributes, course related attributes, educational experience related attributes, section attributes, and operating unit-related attributes.

202. The system of claim 201, wherein the programmable computer is 10 further configured to: forward the assessment instrument to a separate computer-based system configured to disseminate information to the plurality of recipients.

203. The system of claim 202, wherein the separate computer-based 15 system comprises a campus academic system.

204. The system of claim 203, wherein the campus academic system is configured to provide an environment to disseminate course-related information to a plurality of users of the institution. 20

205. A system for selectively deploying an assessment instrument to a selected subset of a plurality of recipients of an institution, comprising: a programmable computer configured to: receive first input defining the assessment instrument; 25 receive second input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of recipients; and forward the criteria and the assessment instrument to a separate computer-based system of the institution, wherein the separate computer-based system is 30 configured to distribute the assessment instrument to those of the plurality of recipients having associated identifying attributes matching the criteria. 84 WO 2006/110892 PCT/US2006/014043

206. The system of claim 205, wherein the separate computer-based system comprises a campus academic system.

207. A system for selectively deploying an assessment instrument to a 5 selected subset of a plurality of recipients of an institution, comprising: a programmable computer configured to: receive first input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of recipients; 10 forward the first input to a first separate computer-based system of the institution; in response, receive identification of members of the selected subset; and deploy the assessment instrument to the selected subset. 15

208. The system of claim 207, wherein the criteria relates to a combination of at least two of demographic attributes, course-related attributes, section related attributes, educational experience-related attributes; and operating unit-related attributes. 20

209. The system of claim 207, wherein the programmable computer is further configured to: forward the assessment instrument to a second separate computer based system configured to disseminate information to the plurality of recipients. 25

210. The system of claim 209, wherein the second separate computer based system comprises a campus academic system.

211. The system of claim 210, wherein the campus academic system is 30 configured to provide an environment to disseminate course-related information to a plurality of users of the institution. 85 WO 2006/110892 PCT/US2006/014043

212. The system of claim 207, wherein the first separate computer-based system comprises a student information system of the institution.

213. The system of claim 207, wherein the programmable computer is 5 further configured to: store the selected subset as a particular recipient list.

214. The system of claim 213, wherein the programmable computer is further configured to: 10 receive second input identifying a subsequent assessment instrument; receive third input identifying a stored recipient list; and deploy the subsequent assessment instrument to essentially only members of the stored recipient list. 15

215. The system of claim 207, wherein the programmable computer is further configured to: store the criteria as a particular criteria list; 20

216. The system of claim 215, wherein the programmable computer is further configured to: receive second input identifying a subsequent assessment instrument; receive third input identifying a stored criteria list; 25 determine a new selected subset based on the stored criteria list; and deploy the subsequent assessment instrument to essentially only members of the new selected subset. 30

217. The system of claim 215, wherein the programmable computer is further configured to: identify the new selected subset based on those of a different plurality of recipients having associated identifying attributes matching the stored criteria 86 WO 2006/110892 PCT/US2006/014043 list, wherein at least one member of the different plurality of recipients is not a member of the plurality of recipients.

218. The system of claim 215, wherein the programmable computer is 5 further configured to: identify the new selected subset based on those of a different plurality of recipients having associated identifying attributes matching the stored criteria list, wherein at least one member of the plurality of recipients is not a member of the different plurality of recipients. 10

219. Computer readable media containing programming instructions for selectively deploying an assessment instrument to a selected subset of a plurality of recipients, that upon execution thereof, causes one or more processors to perform the steps of: 15 receiving first input defining the assessment instrument; receiving second input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of recipients; identifying the selected subset based on those of the plurality of 20 recipients having associated identifying attributes matching the criteria; and deploying the assessment instrument to the selected subset.

220. Computer readable media containing programming instructions for selectively deploying an assessment instrument to a selected subset of a plurality of 25 recipients, that upon execution thereof, causes one or more processors to perform the steps of: receiving first input defining the assessment instrument; receiving second input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of 30 recipients; and forwarding the criteria and the assessment instrument to a separate computer-based system of the institution, wherein the separate computer-based system is 87 WO 2006/110892 PCTIUS2006/014043 configured to distribute the assessment instrument to those of the plurality of recipients having associated identifying attributes matching the criteria.

221. Computer readable media containing programming instructions for 5 selectively deploying an assessment instrument to a selected subset of a plurality of recipients, that upon execution thereof, causes one or more processors to perform the steps of: receiving first input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of recipients; 10 forwarding the first input to a first separate computer-based system of the institution; in response, receiving identification of members of the selected subset; and deploying the assessment instrument to the selected subset. 15

222. A method for sharing data related to an assessment project of a multi-level institution, the method comprising the steps of: developing at least a portion of the assessment project, based on input received from one or more users, within a first instantiation of a first assessment 20 management system; and exporting the portion of the assessment project in a format importable by a second assessment management system.

223. The method of claim 222, wherein the first and second assessment 25 management system are the same.

224. The method of claim 222, wherein the first assessment management system is a first instantiation of a first computer-based assessment system and the second assessment management system is a second instantiation of a second computer-based 30 assessment system, wherein the first and second computer-based assessment systems are substantially similar. 88 WO 2006/110892 PCT/US2006/014043

225. The method of claim 222, wherein the portion of the assessment project includes a third-party standard.

226. The method of claim 222, wherein the portion of the assessment 5 project includes a hierarchically arranged set of assessment objectives.

227. The method of claim 222, wherein the portion of the assessment project includes one or more customizable templates. 10 228. The method of claim 227, wherein the portion of the assessment project includes results data associated with the customizable templates.

229. The method of claim 222, wherein the portion of the assessment project includes one or more program objectives and a set of courses aligned with those 15 program objectives.

230. The method according to claim 222, comprising the step of: offering for purchase the at least a portion of the assessment project. 20 231. The method of claim 222, wherein the institution comprises at least three organizational levels including multiple levels of operating units.

232. The method of claim 231, wherein the at least three organizational levels further comprise a programs level, a course level, and a section level. 25

233. The method of claim 222, wherein the assessment project includes at least one customizable assessment project associated with a node corresponding to one of the levels of the institution. 30 234. The method of claim 233, wherein the at least one customizable assessment project is based on an editable template. 89 WO 2006/110892 PCT/US2006/014043

235. The method of claim 234, wherein contents of the editable template are based on a particular organizational level associated with the node associated with the at least one customizable assessment project. 5 236. A method for sharing data related to an assessment project of a multi-level institution, the method comprising the steps of: importing at least a first portion of a first assessment project into a first instantiation of a first assessment management system, the first portion developed within a second instantiation of a second assessment management system; and 10 populating a second assessment project, within the first instantiation, using the imported first portion.

237. The method according to claim 236, comprising the step of: purchasing the first portion of the assessment project. 15

238. The method of claim 236, wherein the first assessment project includes at least one customizable assessment project associated with a node corresponding to one of the levels of the institution. 20 239. The method of claim 236, wherein the at least one customizable assessment project is based on an editable template.

240. The method of claim 236, wherein the first portion includes at least one of a rubric, an instrument, an objective, a goal, a department-level assessment project, 25 an institution-level assessment project, an assessment initiative, an operating unit-level assessment project, a college-level assessment project, a curriculum plan, a program-level assessment project, and an evaluation portfolio.

241. A system for sharing data related to an assessment project of a multi 30 level institution, comprising: a programmable computer configured to: 90 WO 2006/110892 PCT/US2006/014043 develop at least a portion of the assessment project, based on input received from one or more users, within a first instantiation of a first assessment management system; and export the portion of the assessment project in a format 5 importable by a second assessment management system.

242. The system of claim 241, wherein the first and second assessment management system are the same. 10 243. The system of claim 241, wherein the first assessment management system is a first instantiation of a first computer-based assessment system and the second assessment management system is a second instantiation of a second computer-based assessment system, wherein the first and second computer-based assessment systems are substantially similar. 15

244. The system of claim 241, wherein the assessment project includes at least one customizable assessment project associated with a node corresponding to one of the levels of the institution. 20 245. The system of claim 241, wherein the at least one customizable assessment project is based on an editable template.

246. The system of claim 241, wherein the institution comprises at least three organizational levels including multiple levels of operating units. 25

247. A system for sharing data related to an assessment project of a multi level institution, comprising: a programmable computer configured to: import at least a first portion of a first assessment project into a 30 first instantiation of a first assessment management system, the first portion developed within a second instantiation of a second assessment management system; and populate a second assessment project, within the first instantiation, using the imported first portion. 91 WO 2006/110892 PCT/US2006/014043

248. The system of claim 247, wherein the first portion includes at least one of a rubric, an instrument, an objective, a goal, a department-level assessment project, an institution-level assessment project, an assessment initiative, an operating unit-level assessment project, a college-level assessment project, a curriculum plan, a program-level 5 assessment project, and an evaluation portfolio.

249. Computer readable media containing programming instructions for sharing data related to an assessment project of a multi-level institution, that upon execution thereof, causes one or more processors to perform the steps of: 10 developing at least a portion of the assessment project, based on input received from one or more users, within a first instantiation of a first assessment management system; and exporting the portion of the assessment project in a format importable by a second assessment management system. 15

250. Computer readable media containing programming instructions for sharing data related to an assessment project of a multi-level institution, that upon execution thereof, causes one or more processors to perform the steps of: importing at least a first portion of a first assessment project into a 20 first instantiation of a first assessment management system, the first portion developed within a second instantiation of a second assessment management system; and populating a second assessment project, within the first instantiation, using the imported first portion. 25 251. A method for managing assessment projects, comprising the steps of: receiving respective exported data from each of a plurality of different assessment management systems, the respective exported data comprising at least a portion of a respective assessment project; storing the respective exported data; and 30 forwarding one or more of the respective exported data to a requesting assessment management system. 92 WO 2006/110892 PCT/US2006/014043

252. A system for managing assessment projects comprising: a programmable computer configured to attach to a network and receive, via the network, respective exported data from each of a plurality of different assessment management systems, the respective exported data comprising at least a 5 portion of a respective assessment project; memory accessible by the computer and configured to store the respective exported data; and the programmable computer further configured to forward, via the network, one or more of the respective exported data to a requesting assessment 10 management system

253. A method for selectively assembling an assignment binder including a selected subset of a plurality of artifacts, the method comprising the steps of: receiving first input defining a criteria, the criteria related to a 15 plurality of identifying attributes associated, respectively, with each of the plurality of artifacts; identifying the selected subset based on those of the plurality of artifacts having associated identifying attributes matching the criteria, wherein the criteria relates to at least two criterion related to one or more of demographic attributes, course 20 related attributes, operating unit-related attributes, section-related attributes, goal-related attributes, and educational experience-related attributes; and collecting the selected subset of artifacts within the assignment binder. 25 254. The method of claim 253, wherein the criteria relates to demographic attributes about an owner of each of the plurality of artifacts.

255. The method of claim 253, wherein the criteria relates to course related attributes associated with courses associated with the plurality of artifacts. 30

256. The method of claim 253, wherein the criteria relates to operating unit-related attributes associated with one or more units with which the plurality of artifacts may be associated. 93 WO 2006/110892 PCTIUS2006/014043

257. The method of claim 256, wherein an operating unit comprises one of an institution, an educational experience, a section, a college, a department, a program, and a course. 5 258. The method of claim 253, wherein the criteria relates to one or more respective goal-related attributes related to each of the plurality of artifacts.

259. The method of claim 258, wherein the one or more goal-related attributes include at least one of a course goal, a course objective, a program goal, a 10 program objective, a department goal, a department objective, an operating unit goal, an operating unit objective, an external standard; and a third-party accreditation requirement.

260. The method of claim 253, further comprising the step of: sampling the selected subset substantially randomly as part of a 15 determination of which artifacts in the selected subset to collect.

261. The method of claim 253, wherein each of the plurality of artifacts is a response to a respective instrument deployed to a plurality of users. 20 262. The method of claim 261, further comprising the steps of: forwarding the respective instruments to a separate computer-based system configured to disseminate information to the plurality of users; and receiving the plurality of artifacts from the separate computer based system. 25

263. The method of claim 262, wherein the separate computer-based system comprises a campus academic system.

264. The method of claim 253, further comprising the step of: 30 collecting evaluation data associated with each artifact within the selected subset. 94 WO 2006/110892 PCT/US2006/014043

265. The method of claim 253, further comprising the step of: storing the criteria as a named search, wherein another assignment binder can be subsequently assembled using the named search without the criteria being input again. 5

266. A method for selectively assembling an assignment binder, for a multi-level institution, including a selected subset of a plurality of artifacts, the method comprising the steps of: receiving first input defining a criteria, the criteria related to a 10 plurality of identifying attributes associated, respectively, with each of the plurality of artifacts; forwarding the criteria to a separate computer-based system of the institution, wherein the separate computer-based system is configured to store the plurality of artifacts; and 15 receiving from the separate computer-based system the selected subset of those of the plurality of artifacts having associated identifying attributes matching the criteria.

267. The method of claim 266, further comprising the step of: 20 collecting the selected subset of artifacts within the assignment binder.

268. The method of claim 267, wherein the separate computer-based system comprises a campus academic system. 25

269. The method of claim 266, further comprising the step of: storing the criteria as a named search, wherein another assignment binder can be subsequently assembled using the named search without the criteria being input again. 30

270. The method of claim 266, further comprising the step of: collecting a substantially random sample of the selected subset within the assignment binder. 95 WO 2006/110892 PCT/US2006/014043

271. A system for selectively assembling an assignment binder including a selected subset of a plurality of artifacts, comprising: a programmable computer configured to: receive first input defining a criteria, the criteria related to a 5 plurality of identifying attributes associated, respectively, with each of the plurality of artifacts; identify the selected subset based on those of the plurality of artifacts having associated identifying attributes matching the criteria, wherein the criteria relates to at least two criterion related to one or more of demographic attributes, course 10 related attributes, operating unit-related attributes, section-related attributes, goal-related attributes, and educational experience-related attributes; and collect the selected subset of artifacts within the assignment binder.

272. The system of claim 271, wherein the programmable computer is 15 further configured to: sample the selected subset substantially randomly as part of a determination of which artifacts in the selected subset to collect.

273. The system of claim 271, wherein each of the plurality of artifacts is 20 a response to a respective instrument deployed to a plurality of users.

274. The system of claim 273, the programmable computer further configured to: forward the respective instruments to a separate computer-based 25 system configured to disseminate information to the plurality of users; and receive the plurality of artifacts from the separate computer-based system.

275. The system of claim 274, wherein the separate computer-based 30 system comprises a campus academic system. 96 WO 2006/110892 PCT/US2006/014043

276. The system of claim 275, wherein the campus academic system is configured to provide an environment to disseminate course-related information to a plurality of students of an educational institution. 5 277. The system of claim 271, wherein the programmable computer is further configured to: collect evaluation data associated with each artifact within the selected subset. 10 278. The system of claim 271, wherein the programmable computer is further configured to: store the criteria as a named search, wherein another assignment binder can be subsequently assembled using the named search without the criteria being input again. 15

279. A system for selectively assembling an assignment binder, for a multi-level institution, including a selected subset of a plurality of artifacts, comprising: a programmable computer configured to: receive first input defining a criteria, the criteria related to a 20 plurality of identifying attributes associated, respectively, with each of the plurality of artifacts; forward the criteria to a separate computer-based system of the institution, wherein the separate computer-based system is configured to store the plurality of artifacts; and 25 receive from the separate computer-based system the selected subset of those of the plurality of artifacts having associated identifying attributes matching the criteria.

280. The system of claim 279, wherein the programmable computer is 30 further configured to: collect the selected subset of artifacts within the assignment binder. 97 WO 2006/110892 PCTIUS2006/014043

281. The system of claim 280, wherein the separate computer-based system comprises a campus academic system.

282. The system claim 279, wherein the programmable computer is 5 further configured to: store the criteria as a named search, wherein another assignment binder can be subsequently assembled using the named search without the criteria being input again. 10 283. The system of claim 279, wherein the programmable computer is further configured to: collect a substantially random sample of the selected subset within the assignment binder. 15 284. Computer readable media containing programming instructions for selectively assembling an assignment binder including a selected subset of a plurality of artifacts, that upon execution thereof, causes one or more processors to perform the steps of: receiving first input defining a criteria, the criteria related to a 20 plurality of identifying attributes associated, respectively, with each of the plurality of artifacts; identifying the selected subset based on those of the plurality of artifacts having associated identifying attributes matching the criteria, wherein the criteria relates to at least two criterion related to one or more of demographic attributes, course 25 related attributes, operating unit-related attributes, section-related attributes, goal-related attributes, and educational experience-related attributes; and collecting the selected subset of artifacts within the assignment binder. 30 285. Computer readable media containing programming instructions for selectively assembling an assignment binder including a selected subset of a plurality of artifacts, that upon execution thereof, causes one or more processors to perform the steps of: 98 WO 2006/110892 PCT/US2006/014043 receiving first input defining a criteria, the criteria related to a plurality of identifying attributes associated, respectively, with each of the plurality of artifacts; forwarding the criteria to a separate computer-based system of the 5 institution, wherein the separate computer-based system is configured to store the plurality of artifacts; and receiving from the separate computer-based system the selected subset of those of the plurality of artifacts having associated identifying attributes matching the criteria. 99