WO2001063517A1 - Procede d'analyse d'un employe basee sur des resultats d'un enseignement par simulation d'entreprise - Google Patents

Procede d'analyse d'un employe basee sur des resultats d'un enseignement par simulation d'entreprise Download PDF

Info

Publication number
WO2001063517A1
WO2001063517A1 PCT/US2001/003420 US0103420W WO0163517A1 WO 2001063517 A1 WO2001063517 A1 WO 2001063517A1 US 0103420 W US0103420 W US 0103420W WO 0163517 A1 WO0163517 A1 WO 0163517A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
network
goal
student
utilizing
Prior art date
Application number
PCT/US2001/003420
Other languages
English (en)
Inventor
William A. Gray
Douglas S. Coons
Original Assignee
Accenture Llp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US09/511,340 external-priority patent/US6944596B1/en
Application filed by Accenture Llp filed Critical Accenture Llp
Priority to EP01905361A priority Critical patent/EP1269365A4/fr
Priority to AU2001233251A priority patent/AU2001233251A1/en
Priority to CA002401616A priority patent/CA2401616A1/fr
Publication of WO2001063517A1 publication Critical patent/WO2001063517A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q99/00—Subject matter not provided for in other groups of this subclass

Definitions

  • the present invention relates to training programs and more particularly to employee analysis based on results of an education business simulation.
  • the domain expert has knowledge of the domain or field of use of the expert system.
  • the domain expert of an expert for instructing students in an automotive manufacturing facility might be a process control engineer while the domain expert for a medical instruction system might be a doctor or a nurse.
  • the knowledge engineer is a person that understands the expert system and utilizes the expert's knowledge to create an application for the system. In many instances, the knowledge engineer and domain expert are separate people who have to collaborate to construct the expert system.
  • this collaboration takes the form of the knowledge engineer asking questions of the domain expert and incorporating the answers to these questions into the design of the system.
  • This approach is labor intensive, slow and error prone.
  • the coordination of the two separate disciplines may lead to problems.
  • the knowledge engineer can transcribe input from the expert utilizing videotape, audio tape, text and other sources, efforts from people of both disciplines have to be expended. Further, if the knowledge engineer does not ask the right questions or asks the questions in an incorrect way, the information utilized to design the knowledge base could be incorrect.
  • Feedback to the knowledge engineer from the expert system is often not available in prior art system until the construction is completed. With conventional system, there is a time consuming feedback loop that ties together various processes from knowledge acquisition to validation.
  • a method for analyzing performance in a network based training simulation is utilized to present a training simulation to a user such as an employee for achieving a training goal.
  • Information is integrated into the training simulated utilizing the network that helps motivate achievement of the goal by the user.
  • integrated information may include information for helping the user develop skills for more efficiently achieving the goal.
  • Progress of the user towards achieving the goal is measured utilizing the network. This progress measurement may include, for example, tracking skill areas that the user has difficulty mastering.
  • feedback is also provided to the user utilizing the network. The user's progress towards the goal is subsequently reported to the employer of the user utilizing the network.
  • the goal may be one of: (1) service excellence for teaching the user skills in handling a high percentage of customer calls correctly; (2) billing excellence for teaching the user skills for consistently making decision and taking actives that lead to correct customer bills; (3) sales and marketing excellence for teaching the user skills in increasing sales of products and services, (4) deregulation transition excellence for teaching the user skills for correctly responding in competitive scenarios, (5) commercial/industrial excellence for teaching the user skills for increasing revenue, customer satisfaction, and retention, (6) credit and collections excellence for teaching the user skills that help increase revenue and decrease bad debts, and (7) back office excellence for teaching the user skills that increase throughput of back office work items, hi another aspect of the present invention, the goal may include at least one business objective and at least one learning objective.
  • information indicative of the goal maybe presented utilizing the network prior to the presenting of the training simulation
  • the measurement of the user's progress may be stored in a database utilizing the network sot that the measurement of the user's progress may be compared to a measurement of the progress of subsequent other users.
  • the network may be a wide area network capable of communicating using TCP/IP and IPX protocol
  • the training simulation may be displayed in a browser such as an Internet browser like Microsoft's MS Internet Explorer and Netscape's Navigator.
  • Figure 1 is a schematic diagram of a representative hardware environment in accordance with an embodiment of the present invention.
  • Figure 2 illustrates a model for the various aspects of training in accordance with an embodiment of the present invention
  • Figure 3 illustrates a model for focusing the core principles on various types of employees in accordance with an embodiment of the present invention
  • Figure 4 is a flowchart for a process for a network-based training simulation payment scheme in accordance with an embodiment of the present invention
  • Figure 5 is a flowchart for a process for analyzing performance in a network based training simulation in accordance with an embodiment of the present invention
  • Figure 6 is a flowchart for a process for tying training to business effectiveness in a network based simulation environment in accordance with an embodiment of the present invention
  • Figure 7 is a flowchart for a process for capturing employee capability data in a network based training simulation in accordance with an embodiment of the present invention
  • Figure 8A is a flowchart for a process for performing reporting and data analysis in a network based educational business simulation in accordance with an embodiment of the present invention
  • Figure 8B is a flowchart for a process for developing module content in accordance with an embodiment of the present invention
  • Figure 9 A is a block diagram of a system architecture in accordance with an embodiment of the present invention.
  • Figure 9B depicts the timeline and relative resource requirements for each phase of development for a typical application development in accordance with an embodiment of the present invention
  • Figure 9C depicts the potential savings in both functional and technical tasks in accordance with an embodiment of the present invention.
  • FIG. 9D illustrates commonalties in accordance with an embodiment of the present invention.
  • Figure 9E illustrates a development architecture approach in accordance with an embodiment of the present invention.
  • Figure 9F illustrates a small segment of a domain model for claims handlers in the auto insurance industry in accordance with an embodiment of the present invention
  • Figure 9G illustrates an instantiated domain model in accordance with an embodiment of the present invention
  • Figure 9H illustrates an insurance underwriting profile in accordance with an embodiment of the present invention.
  • Figure 10 illustrates a transformation component in accordance with an embodiment of the present invention
  • Figure 11 illustrates the use of a toolbar to navigate and access application level features in accordance with an embodiment of the present invention
  • Figure 12 is a GBS display in accordance with an embodiment of the present invention
  • Figure 13 is a feedback display in accordance with an embodiment of the present invention
  • Figure 14 illustrates a display in which a student has made some mistakes in accordance with an embodiment of the present invention
  • Figure 15 illustrates a journal entry simulation in accordance with an embodiment of the present invention
  • Figure 16 illustrates a simulated Bell Phone Bill journal entry in accordance with an embodiment of the present invention
  • Figure 17 illustrates a feedback display in accordance with an embodiment of the present invention
  • FIGS. 18 and 19 illustrate a feedback display in accordance with an embodiment of the present invention
  • Figure 20 illustrates a feedback display in accordance with an embodiment of the present invention
  • Figure 21 illustrates a simulation display in accordance with an embodiment of the present invention
  • Figure 22 illustrates the steps of the first scenario in accordance with an embodiment of the present invention
  • Figures 23 and 24 illustrate the steps associated with a build scenario in accordance with an embodiment of the present invention
  • Figure 25 illustrates how the tool suite supports student administration in accordance with an embodiment of the present invention
  • Figure 26 illustrates a suite to support a student interaction in accordance with an embodiment of the present invention
  • Figure 27 illustrates the remediation process in accordance with an embodiment of the present invention
  • Figure 28 illustrates a display of journalization transactions in accordance with an embodiment of the present invention
  • Figure 29 illustrates the objects for the journalization task in accordance with an embodiment of the present invention
  • Figure 30 illustrates the mapping of a source item to a target item in accordance with an embodiment of the present invention
  • Figure 31 illustrates target group bundles in accordance with an embodiment of the present invention
  • Figure 32 illustrates a TargetGroup Hierarchy in accordance with an embodiment of the present invention
  • Figure 33 illustrates a small section the amount of feedback in accordance with an embodiment of the present invention
  • Figure 34 illustrates an analysis of rules in accordance with an embodiment of the present invention
  • Figure 35 illustrates a feedback selection in accordance with an embodiment of the present invention
  • Figure 36A illustrates a portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention
  • Figure 36B illustrates another portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention.
  • Figure 36C illustrates an additional portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention
  • Figure 36D illustrates a further portion of a flowchart of the feedback logic in accordance with an embodiment of the present invention
  • Figure 37 illustrates an example of separating out some mistakes for the interface to catch and others for the ICAT to catch has positive and negative impacts in accordance with an embodiment of the present invention
  • Figure 38 is a block diagram of the hierarchical relationship of a transaction in accordance with an embodiment of the present invention.
  • Figure 39 is a block diagram illustrating the feedback hierarchy in accordance with an embodiment of the present invention.
  • Figure 40 is a block diagram illustrating how the simulation engine is architected into an embodiment of the present invention of the invention.
  • Figure 41 is a block diagram setting forth the architecture of a simulation model in accordance with an embodiment of the present invention.
  • Figure 42 illustrates the arithmetic steps in accordance with an embodiment of the present invention
  • Figure 43 illustrates a drag & drop input operation in accordance with an embodiment of the present invention
  • Figure 44 illustrates list object processing in accordance with an embodiment of the present invention
  • Figure 45 illustrates the steps for configuring a simulation in accordance with an embodiment of the present invention
  • Figure 46 illustrates a distinct output in accordance with an embodiment of the present invention
  • Figure 47 is a block diagram presenting the detailed architecture of a system dynamics model in accordance with an embodiment of the present invention.
  • Figure 48 is graphical representation of the object model which is utilized to instantiate the system dynamic engine in accordance with an embodiment of the present invention.
  • Figure 49 is a Plnput Cell for a simulation model in accordance with an embodiment of the present invention.
  • Figure 50 is a Plnput backup cell in a simulation model in accordance with an embodiment of the present invention.
  • Figure 51 is a display illustrating a POutput cell in accordance with an embodiment of the present invention. The steps required to configure the POutput are presented below;
  • Figure 52 is an overview diagram of the logic utilized for initial configuration in accordance with an embodiment of the present invention.
  • Figure 53 is a display of the source item and target configuration in accordance with an embodiment of the present invention.
  • Figure 54 is a display of video information in accordance with an embodiment of the present invention.
  • Figure 55 illustrates a display depicting configured rules in accordance with an embodiment of the present invention
  • Figure 56 illustrates feedback for configured rules in accordance with an embodiment of the present invention
  • Figure 57 illustrates a display with follow-up configuration questions in accordance with an embodiment of the present invention
  • Figure 58 illustrates configuration of aggregate rules in accordance with an embodiment of the present invention
  • Figure 59 illustrates a set of coach items in accordance with an embodiment of the present invention
  • Figure 60 is an ICA Meeting Configuration tool display in accordance with an embodiment of the present invention.
  • Figure 61 illustrates an ICA utility in accordance with an embodiment of the present invention
  • Figure 62 illustrates a configuration utility display in accordance with an embodiment of the present invention
  • Figure 63 illustrates an object editor toolbar in accordance with an embodiment of the present invention
  • Figure 64 illustrates the seven areas that can be configured for a simulation in accordance with an embodiment of the present invention
  • Figure 65 illustrates a display that defines inputs in accordance with an embodiment of the present invention
  • Figure 66 illustrates a list editor in accordance with an embodiment of the present invention
  • Figure 67A illustrates a define student display in accordance with an embodiment of the present invention
  • Figure 67B illustrates a ControlSourceltem display in accordance with an embodiment of the present invention
  • Figure 68 illustrates a simulation workbench in accordance with an embodiment of the present invention
  • Figure 69 illustrates an object viewer in accordance with an embodiment of the present invention. As shown in
  • Figure 70 illustrates an Object Viewer Configuration in an Utilities menu in accordance with an embodiment of the present invention
  • Figure 71 illustrates a log viewer in accordance with an embodiment of the present invention
  • Figure 72 illustrates a Doc Maker display in accordance with an embodiment of the present invention
  • Figure 73 illustrates a Feedback display in accordance with an embodiment of the present invention
  • Figure 74 is an object editor display that illustrates the use of references in accordance with an embodiment of the present invention.
  • Figure 75 presents the detailed design of smart spreadsheets in accordance with an embodiment of the present invention.
  • Figure 1 is a schematic diagram of one possible hardware implementation by which the present invention may be carried out. As shown, the present invention may be practiced in the context of a personal computer such as an IBM compatible personal computer, Apple Macintosh computer or UNIX based workstation.
  • a personal computer such as an IBM compatible personal computer, Apple Macintosh computer or UNIX based workstation.
  • FIG. 1 A representative hardware environment is depicted in Figure 1, which illustrates a typical hardware configuration of a workstation in accordance with one embodiment having a central processing unit 110, such as a microprocessor, and a number of other units interconnected via a system bus 112.
  • the workstation shown in Figure 1 includes a Random Access Memory (RAM) 114, Read Only Memory (ROM) 116, an I/O adapter 118 for connecting peripheral devices such as disk storage units 120 to the bus 112, a user interface adapter 122 for connecting a keyboard 124, a mouse 126, a speaker 128, a microphone 132, and/or other user interface devices such as a touch screen (not shown) to the bus 112, communication adapter 134 for connecting the workstation to a communication network (e.g., a data processing network) and a display adapter 136 for connecting the bus 112 to a display device 138.
  • a communication network e.g., a data processing network
  • display adapter 136 for connecting the bus 112 to
  • the workstation typically has resident thereon an operating system such as the Microsoft Windows NT or Windows/95 Operating System (OS), the IBM OS/2 operating system, the MOS, or UNIX operating system.
  • OS Microsoft Windows NT or Windows/95 Operating System
  • IBM OS/2 operating system the IBM OS/2 operating system
  • MOS the MOS
  • UNIX operating system the operating system
  • OOP Object oriented programming
  • An object is a software package that contains both data and a collection of related structures and procedures. Since it contains both data and a collection of structures and procedures, it can be visualized as a self- sufficient component that does not require other additional structures, procedures or data to perform its specific task.
  • OOP therefore, views a computer program as a collection of largely autonomous components, called objects, each of which is responsible for a specific task. This concept of packaging data, structures, and procedures together in one component or module is called encapsulation.
  • OOP components are reusable software modules which present an interface that conforms to an object model and which are accessed at run-time through a component integration architecture.
  • a component integration architecture is a set of architecture mechanisms which allow software modules in different process spaces to utilize each others capabilities or functions. This is generally done by assuming a common component object model on which to build the architecture. It is worthwhile to differentiate between an object and a class of objects at this point.
  • An object is a single instance of the class of objects, which is often just called a class.
  • a class of objects can be viewed as a blueprint, from which many objects can be formed.
  • OOP allows the programmer to create an object that is a part of another object.
  • the object representing a piston engine is said to have a composition-relationship with the object representing a piston, hi reality, a piston engine comprises a piston, valves and many other components; the fact that a piston is an element of a piston engine can be logically and semantically represented in OOP by two objects.
  • OOP also allows creation of an object that "depends from” another object. If there are two objects, one representing a piston engine and the other representing a piston engine wherein the piston is made of ceramic, then the relationship between the two objects is not that of composition.
  • a ceramic piston engine does not make up a piston engine. Rather it is merely one kind of piston engine that has one more limitation than the piston engine; its piston is made of ceramic, hi this case, the object representing the ceramic piston engine is called a derived object, and it inherits all of the aspects of the object representing the piston engine and adds further limitation or detail to it.
  • the object representing the ceramic piston engine "depends from" the object representing the piston engine. The relationship between these objects is called inheritance.
  • the object or class representing the ceramic piston engine inherits all of the aspects of the objects representing the piston engine, it inherits the thermal characteristics of a standard piston defined in the piston engine class.
  • the ceramic piston engine object overrides these ceramic specific thermal characteristics, which are typically different from those associated with a metal piston. It skips over the original and uses new functions related to ceramic pistons.
  • Different kinds of piston engines have different characteristics, but may have the same underlying functions associated with it (e.g., how many pistons in the engine, ignition sequences, lubrication, etc.).
  • a programmer would call the same functions with the same names, but each type of piston engine may have different/overriding implementations of functions behind the same name. This ability to hide different implementations of a function behind the same name is called polymorphism and it greatly simplifies communication among objects.
  • composition-relationship With the concepts of composition-relationship, encapsulation, inheritance and polymo ⁇ hism, an object can represent just about anything in the real world. In fact, our logical perception of the reality is the only limit on determining the kinds of things that can become objects in object- oriented software. Some typical categories are as follows:
  • Objects can represent physical objects, such as automobiles in a traffic-flow simulation, electrical components in a circuit-design program, countries in an economics model, or aircraft in an air-traffic-control system.
  • Objects can represent elements of the computer-user environment such as windows, menus or graphics objects.
  • An object can represent an inventory, such as a personnel file or a table of the latitudes and longitudes of cities.
  • An object can represent user-defined data types such as time, angles, and complex numbers, or points on the plane.
  • OOP allows the software developer to design and implement a computer program that is a model of some aspects of reality, whether that reality is a physical entity, a process, a system, or a composition of matter. Since the object can represent anything, the software developer can create an object which can be used as a component in a larger software project in the future.
  • OOP enables software developers to build objects out of other, previously built objects.
  • C++ is an OOP language that offers a fast, machine-executable code.
  • C++ is suitable for both commercial-application and systems-programming projects.
  • C++ appears to be the most popular choice among many OOP programmers, but there is a host of other OOP languages, such as Smalltalk, Common Lisp Object System (CLOS), and Eiffel. Additionally, OOP capabilities are being added to more traditional popular computer programming languages such as Pascal.
  • Class hierarchies and containment hierarchies provide a flexible mechanism for modeling real- world objects and the relationships among them.
  • class libraries allow programmers to use and reuse many small pieces of code, each programmer puts those pieces together in a different way.
  • Two different programmers can use the same set of class libraries to write two programs that do exactly the same thing but whose internal structure (i.e., design) may be quite different, depending on hundreds of small decisions each programmer makes along the way.
  • similar pieces of code end up doing similar things in slightly different ways and do not work as well together as they should.
  • Class libraries are very flexible. As programs grow more complex, more programmers are forced to adopt basic solutions to basic problems over and over again.
  • a relatively new extension of the class library concept is to have a framework of class libraries. This framework is more complex and consists of significant collections of collaborating classes that capture both the small scale patterns and major mechanisms that implement the common requirements and design in a specific application domain. They were first developed to free application programmers from the chores involved in displaying menus, windows, dialog boxes, and other standard user interface elements for personal computers.
  • Frameworks also represent a change in the way programmers think about the interaction between the code they write and code written by others.
  • the programmer called libraries provided by the operating system to perform certain tasks, but basically the program executed down the page from start to finish, and the programmer was solely responsible for the flow of control. This was appropriate for printing out paychecks, calculating a mathematical table, or solving other problems with a program that executed in just one way.
  • event loop programs require programmers to write a lot of code that should not need to be written separately for every application.
  • the concept of an application framework carries the event loop concept further. Instead of dealing with all the nuts and bolts of constructing basic menus, windows, and dialog boxes and then making these things all work together, programmers using application frameworks start with working application code and basic user interface elements in place. Subsequently, they build from there by replacing some of the generic capabilities of the framework with the specific capabilities of the intended application.
  • -Application frameworks reduce the total amount of code that a programmer has to write from scratch.
  • the framework is really a generic application that displays windows, supports copy and paste, and so on, the programmer can also relinquish control to a greater degree than event loop programs permit.
  • the framework code takes care of almost all event handling and flow of control, and the programmer's code is called only when the framework needs it (e.g., to create or manipulate a proprietary data structure).
  • a programmer writing a framework program not only relinquishes control to the user (as is also true for event loop programs), but also relinquishes the detailed flow of control within the program to the framework. This approach allows the creation of more complex systems that work together in interesting ways, as opposed to isolated programs, having custom code, being created over and over again for similar problems.
  • a framework basically is a collection of cooperating classes that make up a reusable design solution for a given problem domain. It typically includes objects that provide default behavior (e.g., for menus and windows), and programmers use it by inheriting some of that default behavior and overriding other behavior so that the framework calls application code at the appropriate times.
  • default behavior e.g., for menus and windows
  • Behavior versus protocol Class libraries are essentially collections of behaviors that one can call when one wants those individual behaviors in a program.
  • a framework provides not only behavior but also the protocol or set of rules that govern the ways in which behaviors can be combined, including rules for what a programmer is supposed to provide versus what the framework provides.
  • a framework embodies the way a family of related programs or pieces of software work. It represents a generic design solution that can be adapted to a variety of specific problems in a given domain. For example, a single framework can embody the way a user interface works, even though two different user interfaces created with the same framework might solve quite different interface problems.
  • An embodiment of the present invention of the invention utilizes HyperText Markup Language (HTML) to implement documents on the Internet together with a general- pu ⁇ ose secure communication protocol for a transport medium between the client and the Newco.
  • HTTP or other protocols could be readily substituted for HTML without undue experimentation.
  • Information on these products is available in T. Berners-Lee, D. Connoly, "RFC 1866: Hypertext Markup Language - 2.0" (Nov. 1995); and R. Fielding, H, Frystyk, T. Berners-Lee, J. Gettys and J.C.
  • HTML Hypertext Transfer Protocol - HTTP/1.1 : HTTP Working Group Internet Draft
  • HTML documents are SGML documents with generic semantics that are appropriate for representing information from a wide range of domains. HTML has been in use by the World-Wide Web global information initiative since 1990. HTML is an application of ISO Standard 8879; 1986 Information Processing Text and Office Systems; Standard Generalized Markup Language (SGML).
  • HTML has been the dominant technology used in development of Web-based solutions.
  • HTML has proven to be inadequate in the following areas:
  • UI User Interface
  • Custom “widgets” e.g., real-time stock tickers, animated icons, etc.
  • client-side performance is improved.
  • Java supports the notion of client-side validation, offloading appropriate processing onto the client for improved performance.
  • Dynamic, real-time Web pages can be created. Using the above-mentioned custom UI components, dynamic Web pages can also be created.
  • Sun's Java language has emerged as an nidustry-recognized language for "programming the Internet.”
  • Sun defines Java as: "a simple, object-oriented, distributed, inte ⁇ reted, robust, secure, architecture-neutral, portable, high-performance, multithreaded, dynamic, buzzword- compliant, general-pu ⁇ ose programming language.
  • Java supports programming for the Internet in the form of platform-independent Java applets.”
  • Java applets are small, specialized applications that comply with Sun's Java Application Programming Interface (API) allowing developers to add "interactive content” to Web documents (e.g., simple animations, page adornments, basic games, etc.).
  • API Java Application Programming Interface
  • Applets execute within a Java-compatible browser (e.g., Netscape Navigator) by copying code from the server to client. From a language standpoint, Java's core feature set is based on C++. Sun's Java literature states that Java is basically, "C++ with extensions from Objective C for more dynamic method resolution.”
  • ActiveX includes tools for developing animation, 3-D virtual reality, video and other multimedia content.
  • the tools use Internet standards, work on multiple platforms, and are being supported by over 100 companies.
  • the group's building blocks are called ActiveX Controls, small, fast components that enable developers to embed parts of software in hypertext markup language (HTML) pages.
  • ActiveX Controls work with a variety of programming languages including Microsoft Visual C++, Borland Delphi, Microsoft Visual Basic programming system and, in the future, Microsoft's development tool for Java, code named "Jakarta.”
  • ActiveX Technologies also includes ActiveX Server Framework, allowing developers to create server applications.
  • ActiveX could be substituted for JAVA without undue experimentation to practice the invention.
  • a solution for improving human performance is training.
  • a training program to be an effective performance-based program, a number of elements are recommended:
  • Training environment structured to mirror the job.
  • Figure 2 illustrates a model for the various aspects of training including: (1) a simulation 200 of the job and development of employee's ability to perform their job (technical, business, and customer skills; (2) a training environment 202 to deliver training anytime and anywhere; (3) coaching tips 204 and learning resources 206 to enhance performance development; and (4) A curriculum 208 of content addressing high volume, recurrent development needs.
  • a potential benefit includes making employees more successful at their jobs - business simulations maximize retention & reinforcement of desired employee behaviors.
  • a second potential benefit includes addressing high business impact skill areas - regulated & retail energy suppliers - areas of significant cost or customer service impact.
  • a third potential benefit includes tailoring to each employer's specific business needs and objectives - cost effective delivery through common business process & technology platform.
  • a fourth potential benefit includes delivering training to the point of need - Internet-based - anytime, anywhere.
  • a fifth potential benefit includes developing content rapidly - leverage platform/training assets while facilitating client involvement/approval.
  • a sixth potential benefit includes improving management insight - summary & detail reporting of utilization, certification, scoring, and feedback.
  • a seventh potential benefit includesminimizing fixed training investment - pay-peruse model - costs are tied to training benefits (assessment).
  • An eighth potential benefit includes making training more successful - give training another toolset to address training requirements proactively.
  • Business simulations may serve as the basis for delivering such performance and provide a learning environment that enables participants to acquire technical and social skills better (i.e., with higher retention) and faster (i.e., less time to competence) by:roviding a goal, simulating the workplace (with a focus on application of knowledge and skills), providing a risk-free environment, allowing the learner to make mistakes, allowing the learner to set own pace, providing support/resources/expertise as needed, and allowing performance to confirm employee's competence.
  • a customer service curriculum may be implemented to provide excellent customer service, decrease costs, and increase revenue through employee skills development.
  • Figure 3 illustrates a model for focusing the core principles 300 on various types of employees including new hire employees 302, current employees 304, specific targeted employees 306.
  • One module that may be included in the customer service curriculum is a service excellence module for teaching employees to handle 99% of all call types correctly the first time, within AHT goals, while referring zero calls.
  • this module the following sub-modules maybe included: (1) Resolve Billing Inquiries, (2) Effective High Bill Resolution, (3) Efficient Call Handling, (4) Proactive Customer Problem Solving, (5) Issue Change Meter Orders, (6) Issue New Service Requests, (7) The Benefits of EFT, (8) Maintain Account Information, (9) Issue Outages and Emergencies, (10) Issue Service Orders, and (11) Issue Connects / Disconnects.
  • Another module that may be included in the customer service curriculum is a billing excellence module for teaching employees to consistently make decisions and take actions that lead to correct customer bills.
  • the following sub-modules may be included: (1) Resolve Billing Inquiries, (2) Adjustments: When are they appropriate, (3) Effective High Bill Resolution, (4) Initial Billing Issue Analysis, (5) Budget Billing: When, How, and Why?, (6) Issue Change Meter Orders, (7) Summary Billing as a Service, (8) Resolve Non-service Billing Issues, (9) Resolve Miscellaneous Order Issues, (10) The Benefits of EFT, and (11) Obligations: When are they appropriate.
  • a third module that may be included in the customer service curriculum is a sales and marketing excellence module for teaching employees skills for increasing sales of products and services.
  • this module the following sub-modules maybe included: (1) Effective Marketing and CRM, (2) Effective Cross-selling,
  • An additional module that may be included in the customer service curriculum is a deregulation transition excellence module for teaching employees skills for responding to competitive scenarios correctly 99% or the time.
  • this module the following sub-modules may be included: (1) What is Deregulation, (2) Understanding the Energy Merchant's Role, (3) Understanding the Energy Merchant's Code of Conduct, (4) Understanding the ISO/PX, (5) Effective Marketing and CRM, (6) Effective Cross-selling, (7) Understanding the Components of an Unbundled Bill, (8) Understanding Different Billing Models, (9) Electronic Communications: Understanding EDI, (10) Understanding Enrollment and Registration, (11) Understanding Exception Processing, and (12) Understanding Order Tracking.
  • a further module that may be included in the customer service curriculum is a commercial/industrial excellence module for teaching employees how to increase revenue, customer satisfaction and retention.
  • the following sub-modules may be included: (1) Effective Marketing and CRM, and (2) Effective Rate Selection and Calculation.
  • a credit and collections excellence module may also be included in the customer service curriculum for teaching employees skills for helping to increase revenue and to decrease bad debt.
  • the following sub-modules maybe included: (1) Understanding the Collections Process, (2) Extensions, Arrangements, and Agreements, and (3) Understanding Deposits, Returned Checks, and Bad Debt.
  • Yet another module that may be included in the customer service curriculum is a back office excellence module for teaching employees skills for increasing throughput of back office work items, h this module, the following sub-modules may be included: (1) Effective Service Order Dispatching and Completion, (2) Understanding Meter Reading and Reroutes, and (3) Resolve Miscellaneous Order Issues, (4) Efficient Work Item Management and Completion.
  • Sub-modules are designed to translate business objectives into learning objectives in order to achieve business benefits. As an illustrative example, for a high bill sub-module (in the billing excellence module) the business objective may be to improve the ability to effectively & consistently address high bill complaints.
  • the derived learning objectives may include teaching employees skills for initial customer interaction responses, analyzing root cause of complaints, proper responses to complaints, and correct customer interaction styles. These learning objectives will help achieve business benefits such as increasing customer satisfaction through consistent customer interactions, reinforcing behaviors (selling, collections, etc) to meet business objectives, and reducing lost time/effort resulting from resolution errors.
  • the business objective may be to improve the identification and quick resolution of no billed accounts by employees, hi such a sub-module, the learning objectives may include correctly identifying no bill accounts, quickly analyzing cause of no bill, identifying correct simple no bill fixes, and correctly referring difficult no bills.
  • the potential business benefits from such a sub-module may include the increased ability of more employees to handle no bill accounts, the reduced total number of no bill accounts through increase in resolution, and the more efficiently handling of difficult no bill accounts.
  • the present invention provides for an Internet-based service that delivers high- impact training at the point of need on an anytime, anywhere basis.
  • the present invention is based on the proven teaching concept that people "learn best by doing".
  • the present invention uses interactive multimedia technology to simulate real-world situations and to guide and advise employees as they respond to those situations.
  • the present invention encourages learning by letting employees understand and meet performance goals, and giving them access to a wealth of infonnation from video clips of expert advice to relevant background data to targeted learning resources.
  • the simulation techniques used by the present invention allow employees to work at their own pace, and to learn and make mistakes in a realistic but risk-free environment.
  • the present invention provides employee development on a pay-per-employee basis. Once an employee is enrolled in a training module, he or she can use it over and over again to refresh or enhance their skills without additional cost. As employees complete the present invention scenarios, their progress is measured and captured. The data captured is reported to managers allowing them to gauge the effectiveness of training, to see which individuals are mastering skills, and to identify skill areas that are particularly difficult for employees in general.
  • the present invention development architecture allows for rapid customization of pre-defined business attributes to deliver learning modules. For example, when an employee accesses training in the present invention, he or she will receive training that teaches their company's desired behaviors using their company's specific business scenarios as exercises. This creates a learning context where employees can safely learn to apply new skills and reinforce behaviors in the manner of their company's specific culture and objectives.. This also allows for company- specific tailored content to be developed at a price that is competitive to generic content.
  • the present invention provides for full end-to-end integrated skill development service offering by: (1) providing content customized to meet client business objectives at the point of need, anytime, (2) providing larger content offerings and benefits larger than any one client requires, (3) hosting and support services provided, (4) providing content creation / maintenance services, (5) tailored employee assessment and feedback within context of each module, (6) ease of maintenance over time and priced accordingly, (7) reporting capabilities of per employee skill achievement, participation, and training management, (8) integration of clients existing learning resources into critical learning points within teaching content, plus (9) creation of additional learning resources to fill identified learning gaps.
  • Simulation teaching approach reduce productive time away from the job by as much as 80% over traditional learning approaches. Faster acceptance of company values and culture by the viewing of war stories and messages told by company experts and highly respected executives.
  • FIG 4 is a flowchart for a process 400 for a network-based training simulation payment scheme in accordance with an embodiment of the present invention.
  • a user such as an employee is permitted to initiate a training simulation session utilizing a network in operation 402.
  • the network is then utilized in operation 404 to present the training simulation session to the user.
  • a session charge is assessed each time the training simulation session is initiated.
  • the session charge may be a fixed per use fee that is charged each time a user initiates a session.
  • An invoice is then generated that includes the assessed session charge in operation 408.
  • the invoice may be transmitted utilizing the network to a customer such as the user's employer.
  • a user identifier may be assigned to the user upon initiation of the training session. This user identifier is associated with the associated session charge and stored in a database.
  • the user identifier may include information identifying an employer of the user. This way each session initiated by employees of a particular customer (i.e., an employer) can be tracked.
  • the generation of the invoice maybe accomplished by retrieving the session charges associated with a common customer from the database so that the retrieved session charges may be added together to determine a total charge to the customer.
  • the invoice may then be created which includes the total charge to the customer.
  • the created invoice may include an itemized list of the retrieved session charges.
  • the network may be a wide area network capable of communicating using TCP/IP and JJPX protocol.
  • the presenting of the training simulation session utilizing the network may further include presenting a simulated environment to a user for achieving a goal utilizing the network and then utilizing the network to integrate information into the simulation environment that motivates achievement of the goal by the user. Progress of the user towards achieving the goal may then be measured utilizing the network with feedback being provided that further motivates achievement of the goal to the user utilizing the network.
  • FIG. 5 is a flowchart for a process 500 for analyzing performance in a network based training simulation in accordance with an embodiment of the present invention.
  • a network is utilized to present a training simulation to a user such as an employee for achieving a training goal.
  • Information is integrated into the training simulated utilizing the network that helps motivate achievement of the goal by the user in operation 504.
  • integrated information may include information for helping the user develop skills for more efficiently achieving the goal.
  • progress of the user towards achieving the goal is measured utilizing the network. This progress measurement may include, for example, tracking skill areas that the user has difficulty mastering.
  • feedback is also provided to the user utilizing the network in operation 508. The user's progress towards the goal is subsequently reported to the employer of the user utilizing the network in operation 510,.
  • the goal may be one of: (1) service excellence for teaching the user skills in handling a high percentage of customer calls correctly; (2) billing excellence for teaching the user skills for consistently making decision and taking actives that lead to correct customer bills; (3) sales and marketing excellence for teaching the user skills in increasing sales of products and services, (4) deregulation transition excellence for teaching the user skills for correctly responding in competitive scenarios, (5) commercial/industrial excellence for teaching the user skills for increasing revenue, customer satisfaction, and retention, (6) credit and collections excellence for teaching the user skills that help increase revenue and decrease bad debts, and (7) back office excellence for teaching the user skills that increase throughput of back office work items.
  • the goal may include at least one business objective and at least one learning objective.
  • information indicative of the goal may be presented utilizing the network prior to the presenting of the training simulation, i another embodiment of the present invention, the measurement of the user's progress maybe stored in a database utilizing the network sot that the measurement of the user's progress maybe compared to a measurement of the progress of subsequent other users.
  • the network may be a wide area network capable of communicating using TCP/IP and IPX protocol.
  • the training simulation may be displayed in a browser such as an Internet browser like Microsoft's MS Internet Explorer and Netscape's Navigator.
  • FIG. 6 is a flowchart for a process 600 for tying training to business effectiveness in a network based simulation environment in accordance with an embodiment of the present invention.
  • a business objective for a business process is first selected.
  • An example of a business objective is the objective to improve the ability to effectively and consistently address high bill complaints.
  • the business process is reviewed to identify at least one decision point in the business process at which a decision is required for achieving the business objective. In other words, achieving/attaining the business objective requires the making of a decision at one or more decision points.
  • a training simulation is then presented for achieving the business objective (i.e., a goal) utilizing a network in operation 606.
  • Information is inco ⁇ orated into the training simulation utilizing the network that motivates the making of a correct decision each decision point for achieving the business objective in operation 608.
  • integrated information may include strategies for helping the user develop decision making skills to help the user make the correct decision at the decision point in order to achieve the business objective.
  • the network is also utilized to monitor progress towards achieving the business objective and to provide feedback that further motivates achievement of the business objective.
  • information may be presented indicative of the correct decision utilizing the network prior to the presenting of the training simulation.
  • a session charge may be assessed each time the simulated environment is presented.
  • the business objective is selected from at least one of: service excellence, billing excellence, sales and marketing excellence, deregulation transition excellence, commercial/industrial excellence, credit and collections excellence, and back office excellence.
  • the network is a wide area network capable of communicating using TCP/IP and IPX protocol.
  • FIG. 7 is a flowchart for a process 700 for capturing employee capability data in a network based training simulation in accordance with an embodiment of the present invention.
  • a training simulation is presented to a plurality of employees utilizing a network in operation 702.
  • the performances of the employees in the training simulation are analyzed to generate employee performance measurements for the employees utilizing the network in operation 704.
  • the employee performance measurements i.e., performance data
  • Performance trends are then determined utilizing the stored employee performance measurements in operation 708.
  • the employee performance measurements may be utilized to provide behavioral correlation to employee reviews.
  • the employee performance may be related to business performance metrics. Progress towards the goal may also be monitored utilizing the network so that feedback can be provided to further motivate accomplishment of the goal utilizing the network.
  • the analysis of the performances of the employees in the training simulation to generate employee performance measurements may also include the calculating of a quantitative degree of correctness in achieving the goal.
  • presenting the training simulation may further require that information indicative of a goal be presented utilizing the network and then integrating information that motivates accomplishment of the goal.
  • the network is a wide area network capable of communicating using TCP/IP and IPX protocol.
  • the determined performance trends may be transmitted to an employer of the employees utilizing the network.
  • FIG. 8A is a flowchart for a process 800 for performing reporting and data analysis in a network based educational business simulation in accordance with an embodiment of the present invention.
  • a business objective is selected and then in operation 804 a training simulation is presented to an employee relating to the business objective utilizing a network.
  • information is integrated into the training simulation that motivates accomplishment of the business objective by the employee utilizing the network.
  • the employee's progress towards accomplishment of the business objective is tracked in operation 808 utilizing the network and feedback is provided in operation 810 utilizing the network that further motivates achievement of the business objective.
  • the tracked progress towards the business objective is then stored in a database in operation 812.
  • the business objective maybe selected from: service excellence, , billing excellence, , sales and marketing excellence, deregulation transition excellence, commercial/industrial excellence, credit and collections excellence, and/or back office excellence.
  • the tracking of the employee's progress may include tracking the employee's behavior and rationales during the training simulation.
  • the network may be a wide area network capable of communicating using TCP/IP and IPX protocol.
  • a report may be generated based on the tracked progress towards the business object.
  • the effectiveness of the training simulation in teaching accomplishment of the business objective to the employee may be gauged based on the tracked progress and feedback.
  • difficulties encountered by the employee in accomplishing the business object may also be identified based on the tracked progress and feedback.
  • FIG. 8B is a flowchart for a process 850 for developing module content in accordance with an embodiment of the present invention.
  • module specifications are first created in operation 852.
  • learning program detailed design is performed and a learning prototype is created in operations 854 and 856.
  • a learning product is created in operation 858 which is then tested in 860. Subsequently, continuous improvements to the learning product may be made in the future in operation 862.
  • the creation of module specifications in operation 852 includes the creating of a module specification package.
  • the performance of learning program detailed design in operation 854 may include the operations of defining detailed learning objectives for the learning program, designing learning activities for the learning program, designing reference items for the learning program, preparing a learning test plan for the learning program, and finally prepare a learning design report for the learning program.
  • the creation of a learning products prototype in operation 856 may include creating a prototype and then conducting a review of the prototype.
  • the creating of the learning product in operation 858 may include the developing of learning content, media, reference items, evaluation materials, and a maintenance and support plan. Further, the creating of the learning product may also include the reviewing and assembly of the product.
  • the testing of the learning product in operation 860 may include validating a learning test plan, preparing and executing the learning test plan and then performing any necessary fixed on the learning product. Continuous improvement of the learning product in operation 862 may be conducted during and after the execution of each module of the learning product.
  • the module specifications may be created both at a user's site and offsite.
  • the module specifications maybe a "hand-off of information to a functional team offsite.
  • SMEs who will be available for consultation during the entire development of the module.
  • SME include area of expertise, phone number, email address, and preferred method of contact. Also, include the reviewer(s) for the module, areas of module that will be reviewed by this person(s), and dates of the reviews.
  • the Module Specifications Package will outline much of the information required to complete a module. From this information, detailed learning objectives should be established. These objectives will serve as the building blocks for the specific activities within the module, and will include terminal, enabling, and performance objectives. The learning objectives will then guide the design and development of the activities, all the way down to the exercise-specific level. All content developed will directly support one or more of these learning objectives.
  • Designing learning activities consists of adding more details to the format, structure, and function of the activities and providing information on how the learning objectives will be best addressed in the exercises.
  • the information added in the design phase consists of the format for the exercise, the types of quizzes to be used, how many practice vs. assessed exercises to provide, etc. Outlining the specific characteristics of example accounts to be used within activities must also take place at this time.
  • the design of learning activities will provide a higher level overview of the activity without the specific content details that will be added during development.
  • Video talking points should be general enough to allow SMEs to improvise and customize their stories, but specific enough to ensure appropriate stories are received
  • the peripheral reference items may be designed. These consist of the Reference section and the What's Next sections.
  • the content developer will take the information from the available list of resources (in the Module Specifications Package) and create the Job Aids and What's Next requirements. Additionally, content specific links within feedback to exercises should be identified for inclusion in the Reference section. Inputs
  • Job Aid Needs Includes a description of reference materials and job aids. Also includes the list and alphabetical layout of all reference items.
  • the "Learning Test” will be conducted at the user's site. This test will occur after the module has been completed and tested thoroughly offsite, but before the module is released for production at the user's site.
  • developers When preparing the approach to the learning test, developers will determine how to test users before full-scaled production to ensure that the module is effective and provides a consistent learning experience.
  • the Learning Test Approach will be owned by the offsite Team with interaction from the user's site.
  • the Learning Design Report consists of the Content Design Deliverable Packet that will be submitted to reviewers before final .sign-off. Then the completed and reviewed packet will be used as the input for the development phase of the module.
  • a Lo-Fi Prototype is produced using KX Front Page.
  • the prototype displays the general design and layout of the Module down to the Activity and Exercise level. The exact content of the exercises is not displayed, but the general idea of each exercise is laid out.
  • Content Developers create a storyboard depicting all of the visuals (background, window shots, icons, etc) that make up the module. This prototype is then used to display the Module to the SMEs.
  • the Content for the Module must be developed. This includes the titles, instructions, scenarios, audio, questions, answers and feedback for the exercises and the introduction and Wrap-Ups for the activities and the Module. In addition, the What's Next, Reference Job Aids and Videos for the entire Module will be developed. All Content should be loaded into the database via the Content Maintenance Tool.
  • Video and Audio clips are used to enhance the Module.
  • the video clips are developed for the Module as a whole and then divided out and inco ⁇ orated into the individual activities.
  • the Talent for the Video clips are actual user employees, adding credibility to the information taught.
  • the Audio clips are developed for a specific activity that is scenario based.
  • the Audio Talent includes professional actors thus the Audio clips are not necessarily user specific.
  • the Reference section contains the Reference Icon, the What's Next Icon and the Window Tips for the Screen Scrapes.
  • the Reference Icon contains a list of Job Aids that can be accessed through the Reference Icon or though links in the activities. What's Next offers answers to different 'commonly asked questions' for every page of the Module.
  • the Window Tips appear at the top of each CSS Screen Scrape and further explain the CSS window displayed.
  • the goal for the user in taking the Training is twofold: first, to gain knowledge and second to demonstrate efficient performance at a level desired by the user executives.
  • the process that ensures effective performance is the Certification Plan/Skills Plan. This plan assigns a point value to each question in the exercise within the activity and a weight for each activity in the module. The user must answer a predetermined percentage of the questions correctly throughout the module in order to become certified in the module topic.
  • the final step in the Create Learning Products phase is the assembly and test of the Module. Once all of the data has been entered though the Content Maintenance Tool into the database tables, the Module is ready for testing. First the individual components are tested and then assembled together. The assembled pieces are tested for continuity. The final Module is application tested in the production environment.
  • the "Learning Test” will be conducted at the user's site. This test will occur after the module has been completed and tested thoroughly offsite, but before the module is released for production at the user's site. A pre-selected group of user users and SMEs will take the module and validate its effectiveness and accuracy prior to the official release of the module. The team at the user's site will be responsible for creating the Learning Test Plan. This deliverable will document the detailed plans for executing the Learning Test.
  • the Learning Test Plan is based on the Learning Test Approach and documents how the Learning Test will be executed at the user's site.
  • This task centers on implementing the recommendations/procedures in the Learning Test Plan. Several communications, scheduling, and set up tasks must be completed prior to the execution of the Learning Test. This task will be owned by the development team at the user's site.
  • the development team at the user's site will execute the Learning Test. All results of the Learning Test should be documented, prioritized, and categorized and then sent to the development team offsite.
  • a team offsite will review and implement the fixes that result from the Learning Test and plan the post-production enhancement schedule. Fixes will be "fixed” immediately; enhancement will be reviewed and scheduled for future implementation.
  • the Post-Production Enhancement Schedule will be sent to the development team at the user's site for reference (in case any communication needs to occur with the user).
  • the team responsible for the new module will also become the owners of the process documentation for content development.
  • the content development process for the current module will be reviewed and revised as necessary.
  • the streamlined processes will be documented and implemented by the team responsible for the new module.
  • the members of the Cl meeting should be: the entire development team for the current module, the team lead for the current module, the team lead for the user's site team, the team lead for the next module, and one representative from the development team for the new module.
  • Each deliverable is labeled by its task number in the checklist.
  • the SME Question Log is a living spreadsheet that tracks all of the issues encountered by the Content Development Team. All questions/issues should be entered into the SME Log so that SMEs may review them either formally in interviews or informally, and log responses. If a question is answered during an interview that was not originally in the SME log, enter both the question and response after the interview. If maintained and used correctly, this log should be one of the most valuable resources used during the module design and development process.
  • Type column is a numeric based key unique to each Module. It can be used to sort all of the Q & A by topic. The key will appear as a footer on each page of the spreadsheet. For example, the key for Module 3 reads:
  • Job Aids are available for the users in the Reference section of the Module. These Job Aids are tools for the user that provides information in an easy to use format. The Job Aids are reference items that the user can access at any point throughout the Module in order to better understand a topic or concept. These job aids may also be in the form of a tool that guides the user to do a task more efficiently or more effectively. Research
  • CSS Help has search capabilities that allows you to manipulate your search. Use of the Internet is also a possibility if the necessary materials are not found in the CSS. Previous Modules have used other Utility Co. web pages for content specific information. So keep in mind as a possibility.
  • Job Aids/Reference Needs should contain a list of topics and subtopics.
  • the subtopic is the Job Aid name that you have pulled from research and will be expanded into a Job Aid during development. See Job Aid/Reference Template for the deliverable.
  • Each Screen should have a list of general "...1 just” sentences telling the user what to do at each point they could be at in that particular screen. For Example: ...1 just entered the Activity Screen. ... I just pushed the Assessment Exercise button. ...1 just finished reading the feedback
  • the next set of sentences is more general information that may be throughout the Module or specific to the level they are on: Activity Selection, Exercise Selection, Assessed Exercises, etc. These can be viewed and copied if needed from Modules 1 and 2. Examples of general sentences for entire Module: What if I need to stop the training before I am finished? Examples of general sentences specific for a level: What is Assessment? What if I do not successfully demonstrate my skills in assessment?
  • ⁇ Refer to a What's Next Template to prepare the What's Next deliverable.
  • This template is an Excel spreadsheet that has the sections of the module listed on the left side, areas for page keys, super keys, the question asked, and the answer. There are also standards and tips located on the bottom of the template for your reference.
  • This test is designed to ensure that all components within the application are able to communicate with each other effectively.
  • the Assembly Test was performed immediately after the Component Test.
  • the development team members on both the functional and technical sides conducted this test on the pages they respectively developed. This was decided since the developer of the page is most familiar with the components and links they developed, and could therefore easily navigate through the application. Additionally, their familiarity with the individual pages allowed them to quickly identify whether or not anything was missing or malfunctioning more easily than someone who was unfamiliar with the page.
  • the component test was designed to ensure that all the components of each individual page of the application function correctly and as expected. This test was performed prior to all other tests so that a majority of the system bugs could be identified and fixed prior to the later sequences of testing.
  • the development team members on the functional and technical sides) were responsible for conducting this test since they were most familiar with the individual components they developed and could most easily navigate through the untested application efficiently.
  • testers needed to work through the application as if they were users.
  • testers used scripts. These scripts allowed the testers to complete the training module while simulating all the actions and procedures that the end-users may eventually execute, hopefully encountering any potential problems before the program actually went into production.
  • the Product test was designed to ensure that all pages and components of the application worked correctly and as expected. This test was performed in conjunction with all other tests (functional, performance, assembly, etc.). The component test ensures that each of the individual parts should already be tested and functioning correctly as stand-alone pieces of the whole. The product test then grouped all of these individual components together and allowed The development team to test the application as a whole before rollout to the end users.
  • the testing process uncovered a variety of errors within the execution of the application. Each of these errors had to be corrected before rollout to the end user. There were distinct processes and procedures in place to address issues as they came up, funnel them to the appropriate "fixer”, and retest the subject matter again once the error had been fixed. SIRs were filled out to log the error, and were then handed off to the point person to log the issue and assign it to be fixed. Once fixed, the subject was retested to identify that the error no longer existed.
  • the pu ⁇ ose of this portion is to supply a list of items that the client Subject Matter Experts (SMEs) should use/check off when reviewing the content of a given Module.
  • SMEs Subject Matter Experts
  • the pu ⁇ ose of this portion is to outline the methodology that was followed for the detailed design and development of training by the Project. It is intended to provide the team with a common understanding of their roles and responsibilities, as well as information on the key detailed design and development tasks, deliverables and deadlines. This portion should be read in conjunction with the other portions of the documentation, in order to gain a full understanding of the entire process from design to implementation.
  • This portion is to assist content developers by outlining the standard format and guidelines when designing and developing content for component based training modules. Templates have been developed to provide an illustration of the standards and should be utilized in the design and development phases. This portion should be considered a "living document" and as such will be updated when appropriate improvements are identified.
  • This portion includes the following information:
  • the pu ⁇ ose of this portion is to outline the approach to developing the video content of the Modules.
  • This portion describes in detail the steps taken to prepare, film, edit, select and place all of the video clips used for Module 2.
  • the pu ⁇ ose of this portion is to outline the approach to developing the audio content of the Modules.
  • This portion describes in detail the steps taken to prepare, script, record, select and place all of the video clips used for Module 2.
  • the pu ⁇ ose of this portion is to identify each icon in the tool bar and document the necessary research needed for each during the creation of a new module.
  • What's Next The What's Next? feature answers some commonly asked questions that the user may have related to using and/or navigating through the module.
  • the Notebook feature allows the user to either directly type in notes or cut and paste notes into the workspace provided. The user can save their notes as they continuously enter the notebook. At the end of the module, the user can and print out the their notes.
  • the Video feature contains all of the videos marked as links throughout the module.
  • the Video feature presents all of the videos that appear throughout the module and groups them by topic.
  • Reference materials may be gathered for each training module. This section details how to research existing company job aids and reference materials located on CSS (Customer Service System) and develop new reference materials. This portion also details how to enter reference materials into the database.
  • CSS Customer Service System
  • the Save and Exit icon allows the user to save his/her work and exit the module. The next time the user enters the module, he/she may resume where they left off.
  • the Map is a navigational tool used to access any portion of a module that they user has already entered.
  • Module 2 was further broken down into distinct Activities, each of which addressed several distinct learning objectives. SME's at were consulted to provide information on the issues that the training should address. Since Module 2 was designed to replace existing Instructor-Led- Training, the concepts covered in that training were discussed. The Team and the SME's also decided which additional value-added subjects should be covered in Module 2 to increase the effectiveness of the training, and differentiate the module from its predecessor. The result of these discussions was a list of Learning Objectives, or general areas of subject matter, that guided further design and development of the module details.
  • Module 2 Following is a description of the design of Module 2. This will examine the design of the Activities and Exercises, which taken together compose the entirety of the complete module.
  • Each of the Activities was further broken down into exercises, the number of which varied between each Activity.
  • Each Activity consisted of two types of exercises: practice and assessed.
  • the practice exercises were created to cover all of the learning objectives at least once, to give the user an opportunity to practice their skills in each area, and to learn new concepts.
  • the assessed exercises were then prepared to actually measure the participant's understanding of the subject matter, and focused on each of the different learning objectives a minimum of one time. Fo flowing is a list of the types of exercises that were deemed necessary to address all of the necessary Learning Objectives:
  • the Team determined that participants in Module 2 should be allowed to practice their skills in the first two activities, but that all exercises in Activity 3 should be assessed. The reasoning behind this was that the Problem Resolution Activity built upon the previous two, and that completion of the prior activities provided sufficient knowledge for success in Activity 3. Therefore, the subject matter was introduced and assessed in the first Activities, while in Activity 3 it was simply assessed. SME's agreed that this was a legitimate fonnat for success.
  • This database contained all the text-based information that appeared on each screen in the Module. Items such as page instructions, link names, scenarios, questions, feedback, etc. were pulled from the database by the Module, and placed on the screens that the participant viewed. Therefore, all content within the Module originated from the database.
  • any information that is entered into the database that is formatted must include the proper html tags to ensure that the information appears on the display correctly. All formatting in MS Word is lost when transferred into the database, so the html tags are crucial to a module that appears as desired. There is an html tag cheat sheet around that lists the most common formatting commands, such as:
  • Simulation Engine A simulation engine in accordance with an embodiment of the present invention is based on a Microsoft Visual Basic component developed to help design and test feedback in relation to a Microsoft Excel spreadsheet. These spreadsheet models are what simulate actual business functions and become a task that will be performed by a student The Simulation Engine accepts simulation inputs and calculates various outputs and notifies the system of the status of the simulation at a given time in order to obtain appropriate feedback.
  • the simulation model executes the business function that the student is learning and is therefore the center point of the application.
  • An activity 'layer' allows the user to visually guide the simulation by passing inputs into the simulation engine and receiving an output from the simulation model. For example, if the student was working on an income statement activity, the net sales and cost of goods sold calculations are passed as inputs to the simulation model and the net income value is calculated and retrieved as an output. As calculations are passed to and retrieved from the simulation model, they are also passed to the Intelligent Coaching Agent (ICA).
  • ICA Intelligent Coaching Agent
  • the ICA analyzes the Inputs and Outputs to the simulation model and generates feedback based on a set of rules. This feedback is received and displayed through the Visual Basic Architecture.
  • FIG. 2 is a block diagram of a system architecture in accordance with an embodiment of the present invention.
  • the Presentation 'layer' 910 is separate from the activity 'layer' 920 and communication is facilitated through a set of messages that control the display specific content topics.
  • An embodiment of the present invention enables knowledge workers 900 & 901 to acquire complex skills rapidly, reliably and consistently across an organization to deliver rapid acquisition of complex skills. This result is achieved by placing individuals in a simulated business environment that "looks and feels" like real work, and challenging them to make decisions which support a business' strategic objectives utilizing highly effective learning theory (e.g., goal based learning, learn by doing, failure based learning, etc.), and the latest in multimedia user interfaces, coupled with three powerful, integrated software components.
  • highly effective learning theory e.g., goal based learning, learn by doing, failure based learning, etc.
  • the first of these components is a software Solution Consfruction Aid (SCA) 930 consisting of a mathematical modeling tool 934 which simulates business outcomes of an individual's collective actions over a period of time.
  • SCA Software Solution Consfruction Aid
  • the second component is a knowledge system 950 consisting of an HTML content layer which organizes and presents packaged knowledge much like an online text book with practice exercises, video war stories, and a glossary.
  • the third component is a software tutor 970 comprising an artificial intelligence engine 940 which generates individualized coaching messages based on decisions made by learner.
  • a business simulation methodology that includes support for content acquisition, story line design, interaction design, feedback and coaching delivery, and content delivery is architected into the system in accordance with an embodiment of the present invention.
  • a large number of "pre-designed" learning interactions such as drag and drop association of information 938, situation assessment/action planning, interviewing (one-on- one, one-to-many), presenting (to a group of experts/executives), metering of performance (handle now, handle later), "time jumping” for impact of decisions, competitive landscape shift (while "time jumping", competitors merge, customers are acquired, etc.) and video interviewing with automated note taking are also included in accordance with an embodiment of the present invention.
  • Business simulation in accordance with an embodiment of the present invention delivers training curricula in an optimal manner. This is because such applications provide effective training that mirrors a student's actual work environment. The application of skills "on the job” facilitates increased retention and higher overall job performance. While the results of such training applications are impressive, business simulations are very complex to design and build conectly. These simulations are characterized by a very open-ended environment, where students can go through the application along any number of paths, depending on their learning style and prior experiences/knowledge.
  • Learn by Doing A category of learning approaches called Learn by Doing, is commonly used as a solution to support the first phase (Learn) of the Workforce Performance Cycle. However, it can also be a solution to support the second phase (Perform) of the cycle to enable point of need learning during job performance.
  • Learners are reluctant to adopt learning-by-doing because they are frightened of failure. People work hard to avoid making mistakes in front of others. Business leaders are hesitant to implement learning-by-doing because novice failure may have dramatic safety, legal and financial implications. Imagine a novice pilot learning-by-doing as he accelerates a large jet plane down a runway; likewise, consider a new financial analyst learning-by-doing as he structures a multi-million dollar financial loan. Few employers are willing to endure such ⁇ failures to have a more competent workforce.
  • Such a system would provide advice at key moments in the work flow for problem resolution and/or process improvement, tools to ease task completion, reference material of best practice knowledge, and point of need training courses.
  • a support system that proactively assists the worker in performance of their job tasks at a higher level of competency, productivity and customer satisfaction (both internal and external) would soar.
  • This innovation process is the critical final step in continuous education of the most effective and up-to-date policies, procedures, and information. Without formalized innovation, companies not only risk being a step behind the ever advancing competition, but compound the problem by continuing to train their personnel with outdated strategies and information.
  • One way to formalize this vital step in the Workforce Performance Cycle is to construct Virtual Learning Communities, where many 'experts' can share experiences, submit ideas for improvements, play out "what-if scenarios, and contribute on complex problems that may be insurmountable without significant collaboration with others.
  • Such Learning Communities could nurture up-to- date discussion of what is actually happening within a business, eliminating the traditional information-passing lag that plagues many business as new data travels through co ⁇ orate hierarchies. This increased nimbleness would help organizations quickly address new competitive trends and outdated strategies, identify potential solutions, and implement improved processes in the form of updated training and performance support reference materials.
  • Figure 9B depicts the timeline and relative resource requirements for each phase of development for a typical application development in accordance with an embodiment of the present invention.
  • the chart clearly depicts the relationship between the large number of technical resources required for both the build and test phases of development. This is because the traditional development process used to build BusSim solutions reflects more of a "one off philosophy, where development is done from scratch in a monolithic fashion, with little or no reuse from one application to the next. This lack of reuse makes this approach prohibitively expensive, as well as lengthy, for future BusSim projects.
  • Figure 9C depicts the potential savings in both functional and technical tasks in accordance with an embodiment of the present invention.
  • BusSim solutions are not rooted in tangible business processes, so requirements are difficult to identify in a concrete way. This leaves instructional designers with a 'blue sky' design problem. With few business-driven constraints on the solution, shallow expertise in the content area, and limited technical skills, instructional designers have little help in beginning a design. Typically, only experienced designers have been able to conjure interface, analysis, and feedback designs that meet the learning objectives yet remain technically feasible to implement. To compound the problem, BusSim solutions are very open ended in nature. The designer must anticipate a huge combination of student behavior to design feedback that is helpful and realistic.
  • Coding tasks include the interfaces and widgets that the student interacts with.
  • the interfaces can be made up of buttons, grids, check boxes, or any other screen controls that allow the student to view and manipulate his deliverables.
  • the developer must also code logic that analyzes the student's work and provides feedback interactions. These interactions may take the form of text and/or multimedia feedback from simulated team members, conversations with simulated team members, or direct manipulations of the student's work by simulated team members, hi parallel with these coding efforts, graphics, videos, and audio are being created for use in the application. Managing the development of these assets have their own complications.
  • Risks in the build phase include misinte ⁇ retation of the designs. If the developer does not accurately understand the designer's intentions, the application will not function as desired. Also, coding these applications requires very skilled developers because the logic that analyzes the student's work and composes feedback is very complex.
  • the Test Phase is for testing the application. Testing is performed to verify the application in three ways: first that the application functions properly (functional testing), second that the students understand the interface and can navigate effectively (usability testing), and third that the learning objectives are met (cognition testing). Functional testing of the application can be carried out by the development team or by a dedicated test team. If the application fails to function properly, it is debugged, fixed, recompiled and retested until its operation is satisfactory. Usability and cognition testing can only be carried out by test students who are unfamiliar with the application. If usability is unsatisfactory, parts of the interface and or feedback logic may need to be redesigned, recoded, and retested.
  • test phase is typically where most of the difficulties in the BusSim development cycle are encountered.
  • the process of discovering and fixing functional, usability, and cognition problems is a difficult process and not an exact science.
  • testers operate the application, either by following a test script or by acting spontaneously and documenting their actions as they go. When a problem or unexpected result is encountered, it too is documented.
  • the application developer responsible for that part of the application then receives the documentation and attempts to duplicate the problem by repeating the tester's actions. When the problem is duplicated, the developer investigates further to find the cause and implement a fix. The developer once again repeats the tester's actions to verify that the fix solved the problem. Finally, all other test scripts must be rerun to verify that the fix did not have unintended consequences elsewhere in the application.
  • test students operate the application as it will be operated in production. Ideally, their progress is only impeded by their lack of mastery of the content. As they gain proficiency, they are able to complete the activities and move on. As is often the case, however, they are impeded by unclear instructions, a non-intuitive interface, or other usability shortcomings. When these difficulties are encountered, the facilitators document the problems and student comments on what is needed to improve usability.
  • the Execution Phase refers to the steady state operation of the completed application in its production environment. For some clients, this involves phone support for students. Clients may also want the ability to track students' progress and control their progression through the course. Lastly, clients may want the ability to track issues so they may be considered for inclusion in course maintenance releases.
  • on-line courses One of the key values of on-line courses is that they can be taken at a time, location, and pace that is convenient for the individual student. However, because students are not centrally located, support is not always readily available. For this reason it is often desirable to have phone support for students.
  • Clients may also desire to track students' progress, or control their advancement through the course. Under this strategy, after a student completes a section of the course, he will transfer his progress data to a processing center either electronically or by physically mailing a disk. There it can be analyzed to verify that he completed all required work satisfactorily.
  • One difficulty commonly associated with student tracking is isolating the student data for analysis. It can be unwieldy to transmit all the course data, so it is often imperative to isolate the minimum data required to perform the necessary analysis of the student's progress.
  • BusSim solutions reflects more of a "one off philosophy, where development is done from scratch in a monolithic fashion, with little or no reuse from one application to the next. A better approach would be to focus on reducing the total effort required for development through reuse, which, in turn would decrease cost and development time.
  • the first step in considering reuse as an option is the identification of common aspects of the different BusSim applications that can be generalized to be useful in future applications. In examination of the elements that make up these applications, three common aspects emerge as integral parts of each:
  • Every BusSim application must have a mechanism for interaction with the student.
  • the degree of complexity of each interface may vary, from the high interactivity of a high-fidelity real-time simulation task, to the less complex information delivery requirements of a business case background information task. Regardless of how sophisticated the User Interface (UI), it is a vital piece of making the underlying simulation and feedback logic useful to the end user.
  • UI User Interface
  • Every BusSim application does analysis on the data that defines the cunent state of the simulation many times throughout the execution of the application. This analysis is done either to determine what is happening in the simulation, or to perform additional calculations on the data which are then fed back into the simulation.
  • the analysis may be the recognition of any actions the student has taken on artifacts within the simulated environment (notebooks, number values, interviews conducted, etc.), or it may be the calculation of an ROI based on numbers the student has supplied.
  • Substantive, useful feedback is a critical piece of any BusSim application. It is the main mechanism to communicate if actions taken by the student are helping or hurting them meet their performance objectives.
  • the inte ⁇ retation piece of the set of proposed commonalties takes the results of any analysis performed and makes sense of it. It takes the non-biased view of the world that the Analysis portion delivers (i.e., "Demand is up 3%") and places some evaluative context around it (i.e., "Demand is below the expected 7%; you're in trouble!, or "Demand has exceeded projections of 1.5%; Great job!).
  • Figure 9D illustrates commonalties in accordance with an embodiment of the present invention.
  • framework-based reuse a generic framework or architecture is constructed that contains commonalties.
  • a prefabricated house framework consisting of floors, outside walls, bearing walls and a roof
  • the house can be customized by adding partition walls, wall-paper, woodwork, ca ⁇ eting etc.
  • prefabricated application frameworks are available that contain baseline application structure and functionality. Individual applications are completed by adding specific functionality and customizing the look-and-feel.
  • An example of a commonly used application framework is Microsoft Foundation Classes. It is a framework for developing Windows applications using C++. MFC supplies the base functionality of a windowing application and the developer completes the application by adding functionality within the framework.
  • Framework-based reuse is best suited for capturing template-like features, for example user interface management, procedural object behaviors, and any other features that may require specialization.
  • Some benefits of using a framework include:
  • Extensive functionality can be incorporated into a framework.
  • I can build the plumbing, wiring, and partition walls right into the framework, reducing the incremental effort required for each house. If I know I am going to build a large number of very similar applications, they will have more commonalties that can be included in the framework rather than built individually.
  • component-based reuse key functionality is encapsulated in a component.
  • the component can then be reused in multiple applications.
  • components conespond to appliances such as dishwashers, refrigerators, microwaves, etc.
  • application components with pre-packaged functionality are available from a variety of vendors.
  • An example of a popular component is a Data Grid. It is a component that can be integrated into an application to deliver the capability of viewing columnar data in a spreadsheet-like grid.
  • Component-based reuse is best suited for capturing black-box-like features, for example text processing, data manipulation, or any other features that do not require specialization.
  • Some benefits of using components include:
  • a microwave can be used in virtually any house, whether it's framework is steel or wood, and regardless of whether it was customized for building mansions or shacks. You can put a high-end microwave in a low-end house and vice-versa. You can even have multiple different microwaves in your house.
  • Component technologies such as CORBA, COM, and Java Beans make this kind of flexibility commonplace in application development. The Solution: A Combined Approach
  • a framework-based approach for building BusSim applications is appropriate for developing the user interface, handling user and system events, starting and stopping the application, and other application-specific and delivery platform-specific functions.
  • a component-based approach is appropriate for black-box functionality. That is, functionality that can be used as-is with no specialization required.
  • An application can only be based on one framework. Using the house analogy, if you like the first floor of one framework and the second floor of another, it is difficult or impossible to integrate the features of the two. Or, it is so costly as to erase the benefit of using a framework in the first place. Likewise with application frameworks. You can only use one framework when building an application You can't mix and match features from multiple frameworks, so any framework that we developed would have to compete against existing and future frameworks. With components, however, you can mix and match from multiple vendors.
  • Components are less platform and development tool dependent, leaving more options open for development teams.
  • An appliance like a dishwasher is not restricted for use in a particular type of house.
  • component technologies exist that are independent of platform and development tool. For example ActiveX can be used in almost every development environment for Windows and Java Beans components can be used on a wide variety of platforms.
  • FIG. 9E illustrates a development architecture approach in accordance with an embodiment of the present invention.
  • This diagram illustrates an ideal solution where components are combined with an Application Framework and an Application Architecture to achieve maximum reuse and minimum custom development effort.
  • the Application Architecture is added to provide communication support between the application interface and the components, and between the components.
  • the Application Architecture provides the glue that allows application-to-component and component-to-component communication.
  • the Application Framework provides structure and base functionality that can be customized for different interaction styles.
  • a Component can be thought of as a black box that encapsulates the behavior and data necessary to support a related set of services. It exposes these services to the outside world through published interfaces. The published interface of a component allows you to understand what it does through the services it offers, but not how it does it. The complexity of its implementation is hidden from the user. The following are the key components of the BusSim Toolset.
  • ⁇ Domain Component - provides services for modeling the state of a simulation
  • Profiling Component - provides services for rule-based evaluating the state of a simulation
  • Transformation Component - provides services for manipulating the state of a simulation
  • Remediation Component - provides services for the rule-based delivering of feedback to the student
  • the Domain Model component is the central component of the suite that facilitates communication of context data across the application and the other components. It is a modeling tool that can use industry-standard database such as Informix, Oracle, or Sybase to store its data.
  • a domain model is a representation of the objects in a simulation. The objects are such pseudo tangible things as a lever the student can pull, a form or notepad the student fills out, a character the student interacts with in a simulated meeting, etc. They can also be abstract objects such as the ROI for a particular investment, the number of times the student asked a particular question, etc. These objects are called entities. Some example entities include:
  • An entity can also contain other entities.
  • a personal bank account entity might contain an entity that represents a savings account. Every entity has a set of properties where each property in some way describes the entity.
  • the set of properties owned by an entity in essence, define the entity.
  • Some example properties include:
  • An incident entity on an insurance application owns properties such as "Occurrence Date”, “Incident Type Code”, etc.
  • a journal entry owns properties such as "Credit Account”, “Debit Account”, and "Amount"
  • Figure 9F illustrates a small segment of a domain model for claims handlers in the auto insurance industry in accordance with an embodiment of the present invention.
  • the domain model is created by the instructional designer in a visual editing design tool called the Knowledge Workbench.
  • the designer creates the objects of the domain model using generic entities and properties; that is, not having specific values associated with the entities and properties.
  • FIG. 9G illustrates an instantiated domain model in accordance with an embodiment of the present invention.
  • Creating a domain model in data rather than in code facilitates reuse of the components in multiple applications in multiple domains without code changes.
  • a typical application in the Financial Services domain would have to define classes in code such as 'Customer', 'Account', etc.
  • An Insurance Domain application might have classes such as 'Customer', 'Incident', 'Prior Policy', etc.
  • the analysis logic must be coded to recognize the classes. This requires all logic to be custom-coded for every application; an effort-intensive undertaking that demands a high degree of technical skill.
  • the pu ⁇ ose of the Profiling Component is to analyze the cunent state of a domain and identify specific things that are true about that domain. This information is then passed to the Remediation Component which provides feedback to the student.
  • the Profiling Component analyzes the domain by asking questions about the domain's state, akin to an investigator asking questions about a case. The questions that the Profiler asks are called profiles. For example, suppose there is a task about building a campfire and the student has just thrown a match on a pile of wood, but the fire didn't start. In order to give useful feedback to the student, a tutor would need to know things like: was the match lit?, was the wood wet?, was there kindling in the pile?, etc. These questions would be among the profiles that the Profiling Component would use to analyze the domain. The results of the analysis would then be passed off to the Remediation Component which would use this information to provide specific feedback to the student.
  • a profile is a set of criteria that is matched against the domain.
  • the pu ⁇ ose of a profile is to check whether the criteria defined by the profile is met in the domain.
  • instructional designers create profiles to identify those things that are important to know about the domain for a given task.
  • the set of profiles associated with the cunent task are evaluated to determine which ones are true.
  • Example profiles include:
  • a profile is composed of two types of structures: characteristics and collective characteristics.
  • a characteristic is a conditional (the if half of a rule) that identifies a subset of the domain that is important for determining what feedback to deliver to the student.
  • Example characteristics include:
  • a characteristic's conditional uses one or more atomics as the operands to identify the subset of the domain that defines the characteristic.
  • An atomic only makes reference to a single property of a single entity in the domain; thus the term atomic.
  • Example atomics include:
  • a collective characteristic is a conditional that uses multiple characteristics and/or other collective characteristics as its operands. Collective characteristics allow instructional designers to build richer expressions (i.e., ask more complex questions).
  • Example collective characteristics include:
  • atomics can be used by multiple characteristics
  • characteristics can be used by multiple collective characteristics and profiles
  • collective characteristics can be used by multiple collective characteristics and profiles.
  • Figure 9H illustrates an insurance underwriting profile in accordance with an embodiment of the present invention.
  • the Transformation Component performs calculations on the domain and feeds the results back into the domain for further analysis by the Profiling Component. This facilitates the modeling of complex business systems that would otherwise be very difficult to implement as part of the application.
  • the Transformation Component actually acts on the domain before the Profiling Component does its analysis.
  • the Transformation Component acts as a shell that wraps one or more data modeling components for the pu ⁇ ose of integrating these components into a BusSim application.
  • the Transfonnation Component facilitates the transfer of specific data from the domain to the data modeling component (inputs) for calculations to be performed on the data, as well as the transfer of the results of the calculations from the data modeling component back to the domain (outputs).
  • Figure 10 illustrates a transformation component in accordance with an embodiment of the present invention.
  • the data modeling components could be third party modeling environments such as spreadsheet- based modeling (e.g., Excel, Formulal) or discrete time-based simulation modeling (e.g., PowerSim, VenSim).
  • the components could also be custom built in C++, VB, Access, or any tool that is ODBC compliant to provide unique modeling environments.
  • Using the Transformation Component to wrap a third party spreadsheet component provides an easy way of integrating into an application spreadsheet-based data analysis, created by such tools as Excel.
  • the Transformation Component provides a shell for the spreadsheet so that it can look into the domain, pull out values needed as inputs, performs its calculations, and post outputs back to the domain.
  • the domain would hold the baseline data like how much cash the company has, what its assets and liabilities are, etc.
  • the Transformation Component would be able to look at the data and calculate additional values like cash flow ratios, ROI or NPV of investments, or any other calculations to quantitatively analyze the financial health of the company. Depending on their complexity, these calculations could be performed by pre-existing spreadsheets that a client has aheady spent considerable time developing. Remediation Component
  • the Remediation Component is an expert system that facilitates integration of intelligent feedback into BusSim applications. It has the following features:
  • the Remediation model consists of three primary objects:
  • Concepts are objects that represent real-world concepts that the user will be faced with in the interface. Concepts can be broken into sub-concepts, creating a hierarchical tree of concepts.
  • This tree can be arbitrarily deep and wide to support rich concept modeling. Concepts can also own an arbitrary number of Coach Topics.
  • Coach Topics are objects that represent a discussion topic that may be appropriate for a concept.
  • Coach Topics can own an arbitrary number of Coach Items.
  • Coach Items are items of feedback that may include text, audio, video, URL's, or updates to the
  • Coach Items are owned by Coach Topics and are assembled by the Remediation
  • the BusSim Toolset also includes a set of workbenches that are used by instructional designers to design and build BusSim applications.
  • a workbench is a tool that facilitates visual editing or testing of the data that the BusSim Components use for determining an application's run-time behavior.
  • the BusSim Toolset includes the following workbenches:
  • the Knowledge Workbench is a tool for the creation of domain, analysis and feedback data that is used by the BusSim Components. It has the following features:
  • the interface is intelligent, allowing designers to only paint valid interactions.
  • ⁇ Designer's Task creations are stored in a central repository.
  • the workbench supports check-in / check-out for exclusive editing of a task.
  • the Simulated Student Test Workbench is a tool for the creation of data that simulates student's actions for testing BusSim Component behaviors. It has the following features:
  • the Test Bench generates a simulated application interface based on the Domain Model.
  • the designer can invoke the components to experience the interactions the student will experience in production.
  • the Regression Test Workbench is a tool for replaying and testing of student sessions to aid debugging. It has the following features:
  • the design phase of a BusSim application is streamlined by the use of the Knowledge Workbench.
  • the Knowledge Workbench is a visual editor for configuring the objects of the component engines to control their runtime behavior.
  • the components are based on proven algorithms that capture and implement best practices and provide a conceptual framework and methodology for instructional design.
  • the workbench allows the designer to paint a model of the hierarchy of Concepts that the student will need to master in the activity. This helps the designer organize the content in a logical way.
  • the visual representation of the Concepts helps to communicate ideas to other designers for review.
  • the consistent look and feel of the workbench also contributes to a streamlined Quality Assurance process.
  • standard documentation can be automatically generated for the entire design.
  • the designer adds more detail to the design of the Concept hierarchy by painting in Coach Topics that the student may need feedback on.
  • the designer can associate multiple feedback topics with each Concept.
  • the designer also characterizes each topic as being Praise, Polish, Focus, Redirect or one of several other types of feedback that are consistent with a proven remediation methodology.
  • the designer can then fill each topic with text, video war stories, Web page links, Authorware links, or any other media object that can be delivered to the student as part of the feedback topic.
  • the designer again uses the Knowledge Workbench to configure objects in the Transformation Component.
  • the Transformation Component is used to perform calculations or other analysis of the student's domain.
  • the designer uses the workbench to configure objects in the Profiling Component.
  • the Profiling Component examines the student's domain, looking for conditions that indicate what feedback topics are appropriate for delivery.
  • the Student Simulator Test Workbench allows the designer to exercise the designs. It allows the designer to manipulate the domain as if she were a student. The designer can interact with the simulated interface and invoke the component engines to see the feedback that the student would receive. This capability can also be utilized in a usability test such as a Conference Room Pilot.
  • the workbench supports the design process by allowing the designer great flexibility within the framework of a proven methodology. This allows experienced users to design rich, realistic interactions, and inexperienced users to become competent in a shorter time by learning from the best practices embedded in the workbench. This greatly diminishes the 'blue sky' design problem. Also, since the designs can be tested prior to application construction, there is reduced rework after testing. Lastly, the visual knowledge representation enhances communication within the design team and greatly streamlines the QA process.
  • the toolset greatly reduces effort during functionality testing.
  • the key driver of the effort reduction is that the components can automatically track the actions of the tester without the need to add code support in the application. Whenever the tester takes an action in the interface, it is reported to the domain model. From there it can be tracked in a database. Testers no longer need to write down their actions for use in debugging; they are automatically written to disk. There is also a feature for attaching comments to a tester's actions. When unexpected behavior is encountered, the tester can hit a control key sequence that pops up a dialog to record a description of the enant behavior.
  • the components are deployed to the student's platform. They provide simulated team member and feedback functionality with sub-second response time and enor-free operation. If the client desires it, student tracking mechanisms can be deployed at runtime for evaluation and administration of students. This also enables the isolation of any defects that may have made it to production.
  • Figure 11 illustrates the use of a toolbar to navigate and access application level features in accordance with an embodiment of the present invention.
  • a student uses a toolbar to navigate and also to access some of the application-level features of the application.
  • the toolbar is the inverted L-shaped object across the top and left of the interface.
  • the top section of the toolbar allows the user to navigate to tasks within the cunent activity.
  • the left section of the toolbar allows the student to access other features of the application, including feedback.
  • the student can have his deliverables analyzed and receive feedback by clicking on the Team button. In this task, the student must journalize twenty-two invoices and other source documents to record the flow of budget dollars between internal accounts.
  • Journalizing is the process of recording journal entries in a general ledger from invoices or other source documents during an accounting period. The process entails creating debit and balancing credit entries for each document. At the completion of this process, the general ledger records are used to create a trial balance and subsequent financial reports.
  • ICAT Intelligent Coaching Agent Tool
  • the ICAT is packaged with a user-friendly workbench, so that it maybe reused to increase productivity on projects requiring a similar rule-based data engine and repository.
  • the Intelligent Coaching Agent Tool is a suite of tools— a database and a Dynamic Link Library (DLL) run-time engine — used by designers to create and execute just-in-time feedback of Goal Based training. Designers write feedback and rules in the development tools. Once the feedback is set, the run-time engine monitors user actions, fires rules and composes feedback which describes the business deliverable.
  • DLL Dynamic Link Library
  • the remediation model used within ICAT dynamically composes the most appropriate feedback to deliver to a, student based on student's previous responses.
  • the ICAT model is based on a theory of feedback which has been proven effective by pilot results and informal interviews.
  • the model is embodied in the object model and algorithms of the ICAT. Because the model is built into the tools, all feedback created with the tool will conform to the model.
  • the ICAT plays two roles in student training. First, the ICAT is a teaching system, helping students to fully comprehend and apply information. Second, ICAT is a gatekeeper, ensuring that each student has mastered the material before moving on to additional information. m. The Functional Definition of the ICAT
  • the ICAT is a self contained module, separate from the application. Separating the ICAT from the application allows other projects to use the ICAT and allows designers to test feedback before the application is complete.
  • the ICAT Module is built on six processes which allow a student to interact effectively with the interface to compose and deliver the appropriate feedback for a student's mistakes.
  • the ICAT development methodology is a seven step methodology for creating feedback.
  • the methodology contains specific steps, general guidelines and lessons learned from the field. Using the methodology increases the effectiveness of the feedback to meet the educational requirements of the course.
  • the processes each contain a knowledge model and some contain algorithms. Each process has specific knowledge architected into its design to enhance remediation and teaching.
  • the tools and run-time engine in accordance with an embodiment of the present invention include expert knowledge of remediation. These objects include logic that analyzes a student's work to identify problem areas and deliver focused feedback. The designers need only instantiate the objects to put the tools to work. Embodying expert knowledge in the tools and engine ensures that each section of a course has the same effective feedback structure in place.
  • a GBS Goal-Based Scenario
  • IPS Integrated Performance Support
  • a GBS allows students to learn in a comprehensive simulated environment. Students work in a simulated environment to accomplish real world tasks, and when they make mistakes, remediation is provided to help identify and conect the mistakes. The hands-on experience of the simulated environment and the timely remediation account for the high retention rate from subjects presented utilizing a system in accordance with an embodiment of the present invention.
  • a system in accordance with an embodiment of the present invention can be used in conjunction with an IPS to help users develop deliverables. If a customer service representative (CSR) is completing a form while conducting a phone conversation, the ICAT can be used to observe how the task is completed to provide a live analysis of mistakes.
  • CSR customer service representative
  • a file structure in accordance with an embodiment of the present invention provides a standard system environment for all applications in accordance with an embodiment of the present invention.
  • a development directory holds a plurality of sub-directories.
  • the content in the documentation directory is part of a separate installation from the architecture. This is due to the size of the documentation directory. It does not require any support files, thus it may be placed on a LAN or on individual computers.
  • the development directory has an __Arch, _Tools, _Utilities, Documentation, QED, and XDefault development directory.
  • Each folder has its own directory structure that is inter-linked with the other directories. This structure must be maintained to assure consistency and compatibility between projects to clarify project differences, and architecture updates.
  • the _Arch directory stores many of the most common parts of the system architecture. These files generally do not change and can be reused in any area of the project. If there is common visual basic code for applications that will continuously be used in other applications, the files will be housed in a folder in this directory.
  • the sub-directories in the _Arch directory are broken into certain objects of the main project.
  • Object in this case refers to parts of a project that are commonly refened to within the project.
  • modules and classes are defined here, and the directory is analogous to a library of functions, APIs, etc... that do not change.
  • the IcaObj directory stores code for the Intelligent Coaching Agent (ICA).
  • the InBoxObj directory stores code for the InBox part of the project and so on.
  • the file structure uses some primary object references as file directories.
  • the IcaObj directory is a component that contains primary objects for the ICA such as functional forms, modules and classes.
  • the BrowserObj directory contains modules, classes and forms related to the browser functionality in the architecture.
  • the HTMLGlossary directory contains code that is used for the HTML reference and glossary component of the architecture.
  • the IcaObj directory contains ICA functional code to be used in an application. This code is instantiated and enhanced in accordance with an embodiment of the present invention.
  • the InBoxObj directory contains code pertaining to the inbox functionality used within the architecture. Specifically, there are two major components in this architecture directory. There is a new .ocx control that was created to provide functionality for an inbox in the application. There is also code that provides support for a legacy inbox application.
  • the PracticeObj directory contains code for the topics component of the architecture. The topics component can be implemented with the HTMLGlossary component as well.
  • the QmediaObj directory contains the components that are media related.
  • An example is the QVEDctrl.cls.
  • the QVIDctrl is the code that creates the links between QVID files in an application and the system in accordance with an embodiment of the present invention.
  • the SimObj directory contains the Simulation Engine, a component of the application that notifies the tutor of inputs and outputs using a spreadsheet to facilitate communication.
  • the StaticObj directory holds any component that the application will use statically from the rest of the application.
  • the login form is kept in this folder and is used as a static object in accordance with an embodiment of the present invention.
  • the SysDynObj directory contains the code that allows the Systems Dynamics Engine (Powersim) to pass values to the Simulation Engine and return the values to the tutor.
  • the VBObj directory contains common Visual Basic objects used in applications. For example the NowWhat, Visual Basic Reference forms, and specific message box components are stored in this folder.
  • the _Tools directory contains two main directories. They represent the two most used tools in accordance with an embodiment of the present invention.
  • the two directories provide the code for the tools themselves. The reason for providing the code for these tools is to allow a developer to enhance certain parts of the tools to extend their ability. This is important for the cunent project development and also for the growth of the tools.
  • the Icautils directory contains a data, database, default, graphics, icadoc, and testdata directory.
  • the p pose of all of these directories is to provide a secondary working directory for a developer to keep their testing environment of enhanced Icautils applications separate from the project application. It is built as a testbed for the tool only. No application specific work should be done here. The pmpose of each of these directories will be explained in more depth in the project directory section.
  • the TestData folder is unique to the _Tools/ICAUtils directory. It contains test data for the regression bench among others components in ICAUtils.
  • the Utilities directory holds the available utilities that a Business Simulation project requires for optimal results. This is a repository for code and executable utilities that developers and designers may utilize and enhance in accordance with an embodiment of the present invention. Most of the utilities are small applications or tools that can be used in the production of simulations which comprise an executable and code to go with it for any enhancements or changes to the utility. If new utilities are created on a project or existing utilities are enhanced, it is important to notify the managers or developers in charge of keeping track of the Business Simulation assets. Any enhancements, changes or additions to the Business Simulation technology assets are important for future and existing projects. Documentation
  • a Documentation directory is used to store pertinent documentation.
  • the documentation directory is structured as follows. Most of the directories are labeled after the specific information held within them. The following is a list of all the documentation directories and a description of what is contain in each.
  • Ref Website This directory contains The Business Simulation Reference website, which is a general reference for many things. If the website has not been set up for users on a LAN or website, all you need to do is go into the root directory of website and double click on index.htm. This is the main page for the site.
  • This directory contains any documentation on classes and modules that are used in the architecture. For example there are documents here on the ICAMeeting class, the Inbox class etc.
  • Database - This directory contains any documents describing the databases that are included and- used in the Architecture.
  • ICAObj overview doc contains a description of the model and each element in the database.
  • HTML Component - This directory contains relevant documentation about the HTML part of the architecture.
  • This directory should contain the documents that describe the process of the application or related information.
  • ReferenceApp - This directory contains documents with descriptions and views of the reference app. (QED) for explanation and documentation. Testing conditions are stored in the Testing directory.
  • Standards&Templates This directory contains any type of architecture relevant coding standard documents or templates that a developer is required to follow.
  • UserGuides This directory has 6 sub-directories. Each one of these sub-directories contains user guides for a given tool or component in accordance with an embodiment of the present invention which include user guides for the architecture, the Tutor Suite, ICA Utilities, the simulation Engine and the System Dynamics Engine. There is also a directory for other documentation that contains user guides for any other tools or code like third party controls etc.
  • This directory contains the WF_Develop.doc which includes the workflow documentation for an application.
  • the sample project directory, QED has the same structure that a real project would be designed after.
  • the QED directory has all custom architecture code, databases, spreadsheets, and any other application specific files stored in it.
  • the QED project directory stores a Design and SrcVB directory.
  • the Design directory contains all relevant files for a designer.
  • the SrcVB directory is used for a developer.
  • the root directory of the Design and SrcVB directory contain a few important files to note. Both have two .rtf files, a few log files and an .ini file.
  • the .rtf files are the feedback that is output from the tutor, the logs are also output from the tutor and the .ini file is for ICAUtils initialization.
  • the design directory has three subdirectories that contain a data directory, which stores .xls files, sim models, and any other important data like html and video. It also has a database directory that holds any relevant databases for development and application use.
  • the last directory is the icadoc directory which includes all .tut files or .ica files, which are both created with the tutor.
  • the SrcVB directory stores all of the directories previously described. The reason for duplicating the data and database directories is to assure that a developer does not interfere with the designer's files. The developer tends to not do as much design work and can easily corrupt files. This duplication of directories provides a safer environment for the developer to test in. As was mentioned above, the developer tends to have a lot more to do with the application build than the design so there needs to be more content in the SrcVB directory.
  • the SrcVB directory also contains an .exe and .vbp file which are created in a developers visual basic application. The following are directories found in the SrcVB directory that are not found in the Design directory followed by a short definition:
  • the _CustomArch directory contains any application specific architecture. Look in the QED folder for an example.
  • the _CustomDistribution directory contains any files that need to be distributed with the application.
  • the Default directory contains any backup files that might need to be copied and reused later. Some files occasionally are corrupted and need to be replaced.
  • the Fonts directory contains application specific font libraries.
  • the Graphics directory contains any relevant graphics for the application.
  • the Help directory contains all files for a help reference layer in the application. This can be implemented in many ways but most commonly in an HTML form.
  • the Saved directory is for saved information that is produced by the application. This can be used for saving student information or saving temporary information for the application steps.
  • the StudentData directory is for storing any relevant student data, lists of students, their personal information or any relevant student data that needs to be saved.
  • the XDefault Development environment is used to provide a shell for any new project. A developer would rename this directory as an acronym of the project.
  • QED is the default for the installation sample application.
  • the XDefault development directory is a shell and serves as a building block for a startup project. A good idea is to use the QED sample application and build the XDefault Development project with the sample code in QED. Shared Development
  • the last directory to be mentioned is the shared development directory which is placed on a LAN or central network area and is shared by all designers and developers of a project to assure that files in the project are up to date, managed properly and appropriately synchronized.
  • a source control program is used to restrict access to a file to one application at a time.
  • the ICAT has a model of remediation architected into the system in accordance with an embodiment of the present invention. Feedback should help students complete tasks and learn the underlying concepts. To achieve this goal, the ICAT reviews student's work with the following objectives in mind.
  • the tutor When identifying problems, the tutor needs to prompt the student to reflect on a problem and start to point the student towards the answer. The tutor should not tell the student the answer, but instead should attempt to provide an appropriate answer or give the student a question to think about.
  • An embodiment of the present invention utilizes feedback to address the balancing task.
  • the theory is centered on the idea of severity. Severe enors require severe feedback while mild enors require mild feedback. If a student writes a paper on the wrong subject, a human tutor will spend little time reviewing the paper, but instead, identify it as a serious mistake and ask the student to rewrite the paper. If the student simply misses one paragraph of the argument, then the tutor will focus the student on that paragraph. Finally, if the paper is conect except for a couple of spelling mistakes, the tutor will point out the specific mistakes and ask the student to conect them. The point is that because a tutor and a student do not want to waste each others' time, they will match the severity of the enor with the severity of the feedback.

Abstract

L'invention concerne un procédé destiné à analyser une performance dans une simulation de formation basée sur un réseau. On utilise un réseau afin de présenter une simulation de formation à un utilisateur afin de réaliser un objectif de formation (502). On intègre dans l'entraînement simulé, par utilisation du réseau, un information qui aide à motiver l'utilisateur (504) pour qu'il réalise l'objectif. Les progrès de l'utilisateur dans la réalisation de l'objectif sont mesurés par utilisation du réseau (506). De façon à obtenir une réalisation motivante supplémentaire de l'objectif, l'utilisateur reçoit une réaction par l'intermédiaire du réseau (508). Les progrès de l'utilisateur par rapport à l'objectif sont rapportés postérieurement à l'employeur de l'utilisateur qui utilise le réseau (510).
PCT/US2001/003420 2000-02-23 2001-02-02 Procede d'analyse d'un employe basee sur des resultats d'un enseignement par simulation d'entreprise WO2001063517A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP01905361A EP1269365A4 (fr) 2000-02-23 2001-02-02 Procede d'analyse d'un employe basee sur des resultats d'un enseignement par simulation d'entreprise
AU2001233251A AU2001233251A1 (en) 2000-02-23 2001-02-02 Method for an employee analysis based on results of an education business simulation
CA002401616A CA2401616A1 (fr) 2000-02-23 2001-02-02 Procede d'analyse d'un employe basee sur des resultats d'un enseignement par simulation d'entreprise

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US51122000A 2000-02-23 2000-02-23
US09/511,340 US6944596B1 (en) 2000-02-23 2000-02-23 Employee analysis based on results of an education business simulation
US09/511,340 2000-02-23
US09/511,220 2000-02-23

Publications (1)

Publication Number Publication Date
WO2001063517A1 true WO2001063517A1 (fr) 2001-08-30

Family

ID=27057156

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/003420 WO2001063517A1 (fr) 2000-02-23 2001-02-02 Procede d'analyse d'un employe basee sur des resultats d'un enseignement par simulation d'entreprise

Country Status (4)

Country Link
EP (1) EP1269365A4 (fr)
AU (1) AU2001233251A1 (fr)
CA (1) CA2401616A1 (fr)
WO (1) WO2001063517A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007062450A1 (fr) * 2005-12-02 2007-06-07 University Of New England Evaluation de services educatifs
US9720707B1 (en) 2016-12-15 2017-08-01 Accenture Global Solutions Limited Generating a set of user interfaces
US10339502B2 (en) 2015-04-06 2019-07-02 Adp, Llc Skill analyzer
US10984361B1 (en) 2016-12-15 2021-04-20 Accenture Global Solutions Limited Providing a set of social communication channels to a set of client devices
US11126949B1 (en) 2016-12-15 2021-09-21 Accenture Global Solutions Limited Generating a user interface for an employee

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4828977A (en) * 1988-07-28 1989-05-09 General Electric Company Conveyor line training simulator system and method
US4928704A (en) * 1989-01-31 1990-05-29 Mindcenter Corporation EEG biofeedback method and system for training voluntary control of human EEG activity
JPH0895938A (ja) * 1994-09-28 1996-04-12 Nippon Telegr & Teleph Corp <Ntt> 時系列データの学習方法及び時系列データの学習装置
US5863208A (en) * 1996-07-02 1999-01-26 Ho; Chi Fai Learning system and method based on review
JP2000267554A (ja) * 1999-03-16 2000-09-29 Gakushu Kankyo Kenkyusho:Kk 学習支援装置および学習支援方法、並びに記録媒体

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4828977A (en) * 1988-07-28 1989-05-09 General Electric Company Conveyor line training simulator system and method
US4928704A (en) * 1989-01-31 1990-05-29 Mindcenter Corporation EEG biofeedback method and system for training voluntary control of human EEG activity
JPH0895938A (ja) * 1994-09-28 1996-04-12 Nippon Telegr & Teleph Corp <Ntt> 時系列データの学習方法及び時系列データの学習装置
US5863208A (en) * 1996-07-02 1999-01-26 Ho; Chi Fai Learning system and method based on review
US6212358B1 (en) * 1996-07-02 2001-04-03 Chi Fai Ho Learning system and method based on review
JP2000267554A (ja) * 1999-03-16 2000-09-29 Gakushu Kankyo Kenkyusho:Kk 学習支援装置および学習支援方法、並びに記録媒体

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
DATABASE ABI/INFORM(R) [online] ROBERTS: "Training via the desktop", XP002940974, retrieved from 01688293 accession no. Dialog Database accession no. 03-39283 *
HR MAGAZINE, vol. 43, no. 9, August 1998 (1998-08-01), pages 98 - 104 *
See also references of EP1269365A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007062450A1 (fr) * 2005-12-02 2007-06-07 University Of New England Evaluation de services educatifs
US10339502B2 (en) 2015-04-06 2019-07-02 Adp, Llc Skill analyzer
US9720707B1 (en) 2016-12-15 2017-08-01 Accenture Global Solutions Limited Generating a set of user interfaces
US10984361B1 (en) 2016-12-15 2021-04-20 Accenture Global Solutions Limited Providing a set of social communication channels to a set of client devices
US11126949B1 (en) 2016-12-15 2021-09-21 Accenture Global Solutions Limited Generating a user interface for an employee

Also Published As

Publication number Publication date
EP1269365A1 (fr) 2003-01-02
EP1269365A4 (fr) 2009-09-09
CA2401616A1 (fr) 2001-08-30
AU2001233251A1 (en) 2001-09-03

Similar Documents

Publication Publication Date Title
US6944596B1 (en) Employee analysis based on results of an education business simulation
US6023692A (en) Goal based tutoring system with behavior to control flow of presentation
US6493690B2 (en) Goal based educational system with personalized coaching
US6611822B1 (en) System method and article of manufacture for creating collaborative application sharing
US7047279B1 (en) Creating collaborative application sharing
US6016486A (en) System method and article of manufacture for a goal based system utilizing an activity table
US6535861B1 (en) Goal based educational system with support for dynamic characteristics tuning using a spread sheet object
US6018732A (en) System, method and article of manufacture for a runtime program regression analysis tool for a simulation engine
US6003021A (en) System, method and article of manufacture for a simulation system for goal based education
US5987443A (en) System, method and article of manufacture for a goal based educational system
US6782374B2 (en) System, method and article of manufacturing for a runtime program analysis tool for a simulation engine
US6073127A (en) System, method and article of manufacture for a goal based system with dynamic feedback information
US6067538A (en) System, method and article of manufacture for a simulation enabled focused feedback tutorial system
US6101489A (en) System, method and article of manufacture for a goal based system utilizing a time based model
US7280991B1 (en) Creating collaborative simulations for creating collaborative simulations with multiple roles for a single student
US6134539A (en) System, method and article of manufacture for a goal based education and reporting system
US6067537A (en) System, method and article of manufacture for a goal based educational system with support for dynamic personality feedback
US6064998A (en) System, method and article of manufacture for a simulation engine with an expert system example processing engine
US6026386A (en) System, method and article of manufacture for a goal based educational system with support for dynamic media control
US6032141A (en) System, method and article of manufacture for a goal based educational system with support for dynamic tailored feedback
US6023691A (en) Goal based stimulator utilizing a spreadsheet architecture
US6029159A (en) System, method and article of manufacture for a simulation enabled accounting tutorial system
US6125358A (en) System, method and article of manufacture for a simulation system for goal based education of a plurality of students
US6029156A (en) Goal based tutoring system with behavior to tailor to characteristics of a particular user
US6085184A (en) System, method and article of manufacture for a dynamic toolbar in a tutorial system

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CR CU CZ DE DK DM DZ EE ES FI GB GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2401616

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 2001905361

Country of ref document: EP

WWP Wipo information: published in national office

Ref document number: 2001905361

Country of ref document: EP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

NENP Non-entry into the national phase

Ref country code: JP