WO2019040613A1 - System for dynamically provisioning cyber training environments - Google Patents

System for dynamically provisioning cyber training environments Download PDF

Info

Publication number
WO2019040613A1
WO2019040613A1 PCT/US2018/047509 US2018047509W WO2019040613A1 WO 2019040613 A1 WO2019040613 A1 WO 2019040613A1 US 2018047509 W US2018047509 W US 2018047509W WO 2019040613 A1 WO2019040613 A1 WO 2019040613A1
Authority
WO
WIPO (PCT)
Prior art keywords
mission
user
plugin
click
objective
Prior art date
Application number
PCT/US2018/047509
Other languages
French (fr)
Inventor
Laura Lee
Raymond C. PROUTY
Joseph T. KOWTKO
Ryan Pressley
Original Assignee
Circadence Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Circadence Corporation filed Critical Circadence Corporation
Publication of WO2019040613A1 publication Critical patent/WO2019040613A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/10—Office automation; Time management
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/02—Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00—Simulators for teaching or training purposes

Definitions

  • training systems are not flexible and provide limited training benefits.
  • existing training systems are designed to implement fixed training sessions. That is, these training systems include one or more predesigned or fixed training applications. The training system simply implements that single fixed training application or selects from one of a small set of fixed training applications. Thus, students see the same training environments over and over. If the operator desires to present students with a different training session or environment, an entirely new training application must be built and loaded into the training system.
  • This "select from fixed training sessions" configuration is consistent with the goal of existing training sessions: to create a training session in which a student practices or implements one or more specific tasks.
  • the training is used to train the student on a particular task and to increase their proficiency in implementing the task.
  • each cyber threat is very different.
  • a student's ability to perform a particular designated task is insufficient in helping the student understand when to perform the task or how to use it in conjunction with other tasks or techniques in order to address a cyber threat.
  • a dynamic, virtual network training system and method are needed to provide an isolated and controlled network environment with the level of complexity needed to train experts how to rapidly respond to cyber-attacks, terrorism, and cyber-crime, and how to stop them.
  • a method and system are needed to generate a plurality of varied or variable training missions, such as having different environments and objectives.
  • Embodiments of the invention comprise methods and systems which allow a user or operator to easily create cyber-training environments for use in a cyber-training system, and preferably methods and systems for designing and configuring specific training mission hardware and software environments.
  • the environments are configured as missions.
  • the missions may have a plurality of features, such as training objectives, a mission storyline, a mission order and mission objectives, relative to a mission environment.
  • the mission environment comprises a virtual environment, such as defined by a virtual network having virtual machines or devices.
  • aspects of the invention comprise hardware and software which are uniquely configured to automate the creating of such cyber-training environments or missions, thus solving problems with and providing advantages over existing technology which requires cyber training platforms to be custom design or configured with a static configuration.
  • FIGURE 1 illustrates a graphic user interface according to embodiments of the present invention.
  • FIGURE 2 illustrates a system architecture according to embodiments of the present invention.
  • FIGURE 3 illustrates how a controller will operate in different states according to embodiments of the present invention.
  • FIGURE 4 A illustrates a block diagram of a system according to embodiments of the present invention.
  • FIGURE 4B illustrates a software architecture according to embodiments of the present invention.
  • FIGURES 5 A, 5B and 5C illustrate changing plug-in states when switching to a different mission according to embodiments of the present invention.
  • FIGURE 6 illustrates communication between a user and the server using Webswing according to embodiments of the present invention.
  • FIGURE 7 illustrates a create new mission top level graphic according to embodiments of the present invention.
  • FIGURE 8 illustrates a graphic user interface for creating new missions according to embodiments of the present invention.
  • FIGURE 9 illustrates a graphic user interface for opening an existing mission according to embodiments of the present invention.
  • FIGURE 10 illustrates a save changes dialog according to embodiments of the present invention.
  • FIGURE 11 illustrates a graphic user interface for copying a mission according to embodiments of the present invention.
  • FIGURE 12 illustrates an advanced mission setting dialog according to embodiments of the present invention.
  • FIGURE 13 illustrates an export mission summary dialog according to embodiments of the present invention.
  • FIGURE 14 illustrates an export files created interface according to embodiments of the present invention.
  • FIGURE 15 illustrates a storyline video or text according to embodiments of the present invention.
  • FIGURE 16 illustrates a storyline interface according to embodiments of the present invention.
  • FIGURE 17 illustrates a mission orders interface in a game environment according to embodiments of the present invention.
  • FIGURE 18 illustrates a define mission order dialog according to embodiments of the present invention.
  • FIGURE 19 illustrates a situation editor according to embodiments of the present invention.
  • FIGURE 20 illustrates a mission order page settings dialog according to embodiments of the present invention.
  • FIGURE 21 illustrates a mission objectives interface according to embodiments of the present invention.
  • FIGURE 22 A illustrates a mission objectives graphic user interface or tab according to embodiments of the present invention.
  • FIGURE 22B illustrates a search for skill interface according to embodiments of the present invention.
  • FIGURE 23 illustrates a skills interface according to embodiments of the present invention.
  • FIGURE 24 illustrates an umpire check dialog according to embodiments of the present invention.
  • FIGURE 25 illustrates a solutions and hints user interface according to embodiments of the present invention.
  • FIGURE 26 illustrates a screenshot defining solutions and hints according to embodiments of the present invention.
  • FIGURE 27 illustrates a define solution interface according to embodiments of the present invention.
  • FIGURE 28 illustrates an add hint interface according to embodiments of the present invention.
  • FIGURE 29 illustrates a network map screenshot according to embodiments of the present invention.
  • FIGURE 30 illustrates a network map options interface according to embodiments of the present invention.
  • FIGURE 31 illustrates a screenshot of key terrain according to embodiments of the present invention.
  • FIGURE 32 illustrates a screenshot of a rubberband zoom according to embodiments of the present invention.
  • FIGURES 33A and 33B illustrate icons that can be added to the network map according to embodiments of the present invention.
  • FIGURE 34 illustrates a mouseover feedback interface according to embodiments of the present invention.
  • FIGURE 35 illustrates editable properties dialogs according to embodiments of the present invention.
  • FIGURE 36A illustrates an arrange item interface according to embodiments of the present invention.
  • FIGURE 36B illustrates lock item interface according to embodiments of the present invention.
  • FIGURE 37 illustrates a network graph map area according to embodiments of the present invention.
  • FIGURE 38A illustrates a defining events interface according to embodiments of the present invention.
  • FIGURE 38B illustrates an add events interface according to embodiments of the present invention.
  • FIGURE 39 illustrates a step for id dialog according to embodiments of the present invention.
  • FIGURE 40A-40D illustrates quizzes interfaces according to embodiments of the present invention.
  • FIGURE 41 illustrates a checklist tab or interface according to embodiments of the present invention.
  • FIGURE 42 illustrates a define mission dialog according to embodiments of the present invention.
  • FIGURE 43 illustrates a mission verification summary dialog according to embodiments of the present invention.
  • FIGURE 44 illustrates an advanced mission settings dialog according to embodiments of the present invention.
  • FIGURE 45 illustrates an export summary dialog according to embodiments of the present invention.
  • FIGURE 46 illustrates a mission development methodology walkthrough according to embodiments of the present invention.
  • FIGURE 47 illustrates a mission development methodology walkthrough according to embodiments of the present invention.
  • FIGURE 48 illustrates a welcome page according to embodiments of the present invention.
  • FIGURE 49 illustrates a user pre-requisite skills interface according to embodiments of the present invention.
  • FIGURE 50 illustrates a key terrain interface according to embodiments of the present invention.
  • FIGURE 51 illustrates a mission characteristics interface according to embodiments of the present invention.
  • FIGURE 52 illustrates a training outcomes interface according to embodiments of the present invention.
  • FIGURE 53 illustrates an associated mission files interface according to embodiments of the present invention.
  • FIGURE 54 illustrates a mission order page interface according to embodiments of the present invention.
  • FIGURE 55 illustrates a mission objectives interface according to embodiments of the present invention.
  • FIGURE 56 illustrates a solutions and hints interface according to embodiments of the present invention.
  • FIGURE 57 illustrates an edit text hint interface according to embodiments of the present invention.
  • FIGURE 58 illustrates a network map page interface according to embodiments of the present invention.
  • FIGURE 59 illustrates an events interface according to embodiments of the present invention.
  • FIGURE 60 illustrates a questions and answers interface according to embodiments of the present invention.
  • FIGURE 61 illustrates a quizzes page interface according to embodiments of the present invention.
  • FIGURE 62 illustrates a mission library interface according to embodiments of the present invention.
  • FIGURE 63 illustrates a cyber range backbone of Internet services, user emulation, and auto-inject mission modules that support the training environment according to embodiments of the present invention.
  • FIGURE 64 illustrates a small core mission module for eight teams that support the training environment according to embodiments of the present invention.
  • FIGURE 65 illustrates a medium core mission module that support the training environment according to embodiments of the present invention.
  • Embodiments of the invention comprise methods and system for creating or generating cyber environments, and preferably methods and systems for designing and configuring specific training mission hardware and software environments.
  • Figure 1 illustrates one embodiment of aspects of the present invention, wherein the invention is referred to as the "Orion" system or tool.
  • a mission design methodology In one embodiment, this methodology is an iterative, collaborative process embodied in the tool that defines everything for a complete cyber training environment.
  • Network templates The templates define a baseline environment complete with working definitions of the network services/protocols and unique overlays (#3) to include variation and threat actor (it creates full orchestration to support dynamic provisioning).
  • Ability to define variability (services_0) layer as part of the design process.
  • Event driven process for two-sided training events (offense v defense).
  • the system comprises an Orion controller 48, such as associated with a server 40, and one or more clients 41, such as implemented at computing devices such as laptop or desktop computers or the like.
  • the system is preferably configured so that standalone clients will run on any machine that supports Java8, so that the server components will run on any machine that supports Java, so that the web client will run on any machine that support a modern browser, and wherein the system tool is implemented as a state machine (such as to simplify the design process and allow a global logger to be maintained).
  • the system could be configured to operate in other environments.
  • the system may, as described in more detail below, implement a global logger 42, include a plurality of plug-ins 44, include one or more memory devices for storing machine-readable code for execution by the Orion controller 48 (for implementing various of the functionality described herein), and include one or more databases, such as a mission database 46 for storing missions or mission information.
  • the plug-ins 44 may include a storyline plugin, a mission setup plugin, a mission order plugin, a solution plugin, a network map plugin, an events plugin, an Athena Questions and answers plugin, a quizzes plugin and a mission library plugin.
  • the system server(s) 40 are preferably configured to communicate with one or more client devices 41.
  • the client devices 41 may display one or more user interfaces, such as the one or more graphical user interfaces described below, such as to provide information to a user and receiving information from a user, including in configuring the various parameters of a mission, etc.
  • the mission database 46 can include a plurality of preconfigured missions as well as missions that have been saved by mission developers, such that the mission database 46 may grow over time.
  • a mission designer can design a mission from scratch or start from one of the preconfigured or saved missions and use that mission without change or after modifying the mission.
  • the Orion system is a cyber mission builder and provides the capability for creating and editing of a new cyber scenario or mission that can be used in a training environment or platform. For example, using the system, a user can create scenarios involving how to defend a bank' s network from a financial attack using ransomware or a scenario where a hacker tries to steal credit card data. Orion is used to develop new missions and content. It is used by mission designers, not end users of cyber training.
  • Figure 2 depicts the system of the invention and how it fits within the larger ecosystem of a gamified training system, such as a game system referred to as "Ares".
  • a gamified training system such as a game system referred to as "Ares”.
  • an Ares gaming system 22 has the components of a game environment, a cyber security domain, an artificial intelligence (AI) components, and a virtualized network.
  • the virtualized network comprises a virtual environment of virtual machines, plus software tools and the like.
  • Ares starts a game in a phase I where the user selects a specific mission, skill level, learns the mission objectives and uses the interface to perform any necessary mission preparation including taking proficiency quizzes. The game then proceeds to a phase II where the user is immersed in the game trying to accomplish the mission objectives. After the mission play is over, the game transitions to the assessment phase where the player learns what happened and what they did (strengths/weaknesses). This process includes a complete walk through of an after-action report covering the mission
  • the Ares game system may include an automated advisor, such as an automated artificial intelligence advisor (such as under the name "Athena"). This advisor may, for example, be capable of providing a player assistance during a game, such as by answering player questions or providing hints, among other things.
  • the Ares game system may also include an "umpire" (which may be implemented as part of the advisor, or separately therefrom). The umpire may, for example, score the player based upon their actions (and/or inactions) during the mission.
  • the Orion system of the present invention preferably allows an operator to create or generate different missions (e.g., objective- based assignments within a defined environment, such as a cyber environment which includes a virtualized network).
  • missions e.g., objective- based assignments within a defined environment, such as a cyber environment which includes a virtualized network.
  • the high-level architecture of the system is a plugin paradigm using a state-machine life-cycle.
  • Each plugin 44 is an independent wizard for adding/modifying one aspect of the mission.
  • the plugins 44 are maintained by a controller which messages state changes with each plugin. There is a general method for communications between plugins.
  • a global logger 42 is maintained.
  • a top-level state machine is implemented that supports the mission building guidelines.
  • the purpose of the TLSM is to provide a concrete implementation of the guidelines. It will provide a means to collect and store artifacts that are produced outside of the Orion environment (e.g., mission video, adversary attack motivations, mission art, ...)
  • the status of the TLSM is maintained by the Orion controller and is visible in a panel on the graphical user interface (GUI).
  • GUI graphical user interface
  • the Orion or system controller 48 provides the glue for the application - keeping track of what is going on and communicating to individual plugins 44 as needed.
  • Table 1 describes the Orion System Controller States. Figure 3 and Table 1 illustrate how the controller 48 will operate within these states.
  • a tab is generated for the plugin on the main window as in Fig. 1.
  • Idle There are no pending user commands to process at the main window or main menu level.
  • the controller will create a data directory for each plugin under the main mission directory.
  • Orion has some data which is "shared" among the plugins. To maintain the integrity of the data, the data is preferably maintained by a (set of) global data singleton(s). In one embodiment, the following data is included:
  • the master data is preferably read in when the singleton is instantiated. At this time, the singleton will check to see if there are updates to the master from the mission design databases and use those.
  • Any data which may be editable within Orion is accessed in a synchronized manner.
  • each plugin tab 12 indicates the state of that plugin, such as shown in table 2. These plugin states are used when switching to a different mission to determine action needed, such as illustrated in Figure 5 a, 5b, and 5c (wherein dark gray boxes indicate that a dialog box is invoked).
  • the plugins are instantiated by the Orion controller when a mission is opened or created.
  • a general method of plugin to plugin communication is provided.
  • this communication is a messaging mechanism similar to a post office.
  • One plugin posts a message (e.g., mission objectives when objective added) and other plugin(s) gets the post and processes it as appropriate (e.g., solution and hints to add another page for the new objective).
  • a message e.g., mission objectives when objective added
  • other plugin(s) gets the post and processes it as appropriate (e.g., solution and hints to add another page for the new objective).
  • the processing may use threads with a behind-the-scenes wait- notify mechanism.
  • messages are not received on the GUI thread, so no direct drawing occurs as a result.
  • the PluginMessage class provides the structure for the message to be sent.
  • a message consists of: (1) Source (the plugin that sent the message); (2) Source Name (the name on the plugin tab; (3) Message ID (defined in PluginMessageType enum); and (4) Pay load (any object to send - simple or complex).
  • PluginMessageType enum contains the IDs for the messages. Current values may be: (1) Objectives and (2) Text.
  • Every plugin has a method postMessage(MessageType type, Object payload). This creates a message with the plugin as the source, the plugin tab name as the source name, and the type and payload given.
  • the log plugin simply logs every message it gets - keeping a record of messages posted in the log tab.
  • Every plugin implements the interface IMessageProcessor. It contains one method processMessage(PluginMessage message). The default implementation will do nothing. Override processMessage to be able to handle message(s). For example:
  • processMessage is called on a separate (non-GUI) thread.
  • Webswing When Orion is configured as a thin client using the web browser to deliver Orion capability, Webswing is used to present the Orion experience to the user. Webswing provides a Javascript client, which mirrors a running Java app (such as Orion) on the server. Webswing captures Swing draw calls, and sends it to the Javascript client via a secure websocket connection, which then mirrors the draw call.
  • Javascript client which mirrors a running Java app (such as Orion) on the server.
  • Webswing captures Swing draw calls, and sends it to the Javascript client via a secure websocket connection, which then mirrors the draw call.
  • Figure 6 depicts the communications between the user and the server using Webswing.
  • TLS vl.2. This is between the Mission Server and Orion.
  • the connections utilize both client and server authentication using certificates signed by a recognized Certificate Authority.
  • Basic information for the mission is defined when the mission is created. These parameters include one or more of:
  • this information may be presented via a game environment on a world map, where the user can select the mission to play.
  • Save Changes dialog 100 such as illustrated in Figure 10, may be displayed.
  • control parameters for executing the Orion mission are defaulted to typical values and can be modified by the Mission Designer.
  • Default Location If you have multiple locations where your virtual machines (VMs) are stored, you can select which location by numeric value here.
  • Time Limit After this period of time, your mission will suspend and all your VMs are suspended along with it.
  • Mission parameters (vem.yaml, vsphere.yaml)
  • the storyline component or backstory of the system may include a video 140 as depicted in Figure 15 and/or text which describes the background for the mission as the introduction when the mission is selected.
  • the Video Player area is where the mission video can be viewed.
  • the Now Playing area provides information about the location / size / duration of the video.
  • the Mission Order component may define the following areas for the mission as depicted in the mission orders menu 160 of Fig. 17 and the dialog 170 of Fig. 18:
  • Mission Order Page Settings provides access to settings for the Mission Order page. Currently the Image Type may be set.
  • the Mission Objectives component defines the objectives for the mission and associated badges/skills and umpire checks along with relative scoring.
  • Mission objectives may be viewed in the game environment while playing the game as shown in the mission objectives interface of Figure 21. Points are associated with each objective / skill.
  • Mission Objectives tab illustrated in Figure 22A is divided into the following areas and described below: (1) Objective definition and navigation; (2) Skills addressed by objective; (3) Umpire check associated with objective; and (4) List of Existing Objectives (with associated skills).
  • the objective name may be edited at any time in the Objective field at the top of the tab.
  • the current objective number is shown above the field.
  • the objective score may be chosen at any time by selecting the desired value from the Score dropdown at the top of the tab. The score could be based on "How hard is this objective on a scale of 1 to 10?".
  • Umpire checks form the basis of automated scoring and are required for each mission objective.
  • the Solutions and Hints component defines the solution and associated hint(s) for each mission objective.
  • the solution walks the player through the objective.
  • the hints provide multiple statements to lead the player to the solution.
  • Hint Initially there is a "blank" Hint shown for which text can be entered as illustrated in Figure 28. To add another Hint, click the Add Hint button. A Hint header is added to the hint editing area. Type the desired text in for the hint.
  • the Network Map component shown in Figure 29 defines the network to be used for the mission.
  • the Network is one of the main components which needs to be solved.
  • To access the Network Map click on the Network Map tab of Fig. 1.
  • Control IP Address Show Control IP Address - view the control IP Address assigned to the node(s) on the network map ("Control: " is prepended to the address) .
  • the Network Map Toolbar provides quick access to items for working with the network map of Figure 29.
  • the Select icon is the default and allows for selections on the network map.
  • the Camera icon provides a means to capture and save an image of the current network map. Click this icon to bring up a Save dialog. Browse to desired location, name the file and click the OK button. Click the Cancel button to close the dialog without saving.
  • the Rubberband Zoom icon shown in Figure 32 provides a means to zoom in to a specific desired area, see Figure 32. Click this icon, then click on the network map, hold and drag the cursor to desired rectangular area and release. Network map is zoomed to view only this area.
  • the Zoom In icon increases the size of displayed items viewed on the network map.
  • the Zoom Out icon decreases the size of displayed items viewed on the network map.
  • the Panning icon provides a means to manually move the viewing area of the network map. Click on this icon and then click and hold on the map while moving the cursor to desired viewing area.
  • the Re-center icon provides a means to manually center the viewing area of the network map at a particular point. Click on this icon and then click on desired center point on the network map.
  • the Restore Original Zoom icon resets the zoom level of the network map to the default size.
  • the Refresh icon redraws the page.
  • the Node Box icon provides a means to encapsulate a set of node(s) within a defining box. Click this icon, then click on the network map, hold and drag to enclose desired node(s) and release (the box can be renamed).
  • the Connector icon provides a means to connect two items (nodes or boxes) on the network map with a straight line. Click this icon, then click the first item to be connected on the network map and click the second item to be connected. A straight-line connector is shown between the two items.
  • the Elbow Connector icon provides a means to connect two items (nodes or boxes) on the network map with a right-angled connector. Click this icon, then click the first item to be connected on the network map and click the second item to be connected. A right-angled set of line segments is show between the two items (if the two items are in line with each other - horizontally or vertically - a straight line is shown).
  • the Delete icon deletes any selected items on the network map. If a selected item has connector(s), the connector(s) will also be deleted.
  • the left panel of the Network Map tab provides elements to define and identify information for items on the network map.
  • the Nodes palette as shown in Figure 33A provides icons depicting the various types of nodes which may be added to the network map. Click on one of the icons and then click on network map at desired location. The node is added to the network map.
  • the Attributes palette shown in Figure 33B provides (changing) icons depicting types of attributes which may be assigned to a node on the network map.
  • the Services icon is the first icon in the set.
  • the Key Terrain icon is the second icon in the set.
  • the Operating System icon is the third icon in the set. [00404] Click on this icon and then click on the node on the network map for which the operating system is to be assigned. This will bring up the Operating System dialog.
  • the Mouseover Feedback area shown in Figure 34 displays all of the current attributes / properties / connections / boxes for the network map item over which the cursor is currently "hovering" whether or not the Network Map - Options menu item is checked for that property.
  • the Editable Properties area shown in Figure 35 provides the means to edit the properties (such as name, addresses, operating system) appropriate to an item selected on the network map. Enter values in appropriate fields.
  • Nodes and Boxes may be moved on the network map.
  • Nodes and Boxes may be arranged in a specified order for viewing. This is important when two or more items overlap because it determines which one(s) are in front or back.
  • Nodes and Boxes may be "locked” so that they cannot be selected, edited or moved within the network map.
  • the Events component include a list of various events for the mission.
  • Event ID table header For the list. The events are sorted in ascending alphabetic order. Click the header again to sort in descending alphabetic order.
  • the Quizzes as shown in Figure 40A define a set of questions to be used in "qualifying" for the mission.
  • the questions are multiple choice in nature, may be associated with multiple missions, and optionally provide Key Words.
  • Answer Content (a string of text to be found in any answer field - correct, wrong options, or detailed answer)
  • Mission(s) specific mission(s) for which question(s) may have been tagged.
  • Skill Content (a string of text to be found in any skill associated with the question(s)).
  • the Quizzes tab 290 as illustrated in Figure 40D is divided into the following areas and described below.
  • the question difficulty may be chosen at any time by selecting the desired value from the Difficulty drop down. If the question could apply to multiple difficulty levels, choose "Any".
  • Next Question [00549] To display the next question, click the Next button 296. The information for the next question is displayed.
  • the Checklists tab 300 as shown in Figure 41 provides a convenient look at what is left to do in order to construct a complete mission.
  • the java application is a single window with a tabbed pane, one tab for each key item
  • the main menu may include one or more of the following elements:
  • the user is provided a way to save the mission to a user-selected directory.
  • a subdirectory named ⁇ ORG>_Mission_ ⁇ Difficulty Level> is created with data directories underneath for each plugin and for exporting the mission to game environment.
  • File - Open Mission... brings up a file selector dialog to browse to and select the desired mission.
  • File - Verify Mission Data will examine the current mission data and present a dialog indicating the completeness and consistency of the data. The verification will include [00592] Verify mismatches in exported files (run "orchestration script" to verify files are complete / consistent)
  • Figure 43 is one example of how the dialog 320 may appear.
  • Tooltips is provided for each item.
  • File - Export Mission for game environment writes out the appropriate mission files in format for the target game environment.
  • the user is then presented with a dialog 340 as depicted in Figure 45 indicating the success of the export.
  • FIG 47 An example of what the Mission Development Walkthru may look like is shown in Figure 47. This panel is the Orion implementation of the Mission Design process.
  • Red/Green lights indicate the status of each step.
  • the arrow icon to the right of the pane allows the pane to be "hidden” (collapsed). While collapsed the arrow icon allows the pane to be displayed (expanded) again.
  • Mission Setup Plugin has several tabs allowing the mission designer to define various aspects of the mission, such as one or more of:
  • Print Mission Setup -brings up a print dialog with available printer(s) and print text to selected printer.
  • Mission Storyline Mission_l_Storyline.docx
  • Mission Video Mission l_intro.mp4
  • Mission Thumbnail mission 1. thumb
  • Protocols telnet, vnc, ssh
  • Mission 1 The purpose of Mission 1 is to have a basic offensive mission where early users can get used to Athena and how the game environment works. They will learn skills applicable to Certified Ethical hacking (CEH).
  • CEH Certified Ethical
  • Number of Players One or more Players
  • Network Size Small (-10 VMs)
  • This subtab 350 shown in Figure 49 allows the mission designer to select which skills a player should have prior to playing the mission.
  • a search capability is provided for finding skills in the list.
  • This subtab 360 shown in Figure 50 provides for entry of data which define the key terrain of the mission.
  • This subtab 370 shown in Figure 51 provides for entry of data which define the general characteristics of the mission.
  • This tab 380 shown in Figure 52 allows the mission designer to define training outcomes for the mission.
  • This tab 390 shown in Figure 53 allows the mission designer to associate file(s) with the mission. Some files are known/required while others are optional information.
  • the Mission Order Plugin provides the capability to define information for the mission, such as one or more of:
  • the Mission Order plugin tab 400 provides a styled editor depicted in Figure 54 for entering the various components of the Mission Order as "Pages" within the tab.
  • the mission designer is able to switch between the pages by selecting the desired page to be displayed.
  • a styled editor is implemented to provide tools to edit / format the text for the Mission Order. Capabilities include:
  • the Mission Objectives plugin provides the capability to define Mission Objectives, associated badges/skills and umpire detections.
  • Orion Objective Scoring and GAME ENVIRONMENT Objective Game Points are two separate entities.
  • Orion Objective Scores for "normal" Missions need to be a percentage of total game points (adding up to 100%). This total number plus the Objective % points is used to determine the game points for the objective.
  • objective game points (percentage_points * total_game_points) / 100
  • objective game points (battle_room_score * total_game_points) / sum(all battle_room_score)
  • Hint/Solution points is determined based on a total hint percentage - the percentage of points of objective that would be lost if ALL hints for that objective were taken.
  • hint score ((hint_percentage * objective game points) / 100) / number of hints for that objective
  • ping is the program used to send ICMP diagnostic messages, ...
  • Mission Objectives plugin tab 410 shown in Figure 55 provides the capability to add, delete, or reorder mission objectives in a list.
  • the mission designer is able to assign score points based on mission type (see below).
  • a list of available skills are displayed.
  • the mission designer selects skill(s) from the master database list (see Orion - Database IDD) and assign their score (between 1-10).
  • the mission designer selects Umpire Detection(s) to associate with the objective and assign values to it (see below).
  • Mission scoring is based on the Mission Type.
  • Mission The Objective score is relative. The sum of all Objective scores in the mission must add up to 100%. If the user slides one up, the score(s) is adjusted to re-equal 100%.
  • the mission designer is able to define the Umpire Check associated with the Objective.
  • a callback function to be called with the return of any detection events is defined in Orion - Database IDD.
  • the Mission Solutions and Hints Plugin provides the capability to define the mission solution and associated hint(s) for each objective.
  • the hints provide multiple statements to lead the player to the solution.
  • the solution walks the player through the objective.
  • Mission Solutions and Hints plugin tab 420 depicted in Figure 56 provide the selection of objective and then defining the solution and associated hints for that objective using a styled editor.
  • a styled editor is implemented to provide tools to edit / format the text for the
  • Hint(s) For Hint(s), a list of current hint(s) with first part of the text of the hint are displayed. [00870] To view the full hint without bringing up the editor dialog, hover over the hint text area.
  • Variable - based on a mission variable e.g. remote service
  • dialog 430 such as shown in Figure 57 is used:
  • the Network Map Plugin will provide the capability to define the network for the mission.
  • This report is a file of network map node information including:
  • Network Map plugin tab 440 shown in Figure 58 provides tools to
  • Adversary mark whether this node is an adversary
  • OS Operating System
  • Key Terrain Not Key Terrain, Tier 1 (High), Tier 2 (Medium), Tier 3 (Low)
  • the mission designer is able to indicate whether a node is an adversary.
  • the mission designer is able to manage the service(s) associated with a node on the map.
  • the mission designer is able to manage the network interfaces for a network on the map. This is a key step because the system configures the network and sets up interfaces and IP addresses through this process.
  • the mission designer is able to manage which tool(s) are installed for a node on the map.
  • the system sets up all the installation configuration for that to be completely installed.
  • the mission designer is able to associate a Key Terrain value for a node on the map.
  • the Events Plugin provides the capability to define the various events for a mission.
  • the Events plugin tab 450 as shown in Figure 59 provides tools to define the various events for a mission.
  • the tab displays the list in time order and provide capability to Add, Edit, Delete events in the list
  • Event Technique dropdown is populated with corresponding values from the master list, select value
  • Target(s) field Click on the Target(s) field to bring up Select Target(s) dialog and choose the desired target(s) from list of appropriate ⁇ ?) network map nodes and any existing event targets emitted by earlier Delivery events.
  • the Athena Q&A Plugin provides the capability to define questions and answers for the Artificial Intelligence (AI) in-game Advisor for Game environment (ATHENA).
  • AI Artificial Intelligence
  • ATHENA Artificial Intelligence
  • the types of Q&A include
  • Athena Q&A plugin tab 460 shown in Figure 60 provides the capability to Add /
  • Data for each Q&A include:
  • the Quizzes Plugin provides the capability to pull existing questions from the master quiz database and to define new quiz questions for the mission.
  • the questions are multiple choice in nature, may be associated with multiple missions, and optionally provide Key Words, and a Source for the information.
  • the Quizzes plugin tab 470 shown in Figure 61 provides the capability to Add /
  • skills earned for the question may be selected from list of Badges / Skills.
  • Mission Library plugin 480 shown in Figure 62 allows the mission designer to check in/out any of the mission associated files from the repo.
  • File(s) may be selected on the local file system and drag/dropped into the appropriate field. A file may also be selected on this page and drag/dropped onto the local file system. [001029] The following files are required for the mission to be complete. Only a single file may be selected for these - if another file is selected, then it will replace the existing one (user is warned).
  • Mission Info is for optional file(s) which the mission designer may want included / saved with the mission. Multiple files may be selected.
  • Battle Rooms are places where players can do small technical tasks and reinforce their knowledge in different areas.
  • Missions are end-to-end scenarios where players need to be able to figure out how to solve a problem with a set of tools. As such, Battle Rooms need to be handled differently in some areas. They do not require some of the mission complexity.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Tourism & Hospitality (AREA)
  • Strategic Management (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Human Resources & Organizations (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Quality & Reliability (AREA)
  • Data Mining & Analysis (AREA)
  • Operations Research (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Methods and systems allow a user or operator to easily create cyber-training environments for use in a cyber-training system. In one embodiment, the environments are configured as missions. The missions may have a plurality of features, such as training objectives, a mission storyline, a mission order and mission objectives, relative to a mission environment. The mission environment comprises a virtual environment, such as defined by a virtual network having virtual machines or devices.

Description

SYSTEM FOR DYNAMICALLY PROVISIONING
CYBER TRAINING ENVIRONMENTS
BACKGROUND OF THE INVENTION
[001] As cyber-attacks continue to increase in frequency and become more sophisticated, the need for security systems and highly trained experts to protect industry and government information systems is growing just as fast. This rapidly growing cyber security threat landscape coupled with the shortage of personnel with the expertise required to safeguard critical systems and sensitive information poses a serious security risk for the public and private sectors.
[002] Unfortunately, current training methods are severely challenged to keep up-to-date and provide the training necessary to combat the threat. This highly complex security training has traditionally occurred in the classroom or has been provided by consultants with access to live systems evaluating real-time security threats as they occur. These existing training methodologies and techniques cannot keep up with the rapidly changing security threats nor can they train personnel fast enough. To further complicate existing training programs, real- life cyber threat scenarios become outdated by new threats shortly after training is introduced.
[003] Current training systems are built with the specific target for training in mind and dedicated to staff and students as such. For example, some of these targets may include healthcare, finance, power grid network infrastructure, etc. Current training systems are customized with hardware, software, and built to satisfy the training needs of the targeted industry. Present day systems are generally static in nature and configured once for the targeted industry, then modified manually as training needs and technology changes.
[004] This focused manual customization for each industry target in need of training increases the cost of the overall training system development and support, making current training systems expensive and too costly for most businesses desperately in need of such state of the art training. Such legacy training systems require extensive manual modification and on-going customization to keep up with the student's training needs and the rapid pace of technology evolution in each particular industry where training is required. This fast-paced evolution of technology quickly makes training systems obsolete and in need of revision to keep up with the continual flow of new students, new systems and new operational methods.
[005] Further, even in those situations where computer implemented training systems have been developed, those systems suffer from similar problems. While these systems can be used to train larger numbers of students, the training systems are not flexible and provide limited training benefits. For example, existing training systems are designed to implement fixed training sessions. That is, these training systems include one or more predesigned or fixed training applications. The training system simply implements that single fixed training application or selects from one of a small set of fixed training applications. Thus, students see the same training environments over and over. If the operator desires to present students with a different training session or environment, an entirely new training application must be built and loaded into the training system.
[006] This "select from fixed training sessions" configuration is consistent with the goal of existing training sessions: to create a training session in which a student practices or implements one or more specific tasks. In accordance with the task-based training, the training is used to train the student on a particular task and to increase their proficiency in implementing the task. However, in the real world, each cyber threat is very different. Thus, a student's ability to perform a particular designated task is insufficient in helping the student understand when to perform the task or how to use it in conjunction with other tasks or techniques in order to address a cyber threat.
[007] Given the rapidly changing cyber threat risk and the constant attacks from hackers around the world, a dynamic, virtual network training system and method are needed to provide an isolated and controlled network environment with the level of complexity needed to train experts how to rapidly respond to cyber-attacks, terrorism, and cyber-crime, and how to stop them. In order to implement such a system, a method and system are needed to generate a plurality of varied or variable training missions, such as having different environments and objectives.
SUMMARY OF THE INVENTION
[008] Embodiments of the invention comprise methods and systems which allow a user or operator to easily create cyber-training environments for use in a cyber-training system, and preferably methods and systems for designing and configuring specific training mission hardware and software environments.
[009] In one embodiment, the environments are configured as missions. The missions may have a plurality of features, such as training objectives, a mission storyline, a mission order and mission objectives, relative to a mission environment. Preferably, the mission environment comprises a virtual environment, such as defined by a virtual network having virtual machines or devices.
[0010] Aspects of the invention comprise hardware and software which are uniquely configured to automate the creating of such cyber-training environments or missions, thus solving problems with and providing advantages over existing technology which requires cyber training platforms to be custom design or configured with a static configuration.
[0011] Further objects, features, and advantages of the present invention over the prior art will become apparent from the detailed description of the drawings which follows, when considered with the attached figures.
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] FIGURE 1 illustrates a graphic user interface according to embodiments of the present invention.
[0013] FIGURE 2 illustrates a system architecture according to embodiments of the present invention.
[0014] FIGURE 3 illustrates how a controller will operate in different states according to embodiments of the present invention.
[0015] FIGURE 4 A illustrates a block diagram of a system according to embodiments of the present invention.
[0016] FIGURE 4B illustrates a software architecture according to embodiments of the present invention.
[0017] FIGURES 5 A, 5B and 5C illustrate changing plug-in states when switching to a different mission according to embodiments of the present invention.
[0018] FIGURE 6 illustrates communication between a user and the server using Webswing according to embodiments of the present invention.
[0019] FIGURE 7 illustrates a create new mission top level graphic according to embodiments of the present invention.
[0020] FIGURE 8 illustrates a graphic user interface for creating new missions according to embodiments of the present invention.
[0021] FIGURE 9 illustrates a graphic user interface for opening an existing mission according to embodiments of the present invention.
[0022] FIGURE 10 illustrates a save changes dialog according to embodiments of the present invention. [0023] FIGURE 11 illustrates a graphic user interface for copying a mission according to embodiments of the present invention.
[0024] FIGURE 12 illustrates an advanced mission setting dialog according to embodiments of the present invention.
[0025] FIGURE 13 illustrates an export mission summary dialog according to embodiments of the present invention.
[0026] FIGURE 14 illustrates an export files created interface according to embodiments of the present invention.
[0027] FIGURE 15 illustrates a storyline video or text according to embodiments of the present invention.
[0028] FIGURE 16 illustrates a storyline interface according to embodiments of the present invention.
[0029] FIGURE 17 illustrates a mission orders interface in a game environment according to embodiments of the present invention.
[0030] FIGURE 18 illustrates a define mission order dialog according to embodiments of the present invention.
[0031] FIGURE 19 illustrates a situation editor according to embodiments of the present invention.
[0032] FIGURE 20 illustrates a mission order page settings dialog according to embodiments of the present invention.
[0033] FIGURE 21 illustrates a mission objectives interface according to embodiments of the present invention.
[0034] FIGURE 22 A illustrates a mission objectives graphic user interface or tab according to embodiments of the present invention.
[0035] FIGURE 22B illustrates a search for skill interface according to embodiments of the present invention.
[0036] FIGURE 23 illustrates a skills interface according to embodiments of the present invention.
[0037] FIGURE 24 illustrates an umpire check dialog according to embodiments of the present invention.
[0038] FIGURE 25 illustrates a solutions and hints user interface according to embodiments of the present invention. [0039] FIGURE 26 illustrates a screenshot defining solutions and hints according to embodiments of the present invention.
[0040] FIGURE 27 illustrates a define solution interface according to embodiments of the present invention.
[0041] FIGURE 28 illustrates an add hint interface according to embodiments of the present invention.
[0042] FIGURE 29 illustrates a network map screenshot according to embodiments of the present invention.
[0043] FIGURE 30 illustrates a network map options interface according to embodiments of the present invention.
[0044] FIGURE 31 illustrates a screenshot of key terrain according to embodiments of the present invention.
[0045] FIGURE 32 illustrates a screenshot of a rubberband zoom according to embodiments of the present invention.
[0046] FIGURES 33A and 33B illustrate icons that can be added to the network map according to embodiments of the present invention.
[0047] FIGURE 34 illustrates a mouseover feedback interface according to embodiments of the present invention.
[0048] FIGURE 35 illustrates editable properties dialogs according to embodiments of the present invention.
[0049] FIGURE 36A illustrates an arrange item interface according to embodiments of the present invention.
[0050] FIGURE 36B illustrates lock item interface according to embodiments of the present invention.
[0051] FIGURE 37 illustrates a network graph map area according to embodiments of the present invention.
[0052] FIGURE 38A illustrates a defining events interface according to embodiments of the present invention.
[0053] FIGURE 38B illustrates an add events interface according to embodiments of the present invention.
[0054] FIGURE 39 illustrates a step for id dialog according to embodiments of the present invention. [0055] FIGURE 40A-40D illustrates quizzes interfaces according to embodiments of the present invention.
[0056] FIGURE 41 illustrates a checklist tab or interface according to embodiments of the present invention.
[0057] FIGURE 42 illustrates a define mission dialog according to embodiments of the present invention.
[0058] FIGURE 43 illustrates a mission verification summary dialog according to embodiments of the present invention.
[0059] FIGURE 44 illustrates an advanced mission settings dialog according to embodiments of the present invention.
[0060] FIGURE 45 illustrates an export summary dialog according to embodiments of the present invention.
[0061] FIGURE 46 illustrates a mission development methodology walkthrough according to embodiments of the present invention.
[0062] FIGURE 47 illustrates a mission development methodology walkthrough according to embodiments of the present invention.
[0063] FIGURE 48 illustrates a welcome page according to embodiments of the present invention.
[0064] FIGURE 49 illustrates a user pre-requisite skills interface according to embodiments of the present invention.
[0065] FIGURE 50 illustrates a key terrain interface according to embodiments of the present invention.
[0066] FIGURE 51 illustrates a mission characteristics interface according to embodiments of the present invention.
[0067] FIGURE 52 illustrates a training outcomes interface according to embodiments of the present invention.
[0068] FIGURE 53 illustrates an associated mission files interface according to embodiments of the present invention.
[0069] FIGURE 54 illustrates a mission order page interface according to embodiments of the present invention.
[0070] FIGURE 55 illustrates a mission objectives interface according to embodiments of the present invention. [0071] FIGURE 56 illustrates a solutions and hints interface according to embodiments of the present invention.
[0072] FIGURE 57 illustrates an edit text hint interface according to embodiments of the present invention.
[0073] FIGURE 58 illustrates a network map page interface according to embodiments of the present invention.
[0074] FIGURE 59 illustrates an events interface according to embodiments of the present invention.
[0075] FIGURE 60 illustrates a questions and answers interface according to embodiments of the present invention.
[0076] FIGURE 61 illustrates a quizzes page interface according to embodiments of the present invention.
[0077] FIGURE 62 illustrates a mission library interface according to embodiments of the present invention.
[0078] FIGURE 63 illustrates a cyber range backbone of Internet services, user emulation, and auto-inject mission modules that support the training environment according to embodiments of the present invention.
[0079] FIGURE 64 illustrates a small core mission module for eight teams that support the training environment according to embodiments of the present invention.
[0080] FIGURE 65 illustrates a medium core mission module that support the training environment according to embodiments of the present invention.
DETAILED DESCRIPTION OF EMBODIMENTS
[0081] In the following description, numerous specific details are set forth in order to provide a more thorough description of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without these specific details. In other instances, well-known features have not been described in detail so as not to obscure the invention.
[0082] Embodiments of the invention comprise methods and system for creating or generating cyber environments, and preferably methods and systems for designing and configuring specific training mission hardware and software environments.
[0083] Figure 1 illustrates one embodiment of aspects of the present invention, wherein the invention is referred to as the "Orion" system or tool. Aspects of the invention preferably comprise: (1) A mission design methodology. In one embodiment, this methodology is an iterative, collaborative process embodied in the tool that defines everything for a complete cyber training environment. (2) Network templates. The templates define a baseline environment complete with working definitions of the network services/protocols and unique overlays (#3) to include variation and threat actor (it creates full orchestration to support dynamic provisioning). (3) Ability to define variability (services_0) layer as part of the design process. (4) Event driven process for two-sided training events (offense v defense). (5) Support of real-time assessment data and reporting by defining umpire detections for each event.
[0084] In one embodiment shown in Fig. 4A, the system comprises an Orion controller 48, such as associated with a server 40, and one or more clients 41, such as implemented at computing devices such as laptop or desktop computers or the like. The system is preferably configured so that standalone clients will run on any machine that supports Java8, so that the server components will run on any machine that supports Java, so that the web client will run on any machine that support a modern browser, and wherein the system tool is implemented as a state machine (such as to simplify the design process and allow a global logger to be maintained). Of course, the system could be configured to operate in other environments.
[0085] As illustrated in Figures 4A and 4B, the system may, as described in more detail below, implement a global logger 42, include a plurality of plug-ins 44, include one or more memory devices for storing machine-readable code for execution by the Orion controller 48 (for implementing various of the functionality described herein), and include one or more databases, such as a mission database 46 for storing missions or mission information. The plug-ins 44 may include a storyline plugin, a mission setup plugin, a mission order plugin, a solution plugin, a network map plugin, an events plugin, an Athena Questions and answers plugin, a quizzes plugin and a mission library plugin. As noted above, the system server(s) 40 are preferably configured to communicate with one or more client devices 41. The client devices 41 may display one or more user interfaces, such as the one or more graphical user interfaces described below, such as to provide information to a user and receiving information from a user, including in configuring the various parameters of a mission, etc.
[0086] The mission database 46 can include a plurality of preconfigured missions as well as missions that have been saved by mission developers, such that the mission database 46 may grow over time. A mission designer can design a mission from scratch or start from one of the preconfigured or saved missions and use that mission without change or after modifying the mission.
[0087] The Orion system is a cyber mission builder and provides the capability for creating and editing of a new cyber scenario or mission that can be used in a training environment or platform. For example, using the system, a user can create scenarios involving how to defend a bank' s network from a financial attack using ransomware or a scenario where a hacker tries to steal credit card data. Orion is used to develop new missions and content. It is used by mission designers, not end users of cyber training.
[0088] Figure 2 depicts the system of the invention and how it fits within the larger ecosystem of a gamified training system, such as a game system referred to as "Ares". In one embodiment, for example, an Ares gaming system 22 has the components of a game environment, a cyber security domain, an artificial intelligence (AI) components, and a virtualized network. The virtualized network comprises a virtual environment of virtual machines, plus software tools and the like. In one embodiment, Ares starts a game in a phase I where the user selects a specific mission, skill level, learns the mission objectives and uses the interface to perform any necessary mission preparation including taking proficiency quizzes. The game then proceeds to a phase II where the user is immersed in the game trying to accomplish the mission objectives. After the mission play is over, the game transitions to the assessment phase where the player learns what happened and what they did (strengths/weaknesses). This process includes a complete walk through of an after-action report covering the mission in detail.
[0089] The Ares game system may include an automated advisor, such as an automated artificial intelligence advisor (such as under the name "Athena"). This advisor may, for example, be capable of providing a player assistance during a game, such as by answering player questions or providing hints, among other things. The Ares game system may also include an "umpire" (which may be implemented as part of the advisor, or separately therefrom). The umpire may, for example, score the player based upon their actions (and/or inactions) during the mission.
[0090] Relative to the Ares system (or similar systems), the Orion system of the present invention preferably allows an operator to create or generate different missions (e.g., objective- based assignments within a defined environment, such as a cyber environment which includes a virtualized network).
[0091] In one embodiment, the high-level architecture of the system is a plugin paradigm using a state-machine life-cycle. Each plugin 44 is an independent wizard for adding/modifying one aspect of the mission. The plugins 44 are maintained by a controller which messages state changes with each plugin. There is a general method for communications between plugins. A global logger 42 is maintained.
[0092] A top-level state machine (TLSM) is implemented that supports the mission building guidelines. The purpose of the TLSM is to provide a concrete implementation of the guidelines. It will provide a means to collect and store artifacts that are produced outside of the Orion environment (e.g., mission video, adversary attack motivations, mission art, ...)
[0093] The status of the TLSM is maintained by the Orion controller and is visible in a panel on the graphical user interface (GUI). The panel can be hidden/displayed at any time.
[0094] Orion Controller
[0095] The Orion or system controller 48 provides the glue for the application - keeping track of what is going on and communicating to individual plugins 44 as needed. Table 1 describes the Orion System Controller States. Figure 3 and Table 1 illustrate how the controller 48 will operate within these states.
Figure imgf000012_0001
Discover Appropriate directories are searched for wizard plugin class files.
Initialize Each plugin is instantiated with a reference to the Controller, a
reference to the global Logger, a reference to the main menu bar, and a graphical container for it to display its content. A tab is generated for the plugin on the main window as in Fig. 1.
Idle There are no pending user commands to process at the main window or main menu level.
Open Open an existing mission
Create Create a new mission. The controller will create a data directory for each plugin under the main mission directory.
Save The loaded mission is being saved. This requires integrating the saves of each plugin
Cleanup The application is shutting down
Error An unrecoverable error has occurred
Table 1
[0096] Global Data Singleton
[0097] Orion has some data which is "shared" among the plugins. To maintain the integrity of the data, the data is preferably maintained by a (set of) global data singleton(s). In one embodiment, the following data is included:
[0098] (1) Master Services List from database
[0099] (2) Master Detections List from database
[00100] (3) Master Umpire Check Callback Functions from database
[00101] (4) Master Service Events from database
[00102] (5) Master Adversary Events from database
[00103] (6) Master Malware List from database
[00104] (7) Master Badges/Skills List from database
[00105] (8) Master Quiz Questions List from database [00106] (9) Master Virtual Machine (VM) Templates from database
[00107] (10) Master Tools List from database
[00108] (11) Current Mission Objectives and associated Umpire Detections
[00109] (12) Current Mission Network Diagram Node Information
[00110] (13) Current Mission service/detection variable data
[00111] The master data is preferably read in when the singleton is instantiated. At this time, the singleton will check to see if there are updates to the master from the mission design databases and use those.
[00112] Current mission data is read in when a mission is opened (or structure created when new mission is created).
[00113] Any data which may be editable within Orion is accessed in a synchronized manner.
[00114] Plugin States
[00115] At startup, all plugins are disabled until a mission is opened or created. Once a mission is active, each plugin tab 12 indicates the state of that plugin, such as shown in table 2. These plugin states are used when switching to a different mission to determine action needed, such as illustrated in Figure 5 a, 5b, and 5c (wherein dark gray boxes indicate that a dialog box is invoked). The plugins are instantiated by the Orion controller when a mission is opened or created.
Figure imgf000014_0001
Table 2
[00116] Inter-Plugin Communication
[00117] A general method of plugin to plugin communication is provided. In one embodiment, this communication is a messaging mechanism similar to a post office. One plugin posts a message (e.g., mission objectives when objective added) and other plugin(s) gets the post and processes it as appropriate (e.g., solution and hints to add another page for the new objective).
[00118] The processing may use threads with a behind-the-scenes wait- notify mechanism. In one embodiment, messages are not received on the GUI thread, so no direct drawing occurs as a result.
[00119] Messages
[00120] The PluginMessage class provides the structure for the message to be sent. A message consists of: (1) Source (the plugin that sent the message); (2) Source Name (the name on the plugin tab; (3) Message ID (defined in PluginMessageType enum); and (4) Pay load (any object to send - simple or complex).
[00121] Messages are not sent over a socket so the payload is not being serialized— the actual object arrives at the other plugins who in principal could corrupt the object.
[00122] Message Types
[00123] PluginMessageType enum contains the IDs for the messages. Current values may be: (1) Objectives and (2) Text.
[00124] Sending Messages
[00125] Every plugin has a method postMessage(MessageType type, Object payload). This creates a message with the plugin as the source, the plugin tab name as the source name, and the type and payload given.
[00126] Example: postMessage(MessageType.objectives, objectives) where objectives is an object that contains the objectives.
[00127] All plugins get the message. Those that have need of it use it, others ignore it.
[00128] The Log Plugin
[00129] The log plugin simply logs every message it gets - keeping a record of messages posted in the log tab.
[00130] Processing Messages
[00131] Every plugin implements the interface IMessageProcessor. It contains one method processMessage(PluginMessage message). The default implementation will do nothing. Override processMessage to be able to handle message(s). For example:
[00132] ©Override
[00133] public void processMessage(PluginMessage message) { [00134] switch(message.getType() {
[00135] case objectives:
[00136] dosomething(message.getPayload() ;
[00137] break;
[00138] case (whatever)
[00139] break;
[00140] }
[00141] }
[00142] processMessage is called on a separate (non-GUI) thread.
[00143] Webswing Configuration
[00144] When Orion is configured as a thin client using the web browser to deliver Orion capability, Webswing is used to present the Orion experience to the user. Webswing provides a Javascript client, which mirrors a running Java app (such as Orion) on the server. Webswing captures Swing draw calls, and sends it to the Javascript client via a secure websocket connection, which then mirrors the draw call.
[00145] Figure 6 depicts the communications between the user and the server using Webswing.
[00146] Security Model
[00147] Data in Transit
[00148] Data which traverses an external network (not localhost) will utilize TLS vl.2. This is between the Mission Server and Orion. The connections utilize both client and server authentication using certificates signed by a recognized Certificate Authority.
[00149] MISSION BUILDING
[00150] Mission Definition
[00151] Basic information for the mission is defined when the mission is created. These parameters include one or more of:
[00152] (1) Organization
[00153] (2) Mission Name
[00154] (3) Mission Description
[00155] (4) Mission Number as a unique identifier when combined with an organization [00156] (5) Mission Level of Difficulty
[00157] As illustrated in Figure 7, this information may be presented via a game environment on a world map, where the user can select the mission to play. [00158] Create New Mission
[00159] One example of a methodology for creating a new mission is as follows:
[00160] Click File - New Mission from the mission setup plugin in Fig. 1 ... to bring up the
Create New Mission dialog 80 (see Figure 8).
[00161] Enter the desired Organization Prefix, Mission Name, Mission Description, Mission Number and choose a Mission Difficulty.
[00162] Select / browse to the place where the mission is to be stored and click the New Mission button 82. A folder is created in the selected folder titled <prefix>_mission_<number>_<difficulty> (e.g. XYZ_mission_l_Easy).
[00163] Click the Cancel button to exit the dialog without creating a mission.
[00164] Open Existing Mission
[00165] One example of a methodology for opening an existing mission is as follows:
[00166] Click File - Open Mission... to bring up an Open dialog 90 (see Figure 9).
[00167] Browse to the folder containing the desired mission.
[00168] Select the mission and click the Open button. Orion tabs will populate with the current data of the mission.
[00169] Click the Cancel button to exit the dialog without opening a mission.
[00170] Unsaved Changes
[00171] When switching to another mission (either by creating a new mission or opening a different existing mission) or exiting the Orion application when unsaved changes exist, the
Save Changes dialog 100, such as illustrated in Figure 10, may be displayed.
[00172] Click the Yes button to save the changes prior to switching mission or exiting Orion.
Click the No button to switch or exit without saving the changes.
[00173] Save Mission
[00174] To save the current mission, click File - Save Mission... The mission data for each tab is saved in a folder under the main Mission folder.
[00175] Many of these tabs also provide for save of just their associated information (see appropriate section in this document).
[00176] Copy Mission
[00177] To copy the currently loaded mission to a new one,
[00178] Click File - Save Mission As... to bring up the Save Mission As dialog 110 (see Figure 11). [00179] Edit the Organization Prefix, Mission Name, Mission Description, Mission Number and Mission Difficulty as desired using the dialog 110.
[00180] Select / browse to the place where the mission is to be stored
[00181] Click the New Mission button. The current mission data is copied to the new mission.
[00182] Click the Cancel button to exit dialog without creating a copy of the mission.
[00183] Advanced Mission Settings
[00184] In a preferred embodiment, control parameters for executing the Orion mission are defaulted to typical values and can be modified by the Mission Designer.
[00185] To modify the parameters:
[00186] Click File - Advanced Mission Settings... to bring up the Advanced Mission Settings dialog 120, Figure 12.
[00187] Default Location: If you have multiple locations where your virtual machines (VMs) are stored, you can select which location by numeric value here.
[00188] Boot Timeout: If the virtual machines in the mission don't start up within this time, the mission will fail. If you have a very large number of virtual machines, this may need to be a larger value.
[00189] Publish Timeout: If all the umpire checks, orchestration and transformations don't load and complete within this amount of time, the mission will fail. If you are doing a lot of manipulations of your base virtual machines, you may want a longer duration for a publish timeout.
[00190] Time Limit: After this period of time, your mission will suspend and all your VMs are suspended along with it.
[00191] Enter desired changes to the parameters and click the OK button. The dialog closes. Parameter changes is saved when the mission is saved.
[00192] Click the Cancel button to close the dialog without making any changes.
[00193] Exporting Mission Definition
[00194] The following items may be included in the export:
[00195] Mission Orders - orders.xml (in export/ Assets folder)
[00196] Objectives with associated skills and umpire checks, solutions and hints - services_0.yaml
[00197] Network Map - outxml (in export/Network_Map folder)
[00198] Events - msel.json (in export/Events folder) [00199] Athena Q&A defined in this mission (Mission_Questions.txt, General_Questions.txt, Athena_Questions.txt)
[00200] Quizzes defined in this mission (quiz.txt) - a Quiz Q&A.csv file is also created for easy viewing.
[00201] Mission parameters (vem.yaml, vsphere.yaml)
[00202] Some tabs also provide for import and/or export of just their associated information (see appropriate section in this document).
[00203] To export Orion mission information, click File - Export Mission. The Export Summary dialog 130, Figure 13, is displayed upon completion of export indicating the status of each component.
[00204] Click the Close button to close the dialog. The appropriate file(s) is created in the export folder under the main mission folder as illustrated in Figure 14.
[00205] Mission Storyline
[00206] The storyline component or backstory of the system (an introduction to the user or player of the mission which provides them information regarding the mission, such as a background or plot for the mission) may include a video 140 as depicted in Figure 15 and/or text which describes the background for the mission as the introduction when the mission is selected.
[00207] To access the Storyline, click on the Storyline tab in the interface of Fig. 1.
[00208] Storyline Menu Items
[00209] When the Storyline tab is selected, the items under the Storyline Menu 150 at the top of the page are available as shown in Fig. 16.
[00210] Import Video
[00211] To import a video for the mission,
[00212] Click Storyline - Import Video File... to bring up an Open dialog. (Clicking the Cancel button will close the dialog and without continuing the import process.)
[00213] Browse to the folder containing desired video file.
[00214] Select the video file and click the Open button. The video is imported to the mission and displayed/started in the Video Player area of the tab.
[00215] Click the Cancel button to exit the dialog without importing a video file.
[00216] Remove Video
[00217] To remove a video from the mission, [00218] Click Storyline - Remove Video File as shown in Fig. 16 to bring up a Remove Video File? dialog.
[00219] Click the Yes button to remove the video. The video is removed from the mission. (Or click the No button to cancel removal.)
[00220] Save Storyline
[00221] Click Storyline - Save Storyline Data to save the current information on this tab for the mission.
[00222] Defining the Storyline
[00223] The storyline gives the context for the mission in the form of introductory video and or text depicted in Figure 16.
[00224] Video Player Area
[00225] The Video Player area is where the mission video can be viewed.
[00226] Click the Play/Pause icon to play or pause the video.
[00227] The Now Playing area provides information about the location / size / duration of the video.
[00228] Storyline Text
[00229] Enter any desired storyline text into the text area.
[00230] Mission Order
[00231] The Mission Order component may define the following areas for the mission as depicted in the mission orders menu 160 of Fig. 17 and the dialog 170 of Fig. 18:
[00232] (1) Situation - Describe overall scenario and threat intelligence information
[00233] (2) Mission - Describe the specific mission and skills or core competencies required to go on mission
[00234] (3) Execution - Describe the purpose, key tasks, end state and concept of operations to include user credentials and tools
[00235] (4) Sustainment - Describe how long the mission should take and any requirements on the end of the mission
[00236] (5) Command and Control (C&C) - Describe who commands this mission above the Team Lead and any reporting requirements
[00237] (6) Rules of Engagement (ROE) - Describe any rules about the virtual environment to include out-of-bounds list, tools, traffic generation and key services that must be maintained [00238] (7) References - for each mission objective or task in the Execution description above, provide a list of references that players should read.
[00239] These orders are shown in the game environment, such as in the format illustrated in Figure 17, preferably prior to playing the game.
[00240] To access Mission Orders, click on the Mission Orders tab of Fig. 1.
[00241] Mission Order Menu Items
[00242] When the Mission Orders tab is selected, the items under the Mission Order Menu at the top of the page are available.
[00243] Save Mission Order
[00244] Click Mission Order - Save Mission Order to save the current information on this tab for the mission.
[00245] Import Mission Order
[00246] To import the Mission Order from another mission,
[00247] Click Mission Order - Import Mission Order from the game environment to bring up an Open dialog. (Clicking the Cancel button will close the dialog and without continuing the import process.)
[00248] Browse to the folder containing desired mission orders.xml file.
[00249] Select the desired file and click the Open button. Any current mission order information is replaced with information from the selected file.
[00250] Click the Cancel button to exit the dialog without changing the current mission order information.
[00251] Export Mission Order
[00252] Click Mission Order - Export Mission Order to save the current Orion Mission Order in the correct format. An Export Successful dialog is presented
[00253] The orders.xml file is saved in the Assets folder within the export folder under the main mission folder.
[00254] Defining the Mission Order
[00255] Select the desired area from the list in the toolbar 170, such as illustrated in Figure 18, for which to define the mission order. The appropriate "page" of data is displayed with any current text.
[00256] Use the edit area to describe this area of the mission order as depicted in Figure 19 by using the Orion situation editor 180 as described later in this document.
[00257] Mission Order Page Settings [00258] Mission Order Page Settings provides access to settings for the Mission Order page. Currently the Image Type may be set.
[00259] Click the Settings icon to bring up the Page Settings dialog 190, Figure 20.
[00260] Select the image type among the choices watermark, background and none for this page and click the OK button. The settings are changed and stored with the Mission Order page.
[00261] Click the Cancel button to close the dialog without making any changes.
[00262] Mission Objectives
[00263] The Mission Objectives component defines the objectives for the mission and associated badges/skills and umpire checks along with relative scoring.
[00264] Mission objectives may be viewed in the game environment while playing the game as shown in the mission objectives interface of Figure 21. Points are associated with each objective / skill.
[00265] To access the Mission Objectives, click on the Mission Objectives tab of Fig. 1.
[00266] Mission Objectives Menu Items
[00267] When the Mission Objectives tab is selected, the items under the Mission Objectives Menu 210 of Fig. 22A at the top of the page are available.
[00268] Save Mission Objectives
[00269] Click Mission Objectives - Save Mission Objectives to save the current information on this tab for the mission.
[00270] Import Mission Objectives
[00271] To import mission objectives and associated skills and umpire checks, solutions and hints from an existing mission (or previously saved/exported Orion mission data),
[00272] Click Mission Objective - Import Objectives. This brings up an Import Objectives confirmation dialog warning about data replacement.
[00273] Click the OK button to bring up an Open dialog. (Clicking the Cancel button will close the dialog and without continuing the import process.)
[00274] Browse to the folder containing desired mission services_0.yaml file.
[00275] Select the desired file and click the Open button. Any current mission objectives and associated information is replaced with information from the selected file.
[00276] Click the Cancel button to exit the dialog without changing the current objective information. [00277] Export Mission Objectives
[00278] Click Mission Objective - Export Objectives to save the current Mission Objectives and associated skills and umpire checks, solutions and hints in the correct format. An Export Successful dialog is presented showing that the services_0.yaml file has been saved in the export folder under the main mission folder. Click the OK button to acknowledge and close the dialog.
[00279] Defining the Mission Objectives
[00280] The Mission Objectives tab illustrated in Figure 22A is divided into the following areas and described below: (1) Objective definition and navigation; (2) Skills addressed by objective; (3) Umpire check associated with objective; and (4) List of Existing Objectives (with associated skills).
[00281] Define Objective Name
[00282] The objective name may be edited at any time in the Objective field at the top of the tab. The current objective number is shown above the field.
[00283] Assign Objective Score
[00284] The objective score may be chosen at any time by selecting the desired value from the Score dropdown at the top of the tab. The score could be based on "How hard is this objective on a scale of 1 to 10?".
[00285] Add Objective
[00286] To add an objective, click the Add Objective button 212 at the top of the tab. This will increment the objective count and provide a blank objective / set of skills and umpire checks for definition of a new objective. It will also add a "blank" objective to the Existing Objectives list.
[ 00287] Fill in the Objective name, Score, and associate appropriate Skills and Umpire Check.
[00288] Copy Objective
[00289] To copy the currently displayed objective
[00290] Click the Copy Objective button 214 at the top of the tab. This will add an objective with a blank name to the end of the list with the same skill(s), umpire check and score as the current objective.
[00291] Fill in the objective name.
[00292] Reorder objectives if needed.
[00293] Delete Objective [00294] To delete the currently displayed objective, click the Delete Objective button 216 at the top of the tab. This brings up the Delete Objective dialog.
[00295] Click the OK button to delete the objective (or click the Cancel button to close the dialog without deleting the objective). The currently displayed objective and associated skills and umpire check is removed and remaining objectives renumbered appropriately.
[00296] Next Objective
[00297] To display the next objective, click the Next button 218 at the top of the tab. The information for the next objective is displayed.
[00298] Previous Objective
[00299] To display the previous objective, click the Previous button 220 at the top of the tab. The information for the previous objective is displayed.
[00300] Select Existing Objective to Display
[00301] To display a specific objective, click on the row in the Existing Objectives list 222 and the objective information for that objective is displayed.
[00302] Reorder Objectives
[00303] To reorder an objective, select an objective in the Existing Objectives list 222 and drag up or down the list to desired location. The objectives are renumbered automatically.
[00304] Objective Skills
[00305] Associate Skills
[00306] To associate skill(s) with an objective:
[00307] (1) Select the desired objective in the list of objectives. The list of skills available are displayed in the skills interface 230 of Fig. 23.
[00308] (2) Click the Select checkbox 232 next to desired skill(s) OR click in the associated skill text field. The associated skill is highlighted and the score selection box is enabled.
[00309] (3) Select the desired Skill Score 234 from the list (how hard is this skill on a scale of 1 to 10?).
[00310] To disassociate the skill, click on the checkbox (OR associated skill text field) again to clear it.
[00311] Search for Skill
[00312] To search for specific skill, enter text to search for in the Search for Skills dialog in Figure 22B. The number of results (if any) is shown to the right in Figure 22B.
[00313] Each skill that matches the entered text is outlined as shown in Figure 23. [00314] Use the Down Arrow and Up Arrow buttons to scroll to next / previous occurrence of the entered text. The one that is scrolled to is outlined with a slightly darker outline.
[00315] Objective Umpire Checks
[00316] Umpire checks form the basis of automated scoring and are required for each mission objective.
[00317] Associate Umpire Check
[00318] To associate a single umpire check with an objective:
[00319] (1) Select the desired objective in the list of objectives in Figure 24. The list of umpire checks available are displayed in the umpire check dialog 240.
[00320] (2) Click the Select checkbox 242 next to desired umpire check OR click in one of the associated umpire check text fields. The associated umpire check is highlighted.
[00321] To disassociate the umpire check, click on the checkbox (OR associated umpire check text field) again to clear it. Selecting a different umpire check will also clear the previously selected one.
[00322] Search for Umpire Check
[00323] To search for specific umpire check, enter text to search for in the Search Checks for: field 244. The number of results (if any) is shown to the right.
[00324] Each umpire check whose description and/or name matches the entered text is outlined as shown in Figure 24.
[00325] Use the Down Arrow and Up Arrow buttons to scroll to next / previous occurrence of the entered text. The one that is scrolled to is outlined with a slightly darker outline.
[00326] Solutions and Hints
[00327] The Solutions and Hints component defines the solution and associated hint(s) for each mission objective. The solution walks the player through the objective. The hints provide multiple statements to lead the player to the solution.
[00328] Viewing hint(s) will result in loss of points. The PDF link under the set of objective hints provides the solution for that objective as depicted in Figure 25.
[00329] To access the Solutions and Hints, click on the Solution and Hints tab from Fig. 1.
[00330] Solution and Hints Menu Items
[00331] When the Solutions and Hints tab is selected, the items under the Solution and Hints Menu at the top of the page are available. The interface of Figure 26 may be available, which allows scrolling through hints. [00332] Save Mission Solution
[00333] Click Solution and Hints - Save Mission Solution to save the current information on this tab for the mission.
[00334] Defining the Solutions and Hints
[00335] Defining the Solutions and Hints is illustrated in interface 252 of Figure 26.
[00336] Select Objective
[00337] To choose an objective, select the desired one from the drop down list of names at the top of the tab. The Solution and Hints area will automatically be filled in with any existing information for that objective.
[00338] Define Solution
[00339] Use the Solution area shown in Figure 27 to type in the information needed to walk the player through this objective.
[00340] Add Hint
[00341] Initially there is a "blank" Hint shown for which text can be entered as illustrated in Figure 28. To add another Hint, click the Add Hint button. A Hint header is added to the hint editing area. Type the desired text in for the hint.
[00342] Typically, the full solution is written and then pieces of it are used as hint(s).
[00343] Network Map
[00344] The Network Map component shown in Figure 29 defines the network to be used for the mission. In game environment, the Network is one of the main components which needs to be solved. To access the Network Map, click on the Network Map tab of Fig. 1.
[00345] Network Map Menu Items
[00346] When the Network Map tab is selected, the items under the Network Map Menu at the top of the page are available.
[00347] Save Network Map
[00348] Click Network Map - Save to save the current information on this tab for the mission.
[00349] Import Network Map
[00350] Click Network Map - Import... to browse to and open a previously saved Orion Network Map. The network data is filled in on the map.
[00351] Export Network Map
[00352] Click Network Map - Export to save the current Orion Network Map in the correct format for game environment. An Export Successful dialog is presented showing that the out.xml file has been saved in the Network_Map folder within the export folder under the main mission folder. Click the OK button to acknowledge and close the dialog.
[00353] Clear Drawing
[00354] Click Network Map - Clear Drawing to remove ALL items from the current network map.
[00355] If changes have been made to the network map which have not been saved, the Discard Changes dialog is displayed.
[00356] Click the Yes button to close the dialog and remove ALL items from the current network map OR click the No button to close the dialog without clearing the network map.
[00357] Duplicate
[00358] Click Network Map - Duplicate to make a copy of the currently selected node on the network map. All associated attributes (name, IP Address, OS, services...) are also copied.
[00359] Zoom to Fit
[00360] Click Network Map - Zoom to Fit to zoom in to whatever resolution necessary to show ALL items in the network map area.
[00361] Snap to Grid
[00362] Click Network Map - Snap to Grid to toggle the alignment of nodes to the grid as they are dragged on the network map. Checkmark indicates the feature is turned on.
[00363] Snap Size
[00364] Click Network Map - Snap Size to adjust the size of the grid. The current size is shown on the menu.
[00365] Options
[00366] Click Network Map - Options to toggle the viewing of various details on the network map as shown in Figure 30. Checkmark indicates the feature is turned on as shown in Figure 30.
[00367] Show OS Type - view the operating system (OS) for the node(s) on the network map.
[00368] Show Simple Name - view the name assigned to the node(s) on the network map.
[00369] Show IP Address(es) - view all non-control IP Addresses assigned to the node(s) on the network map.
[00370] Show Control IP Address - view the control IP Address assigned to the node(s) on the network map ("Control: " is prepended to the address) .
[00371] Show Services - view the service(s) assigned to the node(s) on the network map. [00372] Show Key Terrain - view indication as depicted in Figure 31 of Key Terrain designation assigned to the node(s) on the network map (1 ->High, 2 - Medium, 3 - Low).
[00373] Network Map Toolbar
[00374] The Network Map Toolbar provides quick access to items for working with the network map of Figure 29.
[00375] The Select icon is the default and allows for selections on the network map.
[00376] The Camera icon provides a means to capture and save an image of the current network map. Click this icon to bring up a Save dialog. Browse to desired location, name the file and click the OK button. Click the Cancel button to close the dialog without saving.
[00377] The Rubberband Zoom icon shown in Figure 32 provides a means to zoom in to a specific desired area, see Figure 32. Click this icon, then click on the network map, hold and drag the cursor to desired rectangular area and release. Network map is zoomed to view only this area.
[00378] The Zoom In icon increases the size of displayed items viewed on the network map.
[00379] The Zoom Out icon decreases the size of displayed items viewed on the network map.
[00380] The Undo Previous Zoom icon reverses the last zoom action performed.
[00381] The Panning icon provides a means to manually move the viewing area of the network map. Click on this icon and then click and hold on the map while moving the cursor to desired viewing area.
[00382] The Re-center icon provides a means to manually center the viewing area of the network map at a particular point. Click on this icon and then click on desired center point on the network map.
[00383] The Restore Original Zoom icon resets the zoom level of the network map to the default size.
[00384] The Refresh icon redraws the page.
[00385] The Node Box icon provides a means to encapsulate a set of node(s) within a defining box. Click this icon, then click on the network map, hold and drag to enclose desired node(s) and release (the box can be renamed).
[00386] The Connector icon provides a means to connect two items (nodes or boxes) on the network map with a straight line. Click this icon, then click the first item to be connected on the network map and click the second item to be connected. A straight-line connector is shown between the two items. [00387] The Elbow Connector icon provides a means to connect two items (nodes or boxes) on the network map with a right-angled connector. Click this icon, then click the first item to be connected on the network map and click the second item to be connected. A right-angled set of line segments is show between the two items (if the two items are in line with each other - horizontally or vertically - a straight line is shown).
[00388] The Delete icon deletes any selected items on the network map. If a selected item has connector(s), the connector(s) will also be deleted.
[00389] Network Map Components
[00390] The left panel of the Network Map tab provides elements to define and identify information for items on the network map.
[00391] Nodes Palette
[00392] The Nodes palette as shown in Figure 33A provides icons depicting the various types of nodes which may be added to the network map. Click on one of the icons and then click on network map at desired location. The node is added to the network map.
[00393] Attributes Palette
[00394] The Attributes palette shown in Figure 33B provides (changing) icons depicting types of attributes which may be assigned to a node on the network map.
[00395] The Services icon is the first icon in the set.
[00396] Click on this icon and then click on the node on the network map for which service(s) are to be assigned. This will bring up the Services dialog.
[00397] Select the desired service(s) by selecting them in list. To select multiple, hold down shift key (for multiple in succession) or Ctrl key (for separate items). Unselecting a service removes its association with the node.
[00398] Click the OK button to apply the services to the selected node (or click the Cancel button to close the dialog without changing).
[00399] The Key Terrain icon is the second icon in the set.
[00400] Click on this icon and then click on the node on the network map for which key terrain value is to be (un)assigned. This will bring up the Key Terrain dialog.
[00401] Select the desired value.
[00402] Click the OK button to apply the level to the selected node (or click the Cancel button to close the dialog without changing).
[00403] The Operating System icon is the third icon in the set. [00404] Click on this icon and then click on the node on the network map for which the operating system is to be assigned. This will bring up the Operating System dialog.
[00405] Select the desired operating system. Unselecting the operating system removes its association with the node.
[00406] Click the OK button to apply the operating system to the selected node (or click the Cancel button to close the dialog without changing).
[00407] Mouseover Feedback
[00408] The Mouseover Feedback area shown in Figure 34 displays all of the current attributes / properties / connections / boxes for the network map item over which the cursor is currently "hovering" whether or not the Network Map - Options menu item is checked for that property.
[00409] Editable Properties
[00410] The Editable Properties area shown in Figure 35 provides the means to edit the properties (such as name, addresses, operating system) appropriate to an item selected on the network map. Enter values in appropriate fields.
[00411] Network Map Graph Area Actions
[00412] This section describes the various actions which can be taken on the Network Map
Graph Area while defining the network.
[00413] Position Items on Network Map
[00414] Nodes and Boxes may be moved on the network map.
[00415] Click on the item to be moved (hold down mouse button and drag to encompass an area with multiple items).
[00416] Click again on selected item/area and drag to desired location.
[00417] Arrange Item
[00418] Nodes and Boxes may be arranged in a specified order for viewing. This is important when two or more items overlap because it determines which one(s) are in front or back.
[00419] Right Click on the item shown in Figure 36 A to be arranged and select the desired Item Ordering option. The item is moved forward or backward as appropriate.
[00420] Lock Item
[00421] Nodes and Boxes may be "locked" so that they cannot be selected, edited or moved within the network map.
[00422] Right Click on the item to be locked and select Locked as shown in Figure 36B. A checkmark will appear beside the option and the item cannot be selected, edited or moved. Properties will still be shown in Mouseover Feedback area when hovering over the item.
[00423] Right Click on the item and select this option again to unlock the item as shown in Figure 36B.
[00424] Defining the Network Map
[00425] Use the Network Map Graph Area to define the network map for the mission as shown in Figure 37.
[00426] Select from various node types and add to the network map (see Nodes Palette section).
[00427] Select from various attributes and apply to each node on the map (see Attributes Palette section).
[00428] Connect nodes on the map (see Network Map Toolbar section).
[00429] Use boxes to group nodes on the map (see Network Map Toolbar section).
[00430] Enter the appropriate properties for each item on the map (see Editable Properties section).
[00431] Events
[00432] The Events component include a list of various events for the mission.
[00433] To access the Events, click on the Events tab of Figure 1.
[00434] Events Menu Items
[00435] When the Events tab is selected, the items under the Events Menu at the top of the page of Figure 38 A are available.
[00436] Save Events
[00437] Click Events - Save... to save the current information on this tab for the mission.
[00438] Import Events
[00439] To import events for the mission,
[00440] Click Events - Import... to bring up an Open dialog. (Clicking the Cancel button will close the dialog without continuing the import process.)
[00441] Browse to the folder containing desired event file.
[00442] Select the event file and click the Open button. The event information is imported into this mission.
[00443] Click the Cancel button to exit the dialog without importing the events.
[00444] Export Events
[00445] Click Events - Export... to save the current event information in the correct format for the game environment. An Export Successful dialog is presented showing that the msel.json file has been saved in the Events folder within the export folder under the main mission folder. Click the OK button to acknowledge and close the dialog.
[00446] Defining Events
[00447] Defining Events as shown in Figure 38 A.
[00448] Add Event
[00449] Initially there are several "blank" events shown in the event list with a default Event ID as shown in Figure 38B.
[00450] To add additional Event(s), click the Add button 260 at the top of the page of Figure 38 A. Another "blank" row is added to the event list, with the Event ID defaulted to the number events now in the list.
[00451] Delete Event
[00452] To delete an Event,
[00453] Select one of the text fields in the row to be deleted. The Delete button 262 will show the Event ID of the row selected.
[00454] Click the Delete button. The row is removed from the list.
[00455] Duplicate Event
[00456] Select one of the text fields in the row to be duplicated. The Duplicate button 264 will show the Event ID of the row selected.
[00457] Click the Duplicate button. The row is duplicated as an additional row is added to the list.
[00458] Edit Event Field
[00459] General Text Fields
[00460] For most fields in the list,
[00461] Select the field in the list. This will place any current text in the edit box above the list.
[00462] Make changes to the text and click the accept icon. The text is now updated in the list. (Click the cancel icon to clear the edit box and leave the field as it was in the list.)
[00463] Note: Select the field in the list again in order to edit.
[00464] Event Type Field
[00465] To edit the Event Type,
[00466] Click in the Event Type field for the event to be defined/changed. This will bring up the Type dialog with the Event ID shown in the title. [00467] Select the desired Event Type and click the OK button. The dialog closes and selected Event Type is shown in the list. (Click the Cancel button to close the dialog without making any change.)
[00468] Steps Field
[00469] To edit the Steps,
[00470] Click in the Steps field for the event to be defined/changed. This will bring up the Steps for id dialog 270 as shown in Figure 39 with the Event ID shown in the title.
[00471] Select the desired Step(s) in the list - using shift key to select consecutive rows or Ctrl key to select multiple / disjoint rows.
[00472] Click the OK button to close the dialog and apply the steps to the event. (Click the Cancel button to close the dialog without changing the event.)
[00473] To "reorder" the Steps,
[00474] Click in the Steps field for the event to be defined/changed.
[00475] Click the Cancel button the Steps dialog to close the dialog.
[00476] Now use the edit box above the list to enter / reorder the steps - they must be separated by a space.
[00477] Associated Files Field
[00478] To view / edit the Associated Files:
[00479] Click in the Associated Files field for the event being defined/edited. This brings up the Select Associated Files for: id dialog.
[00480] To add a file to the list,
[00481] Click the Add icon at the top of the dialog. This brings up an Open dialog.
[00482] Browse to / select the desired file.
[00483] Click the OK button to close the dialog. The file is listed in the Select Associated Files for: id dialog. (Or click the Cancel button to close the dialog without adding a file.)
[00484] Repeat steps a - c to add additional file(s).
[00485] Click the OK button. The dialog closes and the first associated file is shown in the list.
[00486] To see multiple associated files, click in the Associated Files field again to bring up dialog / view the list in edit box at top of page.
[00487] To delete a file from the list,
[00488] Select the file(s) to be deleted in the dialog list
[00489] Click the Delete icon at the top of the dialog. The file(s) are removed from the list [00490] Click the OK button. The dialog closes and the deleted file(s) are no longer associated with the event.
[00491] Sorting Events
[00492] To sort the event list by Event ID, click on the Event ID table header for the list. The events are sorted in ascending alphabetic order. Click the header again to sort in descending alphabetic order.
[00493] Quizzes
[00494] The Quizzes as shown in Figure 40A define a set of questions to be used in "qualifying" for the mission. The questions are multiple choice in nature, may be associated with multiple missions, and optionally provide Key Words.
[00495] To qualify for a mission in the game environment, the player must first answer several quiz questions as depicted in Figure 40 A and Figure 40B.
[00496] To access the Quizzes, click on the Quizzes tab.
[00497] Quizzes Menu Items
[00498] When the Quizzes tab is selected, the items under the Quizzes Menu at the top of the page are available.
[00499] Save Quizzes
[00500] Click Quizzes - Save Quizzes to save the current information on this tab for the mission.
[00501] Search for Questions
[00502] There is a "master" list of quiz questions delivered with Orion. These questions may be searched for relevancy to the current mission and included / modified as desired.
[00503] Click Quizzes - Search for Questions to bring up the Search for Questions dialog.
[00504] Enter the desired criteria to search by (any combination):
[00505] Difficulty Level (Any does not constrain the search)
[00506] Key Word(s) (specific key words with which question(s) may have been tagged) [00507] Question Content (a string of text to be found in the question)
[00508] Answer Content (a string of text to be found in any answer field - correct, wrong options, or detailed answer)
[00509] Mission(s) (specific mission(s) for which question(s) may have been tagged).
[00510] Skill Content (a string of text to be found in any skill associated with the question(s)).
[00511] Click the Search button to review results (or click the Cancel button to close dialog without searching).
[00512] If no results are found, No Results Found confirmation is displayed - click the OK button to close the confirmation dialog.
[00513] Otherwise, the Select Questions to Include... dialog is displayed as shown in Figure 40C.
[00514] Select the desired question(s).
[00515] To select ALL of the questions click the Select All button 284 at the top of the dialog. All Select boxes are checked.
[00516] To deselect ALL of the questions click the Deselect All button 286 at the top of the dialog. All Select boxes are unchecked.
[00517] To select individual question(s), click the Select box 282 by the desired question(s). Click the box again to unselect.
[00518] Click the Apply button 288 to include the selected question(s) in this mission (or Cancel to close the dialog without including any of the questions). Any questions included are automatically added to the quiz question list for the mission.
[00519] Export Quiz Questions
[00520] Click Quizzes - Export Quiz Questions to export any entered quiz question(s) in the correct format for game environment. An Export Successful dialog is presented showing that the questions which have been defined within this mission have been saved as the Quiz Q&A.csv file in the export folder under the main mission folder.
[00521] If there are no quiz questions defined in this mission, an Export Quiz Questions dialog is brought up.
[00522] Click the OK button to acknowledge the information. No export file is created.
[00523] Defining the Quizzes
[00524] The Quizzes tab 290 as illustrated in Figure 40D is divided into the following areas and described below.
[00525] Question navigation
[00526] Question entry
[00527] Associated Skills
[00528] List of Existing Questions (with associated skill(s))
[00529] Assign Question Difficulty
[00530] The question difficulty may be chosen at any time by selecting the desired value from the Difficulty drop down. If the question could apply to multiple difficulty levels, choose "Any".
[00531] Define Question
[00532] Enter text for the Question, multiple choice answers (Answer, Wrongl, Wrong2,
Wrong3), Detailed Answer (explanation), Applicable Mission IDs, and Key Words for the question displayed. The total number of questions currently defined for the mission is displayed at the top of the page.
[00533] Quiz Question Skills
[00534] Associate Skill(s) with Question
[00535] To associate skill(s) with a question
[00536] Select the desired question in the list of questions. The list of skills available are displayed.
[00537] Click the Select checkbox next to desired skill(s) OR click in the associated skill text field. The associated skill is highlighted.
[00538] To disassociate the skill, click on the checkbox (OR associated skill text field) again to clear it.
[00539] Search for Skill
[00540] To search for specific skill, enter text to search for in the Search Skills for: field. The number of results (if any) is shown to the right.
[00541] Each skill that matches the entered text is outlined.
[00542] Use the Down Arrow and Up Arrow buttons to scroll to next / previous occurrence of the entered text. The one that is scrolled to is outlined with a slightly darker outline.
[00543] Add Question
[00544] To add a question, click the Add Question button 292 at the top of the tab. This will provide a blank question entry area for defining the new question. It will also add a "blank" question to the Existing Questions list. Fill in the fields for this new question.
[00545] Delete Question(s)
[00546] To delete the currently displayed question, click the Delete Question button 294 at the top of the tab. This brings up the Delete Quiz Question dialog.
[00547] Click the OK button to delete the question (or click the Cancel button to close the dialog without deleting question). The currently displayed question is removed.
[00548] Next Question [00549] To display the next question, click the Next button 296. The information for the next question is displayed.
[00550] Previous Question
[00551] To display the previous question, click the Previous button 298. The information for the previous question is displayed.
[00552] Select Existing Question to Display
[00553] To display a specific question, click on the row in the Existing Questions list and the question information for that question is displayed.
[00554] In addition, if the question text is too long to fit in the list, it will end with "..." and holding the cursor over that row will provide a tool tip with the entire text of the question.
[00555] Checklists
[00556] The Checklists tab 300 as shown in Figure 41 provides a convenient look at what is left to do in order to construct a complete mission.
[00557] The checklist on the left shows each of the Orion components with their status(es).
[00558] Green icon - completed
[00559] Red icon - not completed
[00560] Click on an item in the left list to view a description of what needs to be done for that item to be complete. For example - Clicking on Mission Order - Describe Situation may show what needs to be done for Mission Order.
[00561] DETAILED SYSTEM DESIGN
[00562] Main Window
[00563] The java application is a single window with a tabbed pane, one tab for each key item
(plugin). The title contains the name of the current mission.
[00564] The main menu may include one or more of the following elements:
[00565] (1) File
[00566] (2) New Mission... (to create a new mission - define organization, title, description, number, level of difficulty)
[00567] (3) Open Mission... (to open an existing mission)
[00568] (4) Save Mission (to save all data for the currently selected mission)
[00569] (5) Save Mission As ... (to save the currently selected mission as a different mission allowing change of the parameters similar to New Mission...)
[00570] (6) Verify Mission Data... (to verify completion status of mission data) [00571] (7) Advanced Mission Settings... (to maintain various mission settings not specific to a plugin)
[00572] (8) Export Mission for game environment (saves mission and then exports all available mission data in game environment format to export folder under the mission)
[00573] (9) View Orion Users Guide (provide link to view user's guide)
[00574] (10) About Orion... (provide Orion version / licensing information)
[00575] (11) Quit (exit the application)
[00576] (12) Menu items for each plugin (only accessible when the tab for that plugin is selected) - see individual plugins for details.
[00577] Defining a New Mission
[00578] File - New Mission... brings up a dialog shown in Figure 42 where the user can define the basic information for the mission, such as one or more of:
[00579] (1) Organization
[00580] (2) Mission Name
[00581] (3) Mission Number
[00582] (4) Operation
[00583] (5) Mission Level of Difficulty (Easy, Medium, or Hard) - default = Easy
[00584] (6) Mission Type (Offensive, Defensive, Battle Room) - default = Defensive [00585] Once defined, the user is provided a way to save the mission to a user-selected directory. A subdirectory named <ORG>_Mission_<Difficulty Level> is created with data directories underneath for each plugin and for exporting the mission to game environment.
[00586] Opening an Existing Mission
[00587] File - Open Mission... brings up a file selector dialog to browse to and select the desired mission.
[00588] Saving a Mission as Another
[00589] File - Save Mission As... brings up a dialog similar to defining a new mission for the user to update basic information for the mission and select where to save it. The same naming conventions for the mission is followed and all current mission data is copied to the new mission.
[00590] Verify Mission Data
[00591] File - Verify Mission Data will examine the current mission data and present a dialog indicating the completeness and consistency of the data. The verification will include [00592] Verify mismatches in exported files (run "orchestration script" to verify files are complete / consistent)
[00593] Verify all required pieces for the Mission have been defined
[00594] Figure 43 is one example of how the dialog 320 may appear.
[00595] Advanced Mission Settings
[00596] File - Advanced Mission Settings... brings up a dialog 330 to allow update of mission settings (which are defaulted as shown in Figure 44) including:
[00597] Total Game Points - default to 1000 for Easy, 2000 for Medium, 3000 for Hard
[00598] Hint Points - default to 20%
[00599] Default Location - default to 1
[00600] Boot Timeout - default to 20 minutes
[00601] Publish Timeout - default to 40 minutes
[00602] Time Limit - default to 480 minutes
[00603] Tooltips is provided for each item.
[00604] Export Mission for Game environment
[00605] File - Export Mission for game environment writes out the appropriate mission files in format for the target game environment. The user is then presented with a dialog 340 as depicted in Figure 45 indicating the success of the export.
[00606] View Orion User's Guide
[00607] File - View Orion User's Guide will bring up the document in a browser window.
[00608] About Orion
[00609] File - About Orion... brings up a dialog with information about Orion including:
[00610] Orion version number
[00611] Owner copyright information
[00612] Any End User License Agreement (EULA) information required
[00613] Link to Orion User's Guide
[00614] Quit
[00615] File - Quit exits the Orion application. The mission designer is warned if there is any unsaved data.
[00616] Mission Development Methodology Walkthru
[00617] Central to the use of Orion is the Mission Development Methodology Walkthru pane showing in Figure 46 the order of how to develop a mission correctly. The pane is always available with status of where the mission designer is for each step. The pane is also able to be hidden.
[00618] An example of what the Mission Development Walkthru may look like is shown in Figure 47. This panel is the Orion implementation of the Mission Design process.
[00619] Red/Green lights indicate the status of each step. The arrow icon to the right of the pane allows the pane to be "hidden" (collapsed). While collapsed the arrow icon allows the pane to be displayed (expanded) again.
[00620] Welcome Page
[00621] The main "Welcome" page as depicted in Figure 48 for Orion provides help information overview as well as detailed information on each of the plugins. This information is maintained as html files for each item (overview / plugin).
[00622] Mission Setup Plugin
[00623] Information is gathered at the beginning of mission definition that defines what the mission is about. The Mission Setup Plugin has several tabs allowing the mission designer to define various aspects of the mission, such as one or more of:
[00624] (1) User Pre-requi sites
[00625] (2) Key Terrain
[00626] (3) Mission Characteristics
[00627] (4) Training Outcomes
[00628] (5) Associate Mission Related File(s)
[00629] Mission Setup Menu Items
[00630] Print Mission Setup -brings up a print dialog with available printer(s) and print text to selected printer.
[00631] Example of concept:
[00632] Mission:
[00633] Name: Disable Botnet
[00634] Number: 1
[00635] Operation: Goatherd
[00636] Difficulty: Easy
[00637] Type: Offensive
[00638] Mission Files:
[00639] Mission Storyline: Mission_l_Storyline.docx [00640] Mission Video: Mission l_intro.mp4
[00641] Mission Thumbnail: mission 1. thumb
[00642] Mission Coin: Goatherdlnsignia.png
[00643] Other Mission Info:
[00644] Mission 1 Design.pptx
[00645 ] Mission_ l_Easy_Solution.docx
[00646] User Pre-requisites:
[00647] Computer Languages
[00648] - Command Line Interface
[00649] Key Terrain:
[00650] Cyber Key Terrain: Maravian National Bank
[00651] Protocols: telnet, vnc, ssh
[00652] Systems/Data: Command and Control Server
[00653] Baseline Flows:
[00654] Defensive Posture:
[00655] Mission Characteristics:
[00656] Summary: Infiltrate the enemy's environment and disable the command and control web server responsible for thousands of defrauded victims.
[00657] Purpose: The purpose of Mission 1 is to have a basic offensive mission where early users can get used to Athena and how the game environment works. They will learn skills applicable to Certified Ethical Hacking (CEH).
[00658] Certification: CEH
[00659] Domain: Commercial
[00660] Type of Play: One-Sided
[00661 ] Number of Players : One or more Players
[00662] Max Players: 10
[00663] Max Trainers: 5
[00664] Network Size: Small (-10 VMs)
[00665] Network Complexity: Basic
[00666] Location: 23.3790 N / 113.7633 E
[00667] Training Outcomes:
[00668] - Learn about password cracking techniques [00669] - Manage firewall rules
[00670] - Identify Industrial Control Systems ("ICS")
[00671] Mission Setup Plugin Tab
[00672] User Pre-requisites
[00673] This subtab 350 shown in Figure 49 allows the mission designer to select which skills a player should have prior to playing the mission.
[00674] A search capability is provided for finding skills in the list.
[00675] All currently selected skills are displayed in a list to the right.
[00676] Key Terrain
[00677] This subtab 360 shown in Figure 50 provides for entry of data which define the key terrain of the mission.
[00678] Mission Characteristics
[00679] This subtab 370 shown in Figure 51 provides for entry of data which define the general characteristics of the mission.
[00680] Selected items are "highlighted" and fields default as follows:
[00681] Domain - Critical Infrastructure
[00682] Type of Play - Two-sided
[00683] Number of Players - One or More Players
[00684] Max Players - 5 / Max Trainers - 3
[00685] Tactics - Offensive
[00686] Network Size - Small
[00687] Training Outcomes Tab
[00688] This tab 380 shown in Figure 52 allows the mission designer to define training outcomes for the mission.
[00689] Associate Mission Related File(s)
[00690] This tab 390 shown in Figure 53 allows the mission designer to associate file(s) with the mission. Some files are known/required while others are optional information.
[00691] Mission Order Plugin
[00692] The Mission Order Plugin provides the capability to define information for the mission, such as one or more of:
[00693] (1) Situation (overall scenario and threat intelligence information)
[00694] (2) Mission Core Competencies (required to go on mission) [00695] (3) Execution Instructions: User Credentials and Tools
[00696] (4) Sustainment (how long mission should take and requirements on the end of the mission)
[00697] (5) Command & Control (who commands the mission above the Team Lead and any reporting requirements)
[00698] (6) Rules of Engagement (ROE)
[00699] (7) References
[00700] Mission Order Menu Items
[00701] Save Mission Order (save data for this plugin in the associated data folder under the mission)
[00702] Import Mission Order from game environment (browse to select existing mission order file from game environment)
[00703] Export Mission Order to game environment (export mission order in game environment format to export folder under the mission)
[00704] Mission Order Plugin Tab
[00705] The Mission Order plugin tab 400 provides a styled editor depicted in Figure 54 for entering the various components of the Mission Order as "Pages" within the tab.
[00706] The mission designer is able to switch between the pages by selecting the desired page to be displayed.
[00707] A styled editor is implemented to provide tools to edit / format the text for the Mission Order. Capabilities include:
[00708] Basic operations: cut, copy, paste, undo, redo
[00709] Lists: bulleted and numbered lists
[00710] Hyperlinks: insert/edit/delete a hyperlink
[00711] Paragraph styles
[00712] Character styling: Bold, Italic, Underline, Color
[00713] Mission Objectives Plugin
[00714] The Mission Objectives plugin provides the capability to define Mission Objectives, associated badges/skills and umpire detections.
[00715] Mission Objectives Menu Items
[00716] Save Mission Objectives (save data for this plugin in the associated data folder under the mission). [00717] Import Objectives from Game environment (browse to select existing mission objectives (objectives.yaml) file from Game environment - the file contains associated solutions/hints and umpire detections as well).
[00718] Export Objectives to Game environment (export mission objectives in Game environment format (objectives.yaml) to export folder under the mission - the file contains associated solutions/hints and umpire detections as well). The umpire check tag will automatically be assigned a value of "objective<#>" for each objective where <#> is the objective number. See Export Objective Scoring section for details on formatting scores for Game environment.
[00719] Export Objective Scoring
[00720] Orion Objective Scoring and GAME ENVIRONMENT Objective Game Points are two separate entities. In one embodiment, Orion Objective Scores for "normal" Missions need to be a percentage of total game points (adding up to 100%). This total number plus the Objective % points is used to determine the game points for the objective.
[00721] objective game points = (percentage_points * total_game_points) / 100
[00722] For Battle Room Missions, there may be a large number of objectives (30+) which are not necessarily related to each other, so Orion maintains the concept of just a relative score (i.e. 1-10) rather than a percentage. This number is used (as a "percentage" of objective points) along with the total number of points for the game to determine the game points for the objective.
[00723] objective game points = (battle_room_score * total_game_points) / sum(all battle_room_score)
[00724] Hint/Solution points is determined based on a total hint percentage - the percentage of points of objective that would be lost if ALL hints for that objective were taken.
[00725] hint score = ((hint_percentage * objective game points) / 100) / number of hints for that objective
[00726] "Normal" Mission Example
[00727] This is an example based on mission 1 easy (services_0.yaml).
[00728] Given: (1) total game points of 1000; (2) hint % of 20
[00729] objectives:
[00730] - tag: objectivel
[00731] description: Gather credentials [00732] percentage_points: 20
[00733] points: 200
[00734] coreskills:
[00735] - points: 1
[00736] name: Command_Line_Interface
[00737] - points: 1
[00738] name: 0/S_Differences(WindowslLinux_methods)
[00739] - points: 2
[00740] name: Offensive_Tools
[00741] - points: 1
[00742] name: Network_Services_and_Communication
[00743] - points: 1
[00744] name: Pentesting
[00745] hints:
[00746] - points: 8
[00747] text: "You may or may not see an IP address ...\n"
[00748] - points: 8
[00749] text: "You can use nmap to find open ports ...\n"
[00750] - points: 8
[00751] text: "You may be awarded Objective 2 early ...\n"
[00752] - points: 8
[00753] text: "For ssh, use one of these commands ...\n"
[00754] - points: 8
[00755] text: "For vnc: Use one of these commands ...\n"
[00756] - points: 8
[00757] text: "For telnet, you can use the following.. An"
[00758] - tag: objective2
[00759] description: Gain access to C2 server
[00760] percentage_points: 30
[00761] points: 300
[00762] coreskills:
[00763] - points: 20 [00764] name: Command_Line_Interface
[00765] - points: 20
[00766] name: System_Administration
[00767] - points: 20
[00768] name: Network_Tools
[00769] hints:
[00770] - points: 0
[00771] text: "Login to the C2 server. ..An"
[00772] - tag: objective3
[00773] description: Kill Web Service to disable botnet
[00774] percentage_points: 50
[00775] points: 500
[00776] coreskills:
[00777] - points: 2
[00778] name: Command_Line_Interface
[00779] - points: 2
[00780] name: System_Administration
[00781] hints:
[00782] - points: 50
[00783] text: "To verify that this machine is acting ..An"
[00784]
[00785] Battle Room Mission Example
[00786] This is a "partial" example based on battleroom-host easy services_0.yaml.
[00787] Given: (1) total game points of 1000; (2) hint % of 20; (3) assume 20 objectives half with a battleroom_score of 2 and half with battle_room_score of 3
[00788] objectives:
[00789] - tag: objectivel
[00790] description: From the Kali client, use the ping program/utility ...
[00791] points: 60
[00792] battle_room_score: 3
[00793] prompt: 900
[00794] category: Reconnaissance [00795] coreskills:
[00796] - name: Command_Line_Interface
[00797] points: 1
[00798] - name: Core_Cybersecurity_Tools
[00799] points: 1
[00800] - name: Network_Tools
[00801] points: 1
[00802] - name: 0/S_Knowledge_and_Commands
[00803] points: 1
[00804] hints:
[00805] - points: 6
[00806] text: ping is the program used to send ICMP diagnostic messages, ...
[00807] - points: 6
[00808] text: ping LinuxServerIP
[00809] - tag: objective2
[00810] description: From the Windows client, use the ping program/utility to ...
[00811] points: 40
[00812] battle_room_score: 2
[00813] prompt: 900
[00814] category: Reconnaissance
[00815] coreskills:
[00816] - name: Command_Line_Interface
[00817] points: 1
[00818] - name: Core_Cybersecurity_Tools
[00819] points: 1
[00820] - name: Network_Tools
[00821] points: 1
[00822] - name: 0/S_Knowledge_and_Commands
[00823] points: 1
[00824] hints:
[00825] - points: 4
[00826] text: ping is the program used to send ICMP diagnostic messages, ... [00827] - points: 4
[00828] text: ping LinuxServerIP
[00829] Mission Objectives Plugin Tab
[00830] The Mission Objectives plugin tab 410 shown in Figure 55 provides the capability to add, delete, or reorder mission objectives in a list.
[00831] The list of current Objective(s) with associated skill(s) is displayed separately on the left side of the plugin tab to allow for easy viewing / switching between objectives.
[00832] For each objective:
[00833] The mission designer is able to assign score points based on mission type (see below).
[00834] A list of available skills are displayed. The mission designer selects skill(s) from the master database list (see Orion - Database IDD) and assign their score (between 1-10).
[00835] The mission designer selects Umpire Detection(s) to associate with the objective and assign values to it (see below).
[00836] Objective Scoring
[00837] Mission scoring is based on the Mission Type.
[00838] Mission: The Objective score is relative. The sum of all Objective scores in the mission must add up to 100%. If the user slides one up, the score(s) is adjusted to re-equal 100%.
[00839] Battle Room: Each Objective score is independent with a value of 1-10 (10 being most difficult)
[00840] Umpire Detections
[00841] The mission designer is able to define the Umpire Check associated with the Objective.
[00842] Detection events are combined based on the selected value (and, or, data) - default = and.
[00843] Optionally, a callback function to be called with the return of any detection events. The "master list" of callback functions is defined in Orion - Database IDD.
[00844] Optionally, other Umpire Check(s) which are required to be accomplished prior to this one being activated.
[00845] Indicate whether the Umpire Check should be persistent.
[00846] Add, Delete or Edit associated umpire detection events.
[00847] Add Detection Event
[00848] Add Detection by clicking Add Detection button 412. This will bring up a selection list of all detections currently NOT associated with this objective.
[00849] Select the desired detection and click OK. This displays a dynamic dialog with the fields associated with the detection for the user to fill in.
[00850] Click OK to add the detection (or cancel to exit dialog without adding the detection) [00851] Delete Detection
[00852] Delete a detection event by selecting the detection(s) in the list of current detections and clicking the Delete Detection button 414. User is asked to confirm (Are you sure?)
[00853] Edit Detection
[00854] Edit a detection event by selecting the detection in the list of current detections and clicking the Edit Detection button 416.
[00855] This brings up the same dynamic dialog as Add Detection - with current values filled in.
[00856] Modify value(s) as desired and click OK to save / Cancel to exit without saving [00857] Mission Solutions and Hints Plugin
[00858] The Mission Solutions and Hints Plugin provides the capability to define the mission solution and associated hint(s) for each objective. The hints provide multiple statements to lead the player to the solution. The solution walks the player through the objective.
[00859] Mission Solutions and Hints Plugin Menu Items
[00860] Save Mission Solutions (save data for this plugin in the associated data folder under the mission)
[00861] Mission Solutions and Hints Plugin Tab
[00862] The Mission Solutions and Hints plugin tab 420 depicted in Figure 56 provide the selection of objective and then defining the solution and associated hints for that objective using a styled editor.
[00863] A styled editor is implemented to provide tools to edit / format the text for the
Objective Solution / Hint. Capabilities include:
[00864] Basic operations: cut, copy, paste, undo, redo
[00865] Lists: bulleted and numbered lists
[00866] Hyperlinks: insert/edit/delete a hyperlink
[00867] Paragraph styles
[00868] Character styling: Bold, Italic, Underline, Color
[00869] For Hint(s), a list of current hint(s) with first part of the text of the hint are displayed. [00870] To view the full hint without bringing up the editor dialog, hover over the hint text area.
[00871] The mission designer is able to add a hint as follows:
[00872] Click the Add Hint button 422 which adds another "line" to the list of hint(s).
[00873] Choose the type of hint
[00874] Text - just text information
[00875] Variable - based on a mission variable (e.g. remote service)
[00876] For Text hint, a dialog 430 such as shown in Figure 57 is used:
[00877] click on the text area (which will display the first part of existing text for the hint) to bring up the styled editor
[00878] When completed, click OK (or Cancel to close dialog without saving any changes) [00879] For Variable hint -
[00880] Select the appropriate mission variable from list presented (or select + ADD to add a new variable - process described elsewhere)
[00881] "Rows" for each of the variable values is shown.
[00882] Click on desired text area to bring up styled editor to edit the hint (similar to Text hint) [00883] To delete a hint, click the Delete button next to that hint.
[00884] Network Map Plugin
[00885] The Network Map Plugin will provide the capability to define the network for the mission.
[00886] Network Map Menu Items
[00887] Save (save data for this plugin in the associated data folder under the mission)
[00888] Create Network Map Report (user readable file with data describing the network - see Network Map Report section below)
[00889] Clear Drawing (delete all items on the network map)
[00890] Duplicate (make a duplicate of selected item(s))
[00891] Zoom to Fit (zoom as necessary to fit ALL nodes on the screen)
[00892] Snap to Grid (indicate whether items added to network map will automatically snap to the grid)
[00893] Snap Size (provide choice of size - e.g. 1mm - 10mm)
[00894] Options to show various details on the network map
[00895] Show OS Type [00896] Show Simple Name
[00897] Show IP Address(es)
[00898] Show Control IP Address
[00899] Show Services
[00900] Network Map Report
[00901] This report is a file of network map node information including:
[00902] Node Name
[00903] Node Type
[00904] Operating System
[00905] IP Address(es)
[00906] Service(s) Provided
[00907] Tool(s) Installed
[00908] Connections
[00909] Network Map Plugin Tab
[00910] The Network Map plugin tab 440 shown in Figure 58 provides tools to
[00911] Manage the network map layout (select item(s), zoom control, panning control, refresh, delete item(s), move item(s) on the map).
[00912] View the network map as it would be seen in the game environment game UI as well as full development mode.
[00913] Select from various Node types and add to the network map. These nodes are icons representing the type of node (e.g. server, firewall, printer...).
[00914] Apply various attributes to a selected node on the map. The attributes available are based on node type.
[00915] Adversary (mark whether this node is an adversary)
[00916] Services (list from "master list" as defined in Orion - Database IDD)
[00917] Network Interfaces (random vs regular)
[00918] Tools (list from "master list" as defined in Orion - Database IDD)
[00919] Operating System (OS) (current list of operating systems is provided in Appendix A) [00920] Key Terrain: Not Key Terrain, Tier 1 (High), Tier 2 (Medium), Tier 3 (Low)
[00921] Connect nodes on the map.
[00922] Select a node or connector on the map illustrated in Figure 58 and provide a list of properties which can be edited for that item based on the of node. See Appendix B for set of properties per node.
[00923] Network Map Viewing Modes
[00924] Two modes of network map viewing are available to the mission designer.
[00925] Developer View - allows editing of the network map and shows ALL network nodes [00926] Game View - view only mode showing just the nodes which will also be shown in the Game environment Game UI
[00927] Click on the "Show Game View" button 442 to switch to view-only mode (with "floating" nodes such as Trainer being hidden). The button name will change to "Show Developer View". Clicking the newly named button will switch back to Developer View.
[00928] Adversary Node
[00929] The mission designer is able to indicate whether a node is an adversary.
[00930] Right-click on a node will bring up a popup dialog where Adversary may be
(un)checked.
[00931] Services for Node
[00932] The mission designer is able to manage the service(s) associated with a node on the map.
[00933] Right-click on a node brings up a popup dialog to select from options to manage the service(s).
[00934] Add... brings up the selection list of those services NOT associated with this node yet and they can pick ONE to add (or cancel out of process)
[00935] Display field(s) for service (from master service database) and allow user to fill in /edit based on "type" of data for each field.
[00936] Click OK to save configuration for this service with the node (or click Cancel to close dialog without saving). Orion will keep track of whether all required fields were filled in - for completeness checks.
[00937] Delete - brings up the selection list of those which ARE associated with this node and they can pick multiple and delete (with "are you sure? confirmation)
[00938] Configure - brings up the selection list of those which ARE associated with this node and they can pick ONE and configure it with the dynamic dialog as shown for Add...
[00939] Network Interfaces for Network
[00940] The mission designer is able to manage the network interfaces for a network on the map. This is a key step because the system configures the network and sets up interfaces and IP addresses through this process.
[00941] Right-click on a network "box" brings up a popup dialog to define the interface(s). A dialog displaying network interface fields is displayed for the mission designer to define.
[00942] Node Tools Installed
[00943] The mission designer is able to manage which tool(s) are installed for a node on the map. When a user selects a tool for a host computer on the network diagram, the system sets up all the installation configuration for that to be completely installed.
[00944] Right-click on a node brings up a popup dialog to select the tool(s).
[00945] Brings up the selection list of tools available (from "master list" - see Orion- Database
IDD). Multiple selection is allowed.
[00946] Click the OK to associate the tool(s) (or click Cancel to close dialog without selecting the tool(s)).
[00947] Node Key Terrain
[00948] The mission designer is able to associate a Key Terrain value for a node on the map.
[00949] Right-click on a node brings up a popup dialog to select the desired Key Terrain value for this node.
[00950] Events Plugin
[00951] The Events Plugin provides the capability to define the various events for a mission.
[00952] Events Menu Items
[00953] Save (save data for this plugin in the associated data folder under the mission)
[00954] Events Plugin Tab
[00955] The Events plugin tab 450 as shown in Figure 59 provides tools to define the various events for a mission.
[00956] The tab displays the list in time order and provide capability to Add, Edit, Delete events in the list
[00957] To Add an event:
[00958] Click the Add Event button 452. This will bring up the Define Event dialog with blank fields to be selected / filled in.
[00959] Select Event Frequency (Run Once, Periodic) and enter start time (and interval if Periodic)
[00960] Select Event Type
[00961] Select Event Category from dropdown of values from the master Adversary Event list (see Orion - Database IDD)
[00962] Once the Category is selected, the Event Technique dropdown is populated with corresponding values from the master list, select value
[00963] Based on the Category / Technique, appropriate fields is shown for any necessary parameters (from master list). Enter / select values desired.
[00964] Click on the Target(s) field to bring up Select Target(s) dialog and choose the desired target(s) from list of appropriate^?) network map nodes and any existing event targets emitted by earlier Delivery events.
[00965] Click OK to close target dialog and display selected target(s) for the Event. (Click Cancel to close dialog without making any changes)
[00966] Click OK to close dialog and add the event to the Event List in time order (or Cancel to close without adding).
[00967] To Edit an existing event:
[00968] Double-click on the desired row of the list to bring up the Define Event Dialog with current values filled in.
[00969] Make changes desired (similar to Add Event) and click OK to close the dialog and apply the changes to the Event List (or Cancel to close without making changes).
[00970] To Delete an existing event:
[00971] Click on the desired row to highlight it.
[00972] Click the Delete Selected Event button 454 to delete it (with confirmation)
[00973] Athena Q&A Plugin
[00974] The Athena Q&A Plugin provides the capability to define questions and answers for the Artificial Intelligence (AI) in-game Advisor for Game environment (ATHENA). The types of Q&A include
[00975] Mission Specific (specific to (the objectives of) this mission)
[00976] General Cybersecurity (general offense / defense)
[00977] Athena Persona
[00978] Athena Q&A Plugin Menu Items
[00979] Save Athena Q&A (save data for this plugin in the associated data folder under the mission)
[00980] Search for Athena Q&A (search the "master" Athena Q&A data file (see Orion - Database IDD for format) and select which one(s) to include) [00981] Brings up dialog 460 with criteria to search by criteria for search:
[00982] Category (Mission Specific, General Cybersecurity, Athena Persona)
[00983] Question Content
[00984] Answer Content
[00985] Mission Objective (if entered, then the current Mission Number is assumed)
[00986] Click Search to obtain dialog with list of Q&A matching the criteria entered
[00987] Select the desired Q&A and click Apply to bring into the Mission (click Cancel to close dialog without applying any Q&A)
[00988] Export Athena Q&A File (export Athena Q&A in Game environment format to export folder under the mission)
The format for the file(s) is given in the Orion - Database IDD.
[00989] Athena Q&A Plugin Tab
[00990] The Athena Q&A plugin tab 460 shown in Figure 60 provides the capability to Add /
Delete / Edit questions of each category type.
[00991] Data for each Q&A include:
[00992] Category (as indicated by individual tab for each)
[00993] Question
[00994] Answer
[00995] Mission Objective (for Mission Specific category only)
[00996] NOTE: for Mission Specific Q&A - the current Mission Number is assumed.
[00997] Quizzes Plugin
[00998] The Quizzes Plugin provides the capability to pull existing questions from the master quiz database and to define new quiz questions for the mission. The questions are multiple choice in nature, may be associated with multiple missions, and optionally provide Key Words, and a Source for the information.
[00999] Quizzes Plugin Menu Items
[001000] Save Quizzes (save data for this plugin in the associated data folder under the mission)
[001001] Search for Questions (search the "master" quiz data file (see Orion - Database IDD for format) and select which one(s) to include in this mission)
[001002] Brings up dialog with criteria to search by
[001003] criteria for search: [001004] Difficulty Level (Any, Easy, Medium, Hard)
[001005] Key Word(s)
[001006] Question Content
[001007] Answer Content (search in any of the answer fields)
[001008] Mission(s)
[001009] Skill Content
[001010] Click Search to obtain dialog with list of quiz questions matching the criteria entered [001011] Select the desired question(s) and click Apply to bring into the Mission (click Cancel to close dialog without applying any questions)
[001012] Export Quiz Questions (export Quiz Questions in Game environment format to export folder under the mission)
[001013] The format for the file is given in the Orion - Database IDD. (a csv formatted file will also be exported for easier reading by a user.)
[001014] Quizzes Plugin Tab
[001015] The Quizzes plugin tab 470 shown in Figure 61 provides the capability to Add /
Delete / Edit quiz questions.
[001016] Data for each question include:
[001017] Difficulty Level (Any, Easy, Medium, Hard)
[001018] Question
[001019] Answer
[001020] 3 wrong answers
[001021] Detailed Answer
[001022] Applicable Mission ID(s)
[001023] (optional) Key Word(s)
[001024] In addition, skills earned for the question may be selected from list of Badges / Skills.
[001025] The list of current Quiz Question(s) with associated skill(s) is displayed separately to allow for easy viewing / switching between questions.
[001026] Mission Library Plugin
[001027] The Mission Library plugin 480 shown in Figure 62 allows the mission designer to check in/out any of the mission associated files from the repo.
[001028] File(s) may be selected on the local file system and drag/dropped into the appropriate field. A file may also be selected on this page and drag/dropped onto the local file system. [001029] The following files are required for the mission to be complete. Only a single file may be selected for these - if another file is selected, then it will replace the existing one (user is warned).
[001030] Mission Storyline
[001031] Mission Video
[001032] Mission Thumbnail
[001033] Mission Coin
[001034] Other Mission Info is for optional file(s) which the mission designer may want included / saved with the mission. Multiple files may be selected.
[001035] To remove file(s) from this list, select desired file(s) in list and click the Remove Selected File(s) button.
[001036] Double-click on a file to open it with default system editor (if available).
[001037] Battle Room Support
[001038] Battle Rooms are places where players can do small technical tasks and reinforce their knowledge in different areas. Missions are end-to-end scenarios where players need to be able to figure out how to solve a problem with a set of tools. As such, Battle Rooms need to be handled differently in some areas. They do not require some of the mission complexity.
[001039] The following plugins do not apply for Battle Rooms since they are virtual environments that are used outside of a mission context:
[001040] Storyline
[001041] Mission Setup
[001042] Mission Order
[001043] Athena Q&A
[001044] Quizzes
[001045] The tabs for these plugins are not be displayed nor considered for checklists / completeness when the mission designer chooses the Mission Type as Battle Room upon mission creation / save as (see screenshot under Defining a New Mission section of this document).
[001046] In addition, the following differences might also be considered
[001047] Objective Scoring (see Objective Scoring section of this document)
[001048] Work roles
[001049] VM Construction [001050] There are several files which Orion needs to output for the VEM to be able to create/launch a mission. Details of data fields for each file are defined in the Orion - Game environment VEM IDD. These subsections define where the data is derived from within Orion.
[001051] Blueprint
[001052] The blueprint.yaml file depicted in Table 3 outlines all virtual machines (VMs) to be created by the VEM. The following table shows how/where Orion gets data to write this out.
Figure imgf000058_0001
yaml tag parent tag Orion Source
type location
X location
y location
sen i es 1 Vpiv aied
mirror <vmName>
<mirror id> mirror
vm <mirror id>
src <mirror id>
dst <mirror id>
credentials <vmName> From template - not modifiable by user
user credentials From template - not modifiable by user
password credentials From template - not modifiable by user
connections —
from connections
to connections
points connections
X points
y points
Table 3
[001053] Mission Definition
[001054] The mission definition.yaml file depicted in Table 4 contains the high-level mission data for the mission.
Figure imgf000059_0001
yaml tag parent tag Orion Source
numberTrainers missionCharacteristics Mission Setup - Mission Characteristics missionParameters —
missionType mis sionParameters Create Mission / Save Mission As...
Dialog
trainingType mis sionParameters Create Mission / Save Mission As...
Dialog (for regular mission - offensive or defensive)
operation mis sionParameters Create Mission / Save Mission As...
Dialog
missionNumber mis sionParameters Create Mission / Save Mission As...
Dialog
missionTitle mis sionParameters Create Mission / Save Mission As...
Dialog (name)
description mis sionParameters Create Mission / Save Mission As...
Dialog
difficulty mis sionParameters Create Mission / Save Mission As...
Dialog
totalGamePoints mis sionParameters Advanced Settings Dialog
hintPercentage mis sionParameters Advanced Settings Dialog
thumbnail mis sionParameters Mission Setup - Mission Files
Table 4
[001055] Network
[001056] The Mission network.yaml file depicted in Table 5 lists the networks (both static and random) for the mission.
Figure imgf000060_0001
yaml tag parent tag Orion Source gateway route
static network
<ip> Static
domain_name network
servers
<ip> domain_name_server
s
dhcp network
domain_name dhcp
servers
<ip> domain_name_server
s
range dhcp
<from_ip> range
<to_ip> range
routers dhcp
<ip> routers
random_networks —
network random_networks
routes random_networks
dest route
gateway route
members random_networks
<vmName> members
policies —
<domain> policies
members <domain>
<vmName> members
domain_name <domain>
servers yaml tag parent tag Orion Source
<ip> domain_name_server
s
Table 5
[001057] Objectives
[001058] The Mission objectives.yaml file depicted by Table 6 provides the objectives with their skills / scores / hints solutions and umpire check detections for the mission.
[001059] The table below indicates where the information for the file comes from within
Orion.
Figure imgf000062_0001
yaml tag parent tag Orion source
umpire —
<umpiretag> umpire Matches objective tag
description <umpiretag> - not required - Orion will not generate
type <umpiretag> Mission Objectives - Umpire Detections
require <umpiretag> Mission Objectives - Umpire Detections
<umpireCheckTag> require Umpire Detections - dialog to pick from other existing
<umpiretag>(s)
callback <umpiretag> Mission Objectives - Umpire Detections
persist <umpiretag> Mission Objectives - Umpire Detections
detection <umpiretag> Mission Objectives - Umpire Detections
name detection Umpire Detections - pick from list of existing detections in detection library
type detection From detection library
target detection Associated from Network Map OR picked in Mission
Objectives - Umpire Detections
persist detection Umpire Detections
args detection Umpire Detection Configuration dialog
Table 6
[001060] Services
[001061] There are two services yaml files shown by Table 7 which represent data that is loaded at different times into the VEM. The services in the network definition are broken into two layers: Services and Services_0. The first is the basic network which stores a working base definition. The second is an overlay (Services_0) that adds the dynamic variation in the mission to include threat actors. This lets you do many different things to your basic network definition. When a new threat attack comes out, like WannaCry, you can take your service template for a hospital network and quickly create a WannaCry Services_0 overlay, for example.
Figure imgf000063_0001
yaml tag parent tag Orion Source
variables Added when a Service is added to the network
Diagram or a detection is added to an Objective.
A master list is kept and is editable from the network diagram or objective plugin
The services/detection database will need to store the "type" (random_choice, machine_ip, . . .) of the variables used in the service/detection definition
The implication is that there is a list of Variable Types available.
assign variables Handled by the add/edit capability.
<assign> assign Handled by the add/edit capability.
For "Variable Hint" - a set of assign variables may be created for each "choice" of the variable chosen for the hint with the value being the text for that choice (e.g. ssh_hint, vnc_hint, telnet_hint).
[value(s)] <assign> Handled by the add/edit capability.
random_string variables Handled by the add/edit capability.
<random_var> random_string Handled by the add/edit capability.
[value(s)] <random_var> Handled by the add/edit capability.
random_choice variables Handled by the add/edit capability.
<choose> random_choice Handled by the add/edit capability.
[value(s)] <choose> Handled by the add/edit capability.
machine_ips variables Handled by the add/edit capability.
<server> machine_ips Handled by the add/edit capability.
[value(s)] <server> Handled by the add/edit capability. yaml tag parent tag Orion Source
random_ip variables Handled by the add/edit capability.
<net_sim> random_ip Handled by the add/edit capability.
[value(s)] <net_sim> Handled by the add/edit capability.
assignments — Unclear need better definition from Ryan assign_extend assignments See above
<assign> assign_extend See above
For "Variable Hint" an assign_extend is created named hint<#>_obj<#> with <partl> being the variable associated with the hint and <part2> being "hint"
<partl> <assign> See above
<part2> <assign> See above
assign_service assignments See above
<service> assign_service See above
<name> <service> See above
users List of users and trainers that can log into a system
The Orion users will only specify the # of users/trainers. Orion will automatically create the user and trainer ids based on a starting UID for users/trainers
The Orion users will also have the ability to add a random user to a node/host
<vmName> users Name of the Orion node.
Trainer node floats just like random services and ?????
<user> <vmName> User is able to assign the UID for random users uid <user> See above yaml tag parent tag Orion Source
configurations Top level tag is included from Orion.
The serviceName sub tags are added automatically by Orion when the user adds a service to a node on Network Map.
<serviceName> configurations Default name comes from the service database and assigned to a node. Multiple nodes can use one configuration.
name <serviceName> Assumed to be preconfigured in the services database
- no user input is required - Orion will "copy" if it exists
service <serviceName> Assumed to be preconfigured in the services database
- no user input is required - Orion will "copy" if it exists
type <serviceName> Assumed to be preconfigured in the services database
- no user input is required - Orion will "copy" if it exists
event <serviceName> When service type = event - must select an event
(name is in event block)
Events come from events database
Values within the event block is filled in by popping up a dynamic (?) dialog box.
<eventblock> <serviceName> See above
stage <serviceName> Assumed to be preconfigured in the services database
- no user input is required - Orion will "copy" if it exists
ssh <serviceName> User is presented with a dialog box asking for the user passwd and sudo subtags
Defaults is supplied in the services database user ssh See above
passwd ssh See above yaml tag parent tag Orion Source
sudo ssh See above
description <serviceName> Assumed to be preconfigured in the services database
- no user input is required - Orion will "copy" if it exists
reset <serviceName> Assumed to be preconfigured in the services database
- no user input is required - Orion will "copy" if it exists
transform <serviceName> Comes from the services database
Values within the transform block is filled in by popping up a dynamic dialog box.
<transformblock> transform See above
salt <serviceName> Comes from the services database.
Values within the salt block is filled in by popping up a dynamic dialog box.
<saltblock> salt See above
services Contains a list of network nodes that the user has placed on the Network Map.
Only nodes that have a service at the corresponding level is included in this list
<vmName> services See above
<serviceName> <vmName> List of services that the user associated with the node on Network Map
Originated from the services database
yaml tag parent tag Orion Source
groups Yaml files authored by Orion will only include groups for random services. (Orion will put in an entry for each node that uses a static service.)
Random Services is added as a floating node on the map. Users is able to add a service to random list of hosts (nodes)
and the ability to specify how many hosts in the list will get the random service.
<group> groups See above
hosts <group> See above
<vmName> hosts See above
services <group> See above
<serviceName> <group> See above
random <group> See above
<serviceName> random See above
Table 7
[001062] Base level - services. yaml: loaded into the VEM cache at startup - any services w/o variables or random information
[001063] Level 1 - services_0.yaml: loaded into VEM at Mission Game startup - any services with variables or random information
[001064] Orion treats these two files similarly - the difference being that all variable / random information and events must be in services_0.
[001065] For adversary events an adversary "configuration" and "service event" is inserted.
[001066] VEM
[001067] The Mission vem.yaml file depicted in Table 8 is the data fixture file which contains basic metadata for the mission.
Figure imgf000068_0001
yaml tag parent Orion Source
tag
boot_timeout fields Advanced Settings Dialog
default_location fields Advanced Settings Dialog
description fields Derived from mission number on New Mission / Save
Mission As... Dialog
name fields Derived from mission number/difficulty level on New
Mission / Save Mission As... Dialog
networking fields Static text
polymorphic_ctype fields Static text
publish_timeout fields Advanced Settings Dialog
timelimit fields Advanced Settings Dialog
base_services fields Static text
services fields Static text
objectives fields Static text
mission_parameters fields Static text
model — Static text
pk Derived from information on New Mission / Save
Mission As ... Dialog <ORG>_mission_<#>_<Difficulty>
serial_id — yymmdd<##> when file is to be written out
Table 8
[001068] vsphere
[001069] The Mission vsphere.yaml file depicted in Table 9 is the data fixture file for the virtualization model. It describes and includes the data necessary for the virtualization implementation.
Figure imgf000069_0001
yaml tag parent Orion Source
tag
model — Static text
pk Derived from information on New Mission / Save Mission As...
Dialog
<ORG>_mission_<#>_<Difficulty>
serial_id — yymmdd<##> when file is to be written out
Table 9
[001070] CYBER RANGES
[001071] In one embodiment of the invention, the output of the Orion or mission builder system creates a single training mission that may train one or more users. The training mission may be reused and added to a library of other missions for use by students to learn key concepts.
[001072] In other embodiments, created missions may be grouped into a cyber training or exercise event in order to create a realistic virtual world or cyber range complete with simulated users, simulated mission systems, simulated data and emulated adversaries - all connected to a simulated Internet.
[001073] Cyber ranges may begin as racks of computer hardware that can be reconfigured into various network topologies and overlaid with different baseline network services and traffic profiles to coincide with the business or mission of the enterprise environment. Cyber ranges operate in a closed loop environment as a necessity to introduce malware safely and prevent damage to live networks. The cyber range provides for network routing, defensive tools, mission applications and an overlay of virtual users that automatically perform the activities of real users to generate realistic simulated network traffic.
[001074] The formation of a cyber range with which to train users with missions created by the system is a virtualization approach that essentially mimics the real-world internet of backbone routing and connectivity between organizations. Modules and interconnection across virtual environments are essential components of the cyber range where training missions take place.
[001075] In the cyber range embodiment of the system, many mission designer users could create, for example, an ICS mission, a radar mission, a military range mission, and other required missions to meet their training objectives. No coordination is required when each mission designer or student latches on to the cyber range, which interconnects all users just like in the real world. This approach supports key training objectives such as trust relationships while creating a single plane from which the exercise support teams, like the opposing force, can operate using the cyber range across many teams at once. In addition, scenarios involving multi-team echelon reporting can be easily exercised. In other embodiments, entire virtual e- cities can be configured for training in a straightforward and tractable problem.
[001076] In one embodiment, key features of the cyber range embodiment where created missions are used for training may include: (1) the unique ability to "swing and latch" on to networks via the configuration of a single interface for each add-on environment; (2) a uniquely small footprint but is able to handle a large traffic volume and dynamically sets up routes to new environments; (3) scaling to different sizes, such as 2, 8, 16, and 28 mission networks; and (4) quick deployment due to the modular design.
[001077] The approach supports event configurations across multiple game training systems, meaning, it can be launched in one cyber training system and connected to an entirely different system as long as the physical switches that connect the two clusters or systems has the swing network (a virtual local area network distribution identification or "VLANID") defined. These clusters "could" be geographically different as long as the layer 2 traffic on each end of the WAN connection is unscathed. In addition, multiple cyber ranges might even be connected.
[001078] The cyber range can be latched on to user emulation modules, Internet corpus modules, management and monitoring modules, and opposition forces (live and automated) modules.
[001079] Using a cyber range of the invention, such as with missions created by the Orion/mission builder system of the invention as depicted in Figure 70, a port is tagged (perhaps in the blueprint.yaml) as an INTER-MISSION network as well as in the CORE (a baseline or background routing for Internet services) module. The Virtual Environment Manager (VEM) then creates a VLAN to work between the mission modules (Each key aspect of the system has a port tagged for Intermission of communications).
[001080] In the backbone of cyber range, dynamic routing is used via a Border Gateway Protocol (BGP). Each content server and infrastructure device is configured to redistribute the routes of all the interfaces configured. If a new sub-interface is created with a new IP address that has new web content, for example, that IP address is automatically injected into the BGP route table for client consumption. This dynamic routing capability facilitates content, traffic, and purpose flexibility into the product. Any and all valid IP space can be used in the virtual environment resulting in the ability to present regions of the world based on IP address.
[001081] The routing flexibility and solution design makes it very robust, such as a red team adversary emulation mission training platform. Adversary presence using the cyber range embodiment for training missions is simple, flexible, and realistic. A red team can control and distribute Domain Name System (DNS) and route characteristics into the cyber range much like today's adversaries have demonstrated. This feature provides the ability not only to demonstrate the characteristics of a specific attack method, but also enables it to be representative by a particular region and footprint.
[001082] Figure 63 illustrates a cyber range backbone of Internet services, user emulation, and auto-inject mission modules that support the training environment. All missions created by the system hang off of the configured cyber range and are playable using missions developed with the Orion/mission builder system of the invention along with activities interjected by a live- inject mission module used as an opposing force ("OPFOR") or adversary training ground.
[001083] INET-NNW, NW, WNW,W,etc routers act essentially as "ISP's" "ISP" for each mission. If a small training exercise is run with 8 teams, the teams are able to share the "Internet Corpus" module which contains DNS, NTP, SMTP, FTP, and all the browsable websites for Lariat and/or other user traffic generators. The teams can also share the "User Emulation" module, which is a customized Lariat setup to support the teams in some capacity. Then there are two other pick-offs; the "Live Inject" module, and the "Auto-Inject" module. The "Live Inject" module is for a Red (OPFOR) team to work from with access to all the teams via the Cyber Range. The "Auto-Inject" module is for any automated attacks/objectives we put into the game that can be scripted and scored automatically.
[001084] This embodiment of the system, where missions are integrated and used by multiple teams as shown in Figure 64 and 65, provides the ability to modularize many of the disparate functions typically used in training exercises where the prior art used manual configuration. For example, one set of users can be responsible for maintaining the "Internet Corpus" module with updated websites, patches and updates, and added functionality if required and do releases of that module for use in the cyber range. This same approach can be used with the "User Emulation," "Live Inject," and "Auto-Inject modules." These features of the cyber range embodiment allow the mission designers to pick from a catalog of things needed for an interconnected training exercise. All the routing is automatic for each mission provided. [001085] It will be understood that the above described arrangements of apparatus and the method there from are merely illustrative of applications of the principles of this invention and many other embodiments and modifications may be made without departing from the spirit and scope of the invention as defined in the claims.

Claims

WHAT IS CLAIMED IS:
1. A mission design system for a cyber training system, comprising:
at least one user station, each user station comprising at least one display device, at least one user input device, a memory device, and at least one controller;
and
at least one mission server, said at least one mission server comprising:
a processor configured to execute machine readable code;
at least one communication interface which permits said server to transmit information to said at least one user station and to receive information from said at least one user station;
a plurality of mission plugins which define a baseline cyber mission environment, each of the mission plugins configured to generate a corresponding graphical user interface of mission design information, each of the graphical user interfaces have a plurality of variable mission parameters; and
machine readable code executable by said processor configured to cause said mission server to cause said at least one user station to display said graphical user interfaces when selected by a user, to receive input from the user of values for one or more of said plurality of variable mission parameters and, in response thereto, to generate a cyber mission file which is usable by a cyber training system to implement a cyber training mission defined by said cyber mission file within a cyber environment.
2. The mission design system in accordance with Claim 1 wherein said mission plugins comprise one or more of a storyline plugin, a mission setup plugin, a mission order plugin, a mission objective plugin, a solution and hints plugin, a network map plugin, an events plugin, a questions and answers plugin, a quizzes plugin and a mission library plugin.
3. The mission design system in accordance with Claim 2 wherein when said storyline plugin is selected, a storyline user interface is presented to prompt the user to enter a storyline text and/or a storyline video.
4. The mission design system in accordance with Claim 2 wherein when said mission setup plugin is selected, a creating new mission interface is presented to the user prompting the user to enter a mission name, a mission description, and a mission difficulty of a new mission.
5. The mission design system in accordance with Claim 2 wherein when said mission order plugin is selected, a mission order interface is presented to the user prompting the user to generate written mission orders.
6. The mission design system in accordance with Claim 5 wherein said written mission orders includes at least one of a situation describing an overall mission scenario and threat intelligence information, a mission describing specifics of the mission and skills or core competencies required to go on the mission, execution describing a mission purpose, key tasks, an end state and concept of operations including user credentials and tools, sustainment describing how long the mission should take and any requirements on an end of the mission, command and control describing who commands the mission and any reporting requirements, rules of engagement describing any rules about the mission environment including an out-of- bounds list, tools, traffic generation and key services that must be maintained, and references describing what mission players should read.
7. The mission design system in accordance with Claim 2 wherein when said mission objective plugin is selected, a mission objective interface is presented to the user prompting the user to generate written mission objectives including at least one of an objective name, an objective score, skills addressed by an objective, an umpire check associated with an objective that forms a basis for automated scoring, and (4) List of Existing Objectives.
8. The mission design system in accordance with Claim 2 wherein when said solution and hints plugin is selected, a solution and hints interface is presented to the user prompting the user to generate a solution and at least one associated hint for each mission objective.
9. The mission design system in accordance with Claim 2 wherein when said network map plugin is selected, a network map interface is presented to the user presenting network map option choices to the user.
10. The mission design system in accordance with Claim 9 wherein said network map options include an option to show OS type, an option to show a simple name, an option to show an IP address, an option to show services, and an option to show key terrain.
11. The mission design system in accordance with Claim 10 wherein when the show key terrain option is selected, a key terrain ranking is displayed on network elements in the network map designated as key elements.
12. The mission design system in accordance with Claim 2 wherein when said events plugin is selected, a mission objective interface is presented to the user prompting the user to add events to the mission and to designate an event type.
13. The mission design system in accordance with Claim 2 wherein when said questions and answers plugin is selected, a questions and answers interface is presented to the user prompting the user to add, delete or edit questions and answers including a question and answer category.
14. The mission design system in accordance with Claim 2 wherein when said quizzes plugin is selected, a quizzes interface is presented to the user prompting the user to prepare written questions each with a user selected difficulty ranking and at least one associated skill.
15. The mission design system in accordance with Claim 2 wherein when said mission library plugin is selected, a mission library interface is presented to the user presenting mission information concerning a mission selected by the user from a local file system.
16. The mission design system in accordance with Claim 1 wherein said machine readable code is configured to generate a top-level graphical user interface displayed on said user station, the top-level graphical user interface displaying the mission plugins as a plurality of selectable tabs, wherein when a user selects one of the tabs, the corresponding graphical user interface for the selectable plugin is displayed.
17. The mission design system in accordance with Claim 16 wherein the selectable tabs corresponding to the mission plugins on the top-level user interface each include a designator having a visual indication of a completion state of data needed by the corresponding mission plugin to have a complete mission.
18. The mission design system in accordance with Claim 2 further comprising a mission database, the mission database storing saved missions and preconfigured missions.
19. The mission design system in accordance with Claim 18 wherein when said mission setup plugin is selected, an open existing mission interface is presented to the user prompting the user to select an existing mission from the mission database.
20. The mission design system in accordance with Claim 1 further comprising a global data logger configured to maintain and share information among the plurality of mission plugins.
PCT/US2018/047509 2017-08-24 2018-08-22 System for dynamically provisioning cyber training environments WO2019040613A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762549779P 2017-08-24 2017-08-24
US62/549,779 2017-08-24

Publications (1)

Publication Number Publication Date
WO2019040613A1 true WO2019040613A1 (en) 2019-02-28

Family

ID=65439883

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/047509 WO2019040613A1 (en) 2017-08-24 2018-08-22 System for dynamically provisioning cyber training environments

Country Status (1)

Country Link
WO (1) WO2019040613A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112055026A (en) * 2020-09-11 2020-12-08 湖南泛联新安信息科技有限公司 Network target range physical environment construction method and system
CN112579195A (en) * 2020-12-02 2021-03-30 中国电子科技集团公司第二十八研究所 Control interface configuration scheme saving and calling system facing to work habits of controllers
RU2747099C1 (en) * 2019-12-19 2021-04-26 Публичное акционерное общество "Сбербанк России" (ПАО Сбербанк") Automated cybersecurity event testing system
CN116074114A (en) * 2023-03-06 2023-05-05 鹏城实验室 Network target range defense efficiency evaluation method, device, equipment and storage medium
CN116455990A (en) * 2023-06-19 2023-07-18 中国人民解放军陆军航空兵学院 LVC contract training environment construction method and system based on information interaction bus
CN116684301A (en) * 2023-06-26 2023-09-01 北京永信至诚科技股份有限公司 Method, system, equipment and storage medium for realizing cross-range task collaboration

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050192870A1 (en) * 2000-06-21 2005-09-01 Applied Systems Intelligence, Inc. Method and system for providing an intelligent goal-oriented user interface to data and services
US20090254842A1 (en) * 2008-04-05 2009-10-08 Social Communication Company Interfacing with a spatial virtual communication environment
US20110257961A1 (en) * 2010-04-14 2011-10-20 Marc Tinkler System and method for generating questions and multiple choice answers to adaptively aid in word comprehension
US20150040033A1 (en) * 2004-10-14 2015-02-05 Cerulean Studios, Llc System and method for intergrating advanced multimedia features within an instant messaging environment
US20150143374A1 (en) * 2010-05-28 2015-05-21 Bromium, Inc. Securing an internet endpoint using fine-grained operating system virtualization
US20150231502A1 (en) * 2014-02-19 2015-08-20 International Business Machines Corporation Game adjustments through crowdsourcing
US20170032694A1 (en) * 2015-07-28 2017-02-02 Architecture Technology Corporation Real-time monitoring of network-based training exercises
US20170103783A1 (en) * 2015-10-07 2017-04-13 Google Inc. Storyline experience
US20170140660A1 (en) * 2015-09-24 2017-05-18 Circadence Corporation Mission-based, game-implemented cyber training system and method
US9697355B1 (en) * 2015-06-17 2017-07-04 Mission Secure, Inc. Cyber security for physical systems

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050192870A1 (en) * 2000-06-21 2005-09-01 Applied Systems Intelligence, Inc. Method and system for providing an intelligent goal-oriented user interface to data and services
US20150040033A1 (en) * 2004-10-14 2015-02-05 Cerulean Studios, Llc System and method for intergrating advanced multimedia features within an instant messaging environment
US20090254842A1 (en) * 2008-04-05 2009-10-08 Social Communication Company Interfacing with a spatial virtual communication environment
US20110257961A1 (en) * 2010-04-14 2011-10-20 Marc Tinkler System and method for generating questions and multiple choice answers to adaptively aid in word comprehension
US20150143374A1 (en) * 2010-05-28 2015-05-21 Bromium, Inc. Securing an internet endpoint using fine-grained operating system virtualization
US20150231502A1 (en) * 2014-02-19 2015-08-20 International Business Machines Corporation Game adjustments through crowdsourcing
US9697355B1 (en) * 2015-06-17 2017-07-04 Mission Secure, Inc. Cyber security for physical systems
US20170032694A1 (en) * 2015-07-28 2017-02-02 Architecture Technology Corporation Real-time monitoring of network-based training exercises
US20170140660A1 (en) * 2015-09-24 2017-05-18 Circadence Corporation Mission-based, game-implemented cyber training system and method
US20170103783A1 (en) * 2015-10-07 2017-04-13 Google Inc. Storyline experience

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PRICE ET AL.: "Asset Criticality in Mission Reconfigurable Cyber Systems and its Contribution to Key Cyber Terrain", PROCEEDINGS OF THE 50TH HAWAII INTERNATIONAL CONFERENCE ON SYSTEM SCIENCES, January 2017 (2017-01-01), pages 6042 - 6051, XP055578426 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2747099C1 (en) * 2019-12-19 2021-04-26 Публичное акционерное общество "Сбербанк России" (ПАО Сбербанк") Automated cybersecurity event testing system
CN112055026A (en) * 2020-09-11 2020-12-08 湖南泛联新安信息科技有限公司 Network target range physical environment construction method and system
CN112055026B (en) * 2020-09-11 2022-06-10 湖南泛联新安信息科技有限公司 Network target range physical environment construction method and system
CN112579195A (en) * 2020-12-02 2021-03-30 中国电子科技集团公司第二十八研究所 Control interface configuration scheme saving and calling system facing to work habits of controllers
CN112579195B (en) * 2020-12-02 2022-10-04 中国电子科技集团公司第二十八研究所 Control interface configuration scheme saving and calling system facing to work habits of controllers
CN116074114A (en) * 2023-03-06 2023-05-05 鹏城实验室 Network target range defense efficiency evaluation method, device, equipment and storage medium
CN116455990A (en) * 2023-06-19 2023-07-18 中国人民解放军陆军航空兵学院 LVC contract training environment construction method and system based on information interaction bus
CN116455990B (en) * 2023-06-19 2023-08-22 中国人民解放军陆军航空兵学院 LVC contract training environment construction method and system based on information interaction bus
CN116684301A (en) * 2023-06-26 2023-09-01 北京永信至诚科技股份有限公司 Method, system, equipment and storage medium for realizing cross-range task collaboration
CN116684301B (en) * 2023-06-26 2024-01-30 北京永信至诚科技股份有限公司 Method, system, equipment and storage medium for realizing cross-range task collaboration

Similar Documents

Publication Publication Date Title
US11600198B2 (en) System for dynamically provisioning cyber training environments
WO2019040613A1 (en) System for dynamically provisioning cyber training environments
US11666817B2 (en) Mission-based, game-implemented cyber training system and method
KR102113587B1 (en) Mission-based game-implemented cyber education system and method
US10083624B2 (en) Real-time monitoring of network-based training exercises
US10803766B1 (en) Modular training of network-based training exercises
Leitner et al. AIT cyber range: flexible cyber security environment for exercises, training and research
Yamin et al. Serious games as a tool to model attack and defense scenarios for cyber-security exercises
WO2018175551A1 (en) Mission-based, game-implemented cyber training system and method
de Leon et al. ADLES: Specifying, deploying, and sharing hands-on cyber-exercises
Gustafsson et al. Cyber range automation overview with a case study of CRATE
Katsantonis et al. Design and evaluation of cofelet-based approaches for cyber security learning and training
Balon et al. Cybercompetitions: A survey of competitions, tools, and systems to support cybersecurity education
González-Amarillo Ecosystem for the deployment and management of virtual laboratories based on the standard IMS LTI
Weitl-Harms et al. A Systematic Mapping Study on Gamification Applications for Undergraduate Cybersecurity Education.
US20230335425A1 (en) System for dynamically provisioning cyber training environments
Edwards Cyber automated red team tool
Mennecozzi et al. Bridging the gap: adapting a security education platform to a new audience
Jackson et al. Cisco Certified DevNet Associate DEVASC 200-901 Official Cert Guide
Krovich et al. A cloud based entitlement granting engine
Mavridis et al. FGAWeb: A web platform for administrating flexible games in education
Luse et al. Teaching Tip: Hackalytics: Using Computer Hacking to Engage Students in Analytics
Fernandez-Tarraga et al. Training on Social Media Cybersecurity Skills in the Healthcare Context
Thompson et al. Create and Host Cyber Competition Using the Preliminary Persistent Cyber Training Environment (PCTE)
Sánchez et al. Methodology for Automating Attacking Agents in Cyber Range Training Platforms

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18849116

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18849116

Country of ref document: EP

Kind code of ref document: A1