WO2011142933A2 - Real time mission planning - Google Patents
Real time mission planning Download PDFInfo
- Publication number
- WO2011142933A2 WO2011142933A2 PCT/US2011/032459 US2011032459W WO2011142933A2 WO 2011142933 A2 WO2011142933 A2 WO 2011142933A2 US 2011032459 W US2011032459 W US 2011032459W WO 2011142933 A2 WO2011142933 A2 WO 2011142933A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- mission
- computer
- assets
- program code
- elements
- Prior art date
Links
- 238000000034 method Methods 0.000 claims description 62
- 238000007726 management method Methods 0.000 claims description 29
- 238000004590 computer program Methods 0.000 claims description 11
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000010006 flight Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 95
- 230000008569 process Effects 0.000 description 45
- 238000004891 communication Methods 0.000 description 37
- 238000012545 processing Methods 0.000 description 35
- 230000015654 memory Effects 0.000 description 23
- 230000002085 persistent effect Effects 0.000 description 14
- 230000008859 change Effects 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 7
- 239000004744 fabric Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- PEDCQBHIVMGVHV-UHFFFAOYSA-N Glycerine Chemical compound OCC(O)CO PEDCQBHIVMGVHV-UHFFFAOYSA-N 0.000 description 1
- 101100384355 Mus musculus Ctnnbip1 gene Proteins 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000000446 fuel Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000004886 process control Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 210000003813 thumb Anatomy 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
Definitions
- the present disclosure relates generally to missions and, in particular, to a method and system for mission planning.
- the present disclosure provides a method and system for defining, executing, and modifying a mission in real-time.
- Mission planning involves various resources and tasks that come together to form a mission. Different resources may be deployed for different tasks that make up a mission, for
- Missions are typically scripted manually and completed offline before the missions are run by a computer system. Many of these current systems are specific to a single mission type, such as an area search mission for example. The existing solutions are not practical for dynamic applications where missions need to be created and modified on the fly. Once execution of the mission begins, the mission cannot be modified during execution.
- the different advantageous embodiments provide a system comprising a number of computers, a graphical user interface, first program code stored on the computer, and second program code stored on the computer.
- the graphical user interface is executed by a computer in the number of computers .
- the computer is configured to run the first program code to define a mission using a number of mission elements.
- the computer is configured to run the second program code to generate instructions for a number of assets to execute the mission and monitor the number of assets during execution of the mission.
- the different advantageous embodiments further provide a method for mission planning.
- a computer receives a user definition of a mission.
- a mission script is generated using the user definition and a number of mission elements.
- the number of mission elements is assigned to a number of assets. Execution of the number of mission elements by the number of assets is controlled. The number of assets executing the mission is monitored.
- the different advantageous embodiments further provide computer program product for mission planning comprising a computer recordable storage medium and program code stored on the computer recordable storage medium.
- the program code receives a user definition of a mission, generates a mission script using the user definition and a number of mission elements, assigns the number of mission elements to a number of assets, generates instructions for the number of assets to execute the mission using the number of mission elements, and monitors the number of assets executing the mission.
- Figure 1 is an illustration of a network planning
- FIG. 2 is an illustration of a data processing system in accordance with an advantageous embodiment
- Figure 3 is an illustration of a mission planning
- FIG. 4 is an illustration of a mission planner client in accordance with an advantageous embodiment
- FIG. 5 is an illustration of a mission planner server in accordance with an advantageous embodiment
- Figure 6 is an illustration of a graphical user interface in accordance with an advantageous embodiment
- Figure 7 is an illustration of a graphical user interface in accordance with an advantageous embodiment
- Figure 8 is an illustration of a graphical user interface in accordance with an advantageous embodiment
- Figure 9 is an illustration of a graphical user interface in accordance with an advantageous embodiment
- Figure 10 is an illustration of a flowchart illustrating a process for mission planning in accordance with an advantageous embodiment .
- Figure 11 is an illustration of a flowchart illustrating a process for defining a mission in accordance with an
- Figures 1-2 exemplary diagrams of data processing environments are provided in which the advantageous embodiments of the present invention may be implemented. It should be appreciated that Figures 1-2 are only exemplary and are not intended to assert or imply any limitation with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made.
- Network planning environment 100 is a network of computers in which embodiments may be implemented.
- Network planning environment 100 contains network 101, which is the medium used to provide communications links between various devices and computers connected together within network planning environment 100.
- Network 101 may include connections, such as wire, wireless communication links, or fiber optic cables.
- mission planning system 102 In the depicted example, mission planning system 102
- Mission planning system 102 may include a number of servers and a number of clients, in an advantageous embodiment. In another advantageous embodiment, servers and clients of mission planning system 102 may be located in a number of locations and connect to network 101, for example. Mission planning system 102 may include, for example, data processing systems, personal
- Number of users 104 interacts with mission planning system 102 to plan mission 106, monitor mission 106 in a real-time environment, and/or modify mission 105 in a real-time
- Number of users 104 may include human users, robotic and/or machine users, program code, and/or any other suitable user of mission planning system 102.
- Number of assets 108 may exchange data with mission planning system 102 through a wireless communications link or through any other type of communications link using network 101.
- number of assets 108 may be computers and/or assets controlled by a computer.
- Assets controlled by a computer may include, for example, without limitation, mobile assets, vehicles, sensors, machines, charging stations, and/or any other suitable asset.
- Network planning environment 100 may include additional servers processes, clients processes, and other devices not shown.
- Number of assets 108 may execute number of tasks 110 in order to accomplish mission 106, in an illustrative example.
- Mission 106 may be a collection of tasks.
- Number of tasks 110 may combine to form mission 106, for example.
- network planning environment 100 is the Internet with network 101 representing a worldwide collection of networks and gateways chat use the Transmission Control Protocol/Internet Protocol (TCP/IP) suite of protocols to communicate with one another.
- TCP/IP Transmission Control Protocol/Internet Protocol
- network planning environment 100 also may be implemented as a number of different types of networks, such as for example, an intranet, a local area network (LAN) , or a wide area network (WAN) .
- Figure 1 is intended as an example, and not as an architectural limitation for different embodiments.
- Data processing system 200 is an example of a data processing system that may be used to implement servers and clients, such as mission planning system 102 and number of assets 108.
- data processing system 200 includes communications fabric 202, which provides
- processor unit 204 communicates with memory 206, persistent storage 208, communications unit 210, input/output (I/O) unit 212, and display 214.
- memory 206 persistent storage 208
- communications unit 210 communications between processor unit 204, memory 206, persistent storage 208, communications unit 210, input/output (I/O) unit 212, and display 214.
- I/O input/output
- Processor unit 204 serves to execute instructions for software that may be loaded into memory 206.
- Processor unit 204 may be a number of processors, a multi -processor core, or some other type of processor, depending on the particular
- processor unit 204 may be implemented using a number of heterogeneous processor systems in which a main processor is present with secondary processors on a single chip. As another illustrative example, processor unit 204 may be a symmetric tnulti -processor system containing multiple processors of the same type.
- Memory 206 and persistent storage 208 are examples of storage devices 216.
- a storage device is any piece of hardware that is capable of storing information, such as, for example without limitation, data, program code in functional form, and/or other suitable information either on a temporary basis and/or a permanent basis.
- Memory 206 in these examples, may be. for example, a random access memory or any other suitable volatile or non-volatile storage device.
- Persistent storage 208 may take various forms depending on the particular
- persistent storage 208 may contain one or more components or devices.
- persistent storage 208 may be a hard drive, a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above.
- the media used by persistent storage 208 also may be removable.
- a removable hard drive may be used for persistent storage 208.
- Communications unit 210 in these examples, provides for communications with other data processing systems or devices.
- communications unit 210 is a network
- Communications unit 210 may provide communications through the use of either or both physical and wireless communications links.
- Input/output unit 212 allows for input and output of data with other devices that may be connected to data processing system 200.
- input/output unit 212 may provide a connection for user input through a keyboard, a mouse, and/or some other suitable input device. Further, input/output unit 212 may send output to a printer.
- Display 214 provides a mechanism to display information to a user.
- Instructions for the operating system, applications and/or programs may be located in storage devices 216, which are in communication with processor unit 204 through communications fabric 202, In these illustrative examples the instruction are in a functional form on persistent storage 208.
- processor unit 204 may be loaded into memory 206 for execution by processor unit 204.
- the processes of the different embodiments may be performed by processor unit 204 using computer
- program code computer usable program code
- computer readable program code that may be read and executed by a processor in processor unit 204.
- the program code in the different embodiments may be embodied on different physical or tangible computer readable media, such as memory 206 or persistent storage 208.
- Program code 218 is located in a functional form on
- Computer readable media 220 that is selectively removable and may be loaded onto or transferred to data processing system 200 for execution by processor unit 204.
- Program code 218 and computer readable media 220 form computer program product 222 in these examples.
- computer readable media 220 may be computer readable storage media 224 or computer readable signal media 226,
- Computer readable storage media 224 may include, for example, an optical or magnetic disk that is inserted or placed into a drive or other device that is part of persistent storage 208 for transfer onto a storage device, such as a hard drive, that is part of persistent storage 208.
- Computer readable storage media 224 also may take the form of a persistent storage, such as a hard drive, a thumb drive, or a flash memory, that is connected to data processing system 200. In some instances, computer readable storage media 224 may not be removable from data processing system 200. In these
- computer readable storage media 224 is a non-transitory computer readable storage medium.
- program code 218 may be transferred to data processing system 200 using computer readable signal media 226.
- Computer readable signal media 226 may be, for example, a propagated data signal containing program code 218.
- computer readable signal media 226 may be an
- electromagnetic signal an optical signal, and/or any other suitable type of signal.
- signals may be transmitted over communications links, such as wireless communications links, optical fiber cable, coaxial cable, a wire, and/or any other suitable type of communications link.
- communications links such as wireless communications links, optical fiber cable, coaxial cable, a wire, and/or any other suitable type of communications link.
- the communications link and/or the connection may be physical or wireless in the illustrative examples.
- program code 218 may be downloaded over a network to persistent storage 208 from another device or data processing system through computer readable signal media 226 for use within data processing system 200.
- program code stored in a computer readable storage medium in a server data processing system may be downloaded over a network from the server to data processing system 200.
- the data processing system providing program code 218 may be a server computer, a client computer, or some other device capable of storing and transmitting program code 218,
- data processing system 200 The different components illustrated for data processing system 200 are not meant to provide architectural limitations to the manner in which different embodiments may be implemented.
- the different advantageous embodiments may be implemented in a data processing system including components in addition to or in place of those illustrated for data processing system 200,
- the data processing system may include organic components integrated with inorganic components and/or may be comprised entirely of organic
- a storage device may be comprised of an organic semiconductor.
- a storage device in data processing system 200 is any hardware apparatus that may store data.
- Memory 206 persistent storage 208, and computer readable media 220 are examples of storage devices in a tangible form.
- a bus system may be used to implement communications fabric 202 and may be comprised of one or more buses, such as a system bus or an input/output bus.
- the bus system may be implemented using any suitable type of architecture that provides for a transfer of data between different components or devices attached to the bus system.
- a communications unit may include one or more devices used to transmit and receive data, such as a modem or a network adapter.
- a memory may be, for example, memory 206 f or a cache such as found in an interface and memory
- controller hub that may be present in communications fabric 202.
- the phrase "at least one of”, when used with a list of items, means that different combinations of one or more of the items may be used and only one of each item in the list may be needed.
- “at least one of item A, item B, and item C” may include, for example, without
- first component when a first component is connected to a second component, the first component may be connected to the second component without any additional components.
- the first component also may be connected to the second component by one or more other components.
- one electronic device may be connected to another electronic device without any additional electronic devices between the first electronic device and the second electronic device. In some cases, another electronic device may be present between the two electronic devices connected to each other.
- the different advantageous embodiments recognize and take into account that current mission planning systems involve scripting missions offline then running the missions after they are completed offline. Many of these current systems are specific to a single mission type, such as an area search mission for example.
- the existing solutions are not practical for dynamic applications where missions need to be created and modified on the fly. These current systems are rigid and do not provide for modifying a mission once execution starts.
- the systems currently available require a user to be able to write code and script the mission manually as well. Manual scripting of missions is labor extensive and very costly. In addition, manually scripted missions require more extensive testing since they are built from scratch and the testing process increases with the mission complexity.
- the different advantageous embodiments provide a system comprising a number of computers, a graphical user interface, first program code stored on the computer, and second program code stored on the computer.
- the graphical user
- the interface is executed by a computer in the number of computers.
- the computer is configured to run the first program code to define a mission using a number of mission elements.
- the computer is configured to run the second program code to
- the different advantageous embodiments further provide a method for mission planning.
- a computer receives a user- definition of a mission.
- a mission script is generated using the user definition and a number of mission elements.
- the number of mission elements is assigned to a number of assets. Execution of the number of mission elements by the number of assets is controlled. The number of assets executing the mission is monitored.
- the different advantageous embodiments further provide a computer program product for mission planning comprising a computer recordable storage medium and program code stored on the computer recordable storage medium.
- the program code receives a user definition of a mission, generates a mission script using the user definition and a number of mission
- Mission planning environment 300 may be implemented in a network environment, such as network
- Mission planning environment 300 may be any type of environment suitable for generating, monitoring, updating, modifying, and/or approving missions, for example.
- Mission planning environment 300 includes mission planning system 302.
- Mission planning system 302 is an illustrative example of mission planning system 102 in Figure 1.
- Mission planning system 302 defines, executes, and monitors a number of missions involving a number of assets in the different advantageous Mission planning system 302 includes computer system 304.
- Computer system 304 may be implemented as a number of computers and/or data processing systems, such as data processing system 200 in Figure 2.
- Computer system 304 may include number of mission planner clients 306 and mission planner server 308.
- computer system 304 may be any type of environment suitable for generating, monitoring, updating, modifying, and/or approving missions, for example.
- Mission planning environment 300 includes mission planning system 302.
- Mission planning system 302 is an illustrative example of mission planning system 102 in Figure 1.
- Mission planning system 302 defines, executes, and monitors a number of missions
- each data processing system includes an instance of number of mission planner clients 306 and connects to mission planner server 308 at a central data processing system using a network, such as network 101 in Figure 1, for example.
- number of mission planner clients 306 and mission planner server 308 may be implemented on the same data processing system and accessed using a number of different data processing systems, for example.
- Number of mission planner clients 306 is a graphical user interface used to remotely build missions.
- Number of mission planner clients 306 may include number of devices 310 and number of modules 312.
- Number of devices 310 may include, for example, without limitation, a display, data-glove, a personal digital assistant, a laptop, mouse, trackpad, keyboard, joystick, a touchscreen, an optical interface, a visual interface, a tactile interface, video console, wireless controller, wireless three dimensional (3D) controller, and/or any other suitable device.
- a wireless 3D controller may be a remote control pointing device that includes a number of
- a wireless 3D controller may include a number of accelerometers used to directly control a number of mobile assets when given control by a mission element in the mission generated by mission planner server 308.
- number of devices 310 may be available for use with specialized mission elements generated by mission planner server 308. for example.
- Number of mission planner clients 306 communicates with mission planner server 308 through a messaging format, such as, for example, without limitation, extensible markup language - remote procedure call (XML-RPC) .
- XML-RPC extensible markup language - remote procedure call
- number of modules 312 may be written in a language such as
- Number of modules 312 may include, for example, without limitation, data and functions necessary to describe and create mission elements and graphical components needed to manipulate mission elements within the graphical user interface of number of mission planner clients 306 during creation of a mission.
- Number of users 314 is able to create mission elements using number of modules 312 and number of devices 310 of number of mission planner clients 305.
- Mission planner server 308 is a process executing on a computer, such as data processing system 200 in Figure 2, for example.
- Mission planner server 308 listens for incoming XML- RPC communication from number of mission planner clients 306 and processes the incoming communication to query and modify a current mission, for example.
- Mission planner server 308 keeps the current mission in memory, such as memory 206 in Figure 2, before and during execution of the current mission, which allows number of mission planner clients 306 to create and execute the current mission while adding new elements in real time.
- Mission planner server 308 includes number of modules 316 and mission management framework 318.
- number of modules 316 may be written in a language such as Python.
- Number of modules 316 may include, for example, without limitation, data and functions necessary to identify, manipulate, and store mission functions and parameters, as well as parse mission elements for information during execution of the mission.
- Mission management framework 318 is a system that includes a library of mission elements and a library of assets, as well as a number of processes for task, allocation.
- the library of mission elements includes a number of mission elements to be executed in a current mission.
- the number of mission elements is retrieved from mission element database 319 and stored into the library of mission elements by mission planner server 308 during mission creation, for example.
- Mission element database 319 includes a number of mission elements pre-defined by a user.
- the library of assets includes a list of all assets, such as number of assets 324, along with information about each asset.
- the library of assets may include
- Mission management framework 318 identifies the assets and tasks needed to execute a number of mission elements, and sends instructions to the number of assets identified to assign and control execution of a number of tasks and/or missions. When issues arise during execution of a mission, mission management framework 318 autonomously reconfigures the mission to achieve the mission objective using available assets, for example. In other words, the assets become an extension of mission planning system 302, where mission management framework 318 is the brain and the assets are components of the body, for example.
- Mission management framework 318 sends out instructions for the assets to execute tasks and receives back messages from the assets about the status of the tasks and/or the status of the assets. These messages from the assets are used by mission management framework 318 to reconfigure instructions dynamically during a mission execution in a bottom up and top down planning structure .
- Communications system 320 connects computer system 304 to number of assets 324. Communications system 320 receives and transmits information 322 between mission management framework 318 and number of assets 324. Information 322 may include, for example, without limitation, commands 326, programs 328, and messages 330. Communications system 320 may be a wireless communication system or a wired communication system in the different advantageous embodiments.
- number of users 314 may initiate a mission planning task using number of mission planner clients 306 on computer system 304.
- number of users 314 may identify a specific task or mission for mission planning system 302 to execute.
- Number of users 314 may be local to number of assets 324 or may be very remote from number of assets 324.
- number of assets 324 may be in a different location, country, or planet than number of users 314, such as a number of autonomous vehicles deployed on the moon and being controlled by mission management framework 318 from the earth.
- Mission planning system 302 may provide number of users 314 with the capability to build and monitor complex missions in real-time during execution of the missions by number of assets 324 regardless of the proximity, or lack thereof, of number of users 314 to number of assets 324.
- Number of mobile assets 332, number of sensors 334, number of machines 336 , and number of charging stations 338 may be examples of a number of assets that may be included in number of assets 324.
- Number of mobile assets 332 may include, for example, without limitation, number of vehicles 340.
- Number of vehicles 340 may be any type of vehicle including, without limitation, autonomous vehicles, semi -autonomous vehicles, unmanned
- a vehicle may be an automobile, a land vehicle, a marine vessel, an aircraft, a spacecraft, a wall -climbing robot, and/or any other suitable type of vehicle, for example.
- number of users 314 may use number of mission planner clients 306 to build a mission.
- Number of mission planner clients 306 communicates the mission dynamically as it is being built by number of users 314 to mission planner server 308, Mission planner server 308
- the high level mission script may- include high level commands.
- Mission management framework. 318 converts these high level commands into low level commands, and generates a mission script for specific assets in number of assets 324.
- the mission script generated by mission management framework 318 may include commands to turn on and deploy one or more specific assets associated with a mission element.
- a mission script is a number of instructions. These instructions may include commands 326 and/or programs 328 for number of assets 324, for example.
- Mission management framework 318 sends mission 342, having commands 326 and/or programs 328, to number of assets 324 using communication system 320. Number of assets 324 execute mission 342 according to commands 326 and/or programs 328, and communicate messages 330 back to mission management framework 318 during execution of the
- Messages 330 may include, for example, without limitation, status of the mission, status of a number of tasks, and/or status of number of assets 324.
- Mission management framework 318 may use the information received in messages 330 to modify mission 342, sending modified mission 344 back to number of assets 324.
- messages 330 may include information about one or more of number of vehicles 340
- Mission management framework 318 may reconfigure mission 342 to identify another vehicle that is fueled and capable of completing the incomplete task, and send modified mission 344 instructing the new vehicle to complete the incomplete task, in this illustrative example.
- FIG. 3 is not meant to imply physical or architectural limitations to the manner in which different advantageous embodiments may be implemented.
- Other components in addition and/or in place of the ones illustrated may be used. Some components may be unnecessary in some advantageous embodiments.
- the blocks are presented to illustrate some functional components. One or more of these blocks may be combined and/or divided into different blocks when implemented in different advantageous embodiments.
- mission planning system 302 may be implemented with other computer systems in addition to computer system 304.
- computer system 304 may include additional mission planner clients implemented on additional data processing systems for use by number of users 314 in simultaneous mission planning and/or planning of multiple missions, for example.
- Mission planner client 400 is an illustrative example of number of mission planner clients 306 in Figure 3.
- Mission planner client 400 includes number of devices 402, display 404, and number of modules 406.
- Number of devices 402 is an illustrative example of one implementation of number of devices 310 in Figure 3.
- Display 404 may be an example of one type of device in number of devices 402 used by number of users 314 in Figure 3 to interact with mission planner client 400.
- Number of modules 406 include mission element design module 408, element button module 410.
- mission type select module 412 is a number of software modules presented on a user interface as buttons, tabs, text boxes, pull-down menus, and the like, that correspond to a given software module.
- Mission element design module 408 contains all of the data and functions necessary to describe and graphically depict a mission element.
- the mission element may be graphically depicted as a rectangle with
- triangles to indicate lines to connect the mission element with a previous mission element and/or a next mission element, for example, using a device, such as display 404.
- Element button module 410 contains program code necessary to graphically depict a button within a screen of the graphical user interface, as well as program code to create a new mission element when the button depicted is selected, or pressed.
- Element button module 410 spawns a iv.issionTypeSelect window, linked to mission type select module 412, to determine the start and stop parameters for the mission element as well as its label before the creation of the mission element.
- Mission type select module 412 presents a dialog box to allow a user, such as number of users 314 in Figure 3, to define the start and stop types associated with the start and stop parameters of element button module 410, as well as values for the mission element.
- Mission type select module 412 is called after the user selects an elementButton, generated by element button module 410, but before a missionElement is created by element button module 410.
- options within the dialog box presented by mission type select module 412 may be grayed out to show the user what is available in the current state. For example, an element is not allowed to end on a sequence change when it does not begin on a sequence change. In another example, an element cannot begin at the end of a previous element if no elements are defined yet.
- Mission variable define module 414 presents a dialog box to allow the user to change some variables in a mission.
- the variables may be presented in two columns to conserve screen space, in one illustrative example. This dialog is presented immediately after a missioriElement is defined and before a missionElement is created.
- Popup element module 416 presents program code to the current element as static text. Popup element module 416 may be an optional selection by a user who wants to see the code that makes up a mission element. The user selects popup element module 416 to display the code of a mission element. The code is static because the user can view the code but cannot modify the code .
- Select window module 418 presents the user with a selection of a number of different options having a number of different values.
- the values are the code for the functions of each option. Some options may allow for multiple selections while other options only allow for a single selection. These options may be defined when the window is created.
- a user selects one of the
- the user will then see mission component creator 804 illustrated in Figure 8.
- the user selects one of the mission elements in mission element database 80S as illustrated in Figure 8 and selects "OK", resulting in a view of component editor 904 illustrated in Figure 9.
- the user fills out the mission element parameters, then selects the "Time” tab depicted in component editor 904 of Figure 9 and fills out the Start and Stop Conditions, then selects "OK.” This process is repeated for every mission element and every function that the user wants to add to the mission.
- the "save" button depicted as a floppy disk, at the top of the graphical user interface of Figure 8 and Figure 9.
- mission planner client 400 in Figure 4 is not meant to imply physical or architectural limitations to the manner in which different advantageous embodiments may be implemented.
- Other components in addition and/or in place of the ones illustrated may be used. Some components may be unnecessary in some advantageous embodiments.
- the blocks are presented to illustrate some functional components. One or more of these blocks may be combined and/or divided into
- Mission planner server 500 is an illustrative example of mission planner server 308 in Figure 3.
- Mission planner server 500 includes number of modules 502 and mission management framework 504.
- Number of modules 502 is an illustrative example of one implementation of number of modules 316 in Figure 3.
- Number of modules 502 includes mission function module 506, mission element module 508, read library module 510, write mission module 512, and main module 514.
- Mission function module 506 contains data necessary to identify unique mission functions.
- Mission function module 506 includes data such as, without limitation, the name,
- the code may be written in Python, for example.
- the parameters may be any variables in the mission that are capable of being changed by a user, such as number of users 314 in Figure 3, for example.
- a parameter may be area coordinates or a number of iterations.
- a user may change area coordinates for a mission function, for example.
- the number of assets required and time to execute refers to the number of assets required for and the time to execute the mission elements.
- Mission element, module 508 contains information necessary for a mission element registered in the mission.
- Mission element module 508 includes information such as, without
- getCode () may be a function to return the Python code from the mission element, which replaces all instances of parameters with their defined value.
- Read library module 510 contains functions to read mission functions from the file system as well as parse the mission functions for information.
- vehJ'ypeParams may be a data structure to hold parameters for the type of vehicle assets required in the mission. This data structure may contain the type, number, wait flag and release flag values, for example.
- readLibrary (dir) is a function that reads all files in a directory (dir) and returns the contents of the files as a dictionary with the key set as the file name without the extension, removing w .py" for example in Python code, and the contents of the file as its value. This function may assume that all files with " ⁇ " in their name are temporary files and should not be read, in this illustrative example .
- parseFunc is a function that takes the code for a mission element and parses it for several parameters. This function may assume that the first string surrounded by three quotation marks ( "" " ) is the
- the function checks the Requires decorator to gather information about assets required for the mission element, for example.
- the function may also find user defined variables, denoted by a commented line starting with "#" followed by a variable surrounded by "$,” for example. If immediately follows the variable, without whitespace, it is stored as that variable's default value. If a string follows the variable or default value, with whitespace in between, then that string is assumed to be the description of that variable, in this illustrative example.
- Write mission module 512 writes the mission to a python file readable by mission management framework. 504.
- Write mission module 512 may use the following set of functions:
- writeMission (dir, name, sta.rt.Func, stopFunc, onlterations, supportFuncs, missionElements) .
- This set of functions takes in several parameters in order to write the mission file.
- dir is the directory to place the file
- name is the name of the mission.
- the name of the mission will be
- startFunc is the function that should follow the decorator OnStart to be
- stopFunc is the function that should follow the decorator GnStop. onlterations is the list of functions that require the decorator OnIterat.ion .
- supportFuncs is a list of functions that must be printed at the beginning of the mission for use in later elements.
- missionElements is a list of missionElement objects to be written by write mission module 512.
- missionElements is the function that handles all the decorator additions necessary for the mission, for example.
- Main module 514 runs the extensible markup language remote procedure call (XML-RPC) server and stores all of the mission functions. Data on the XML-RPC server is broken into possible functions and active functions. Possible functions are all of the available functions in the library, as read by read library module 510. Active functions are all of the functions that have been defined.
- XML-RPC extensible markup language remote procedure call
- Each possible component dictionary is stored in the possibleComponents dictionary using its component name as the key and the dictionary itself as the value.
- Start and stop functions are defined as the name of the function given by the user. Iteration and support
- Mission elements are defined as a dictionary using the label of the element as the key and a missionElement object as the value.
- Main module 514 also stores, on the XML-RPC server,
- the current mission time may be stored in seconds, in one illustrative example .
- Mission management framework 504 includes mission elements library 516, asset library 518. resource identification process 520, task identification process 522, and mission assignment process 524.
- Mission elements library 516 includes number of mission elements 526.
- Number of mission elements 526 is stored mission elements built by number of modules 502 to be executed in a current mission.
- Number of mission elements 526 may be retrieved from mission element database 534 by number of modules 502 during mission creation, for example.
- Mission element database 534 is an illustrative example of one implementation of mission element database 319 in Figure 3.
- Asset library 518 includes number of assets 528.
- Number of assets 528 is a list of all assets, such as number of assets 324 in Figure 3, along with a current status of each asset.
- number of assets 528 may include information about the availability of an asset, the readiness of an asset for use in a mission and/or task, the health of an asset, the resource use of an asset, the resource depletion by an asset, the current status of an asset, and/or any other suitable information associated with an asset.
- Task identification process 522 receives a mission script from write mission module 512 and uses mission elements library 516 to identify the mission elements, or tasks, needed for the mission.
- Resource identification process 520 receives a mission script from write mission module 512 and uses mission elements library 516 to identify a number of assets from number of assets 528 in asset, library 518 that can accomplish the mission
- Mission assignment process 524 receives the mission elements and assets identified by- resource identification process 520 and task identification process 522, and assign the tasks identified to specific assets identified, generating instructions 530. Instructions 530 are then transmitted by mission management framework 504 to number of assets 532 for mission execution.
- mission planner server 500 in Figure 5 is not meant to imply physical or architectural limitations to the manner in which different advantageous embodiments may be implemented.
- Other components in addition and/or in place of the ones illustrated may be used. Some components may be unnecessary in some advantageous embodiments.
- the blocks are presented to illustrate some functional components. One or more of these blocks may be combined and/or divided into
- Graphical user interface 600 is an illustrative example of one implementation of mission planner client 305 in Figure 3 using number of devices 312 in Figure 3.
- Graphical user interface 600 includes execution control buttons 602 and mission elements 604, Execution control buttons 602 may be selected by a user, such as number of users 314 in Figure 3, to start a mission, stop a mission, advance a mission sequence, and any other number of control features for a
- Mission elements 604 are an illustrative example of mission elements selected from number of mission elements 526 in Figure 5 for inclusion into a mission, for example.
- One or more start functions 606 are selected from a library of available start functions.
- a start function is a function that runs at the start of the mission, for example as an initialization function for the mission.
- graphical user interface 600 depicts start function 606 called
- stop functions 608 are selected from a library of available stop functions.
- a stop function is a function that runs upon completion of the mission.
- a stop function may be, for example, 'without limitation, to clear all tasks and reset the number of assets.
- graphical user interface 600 depicts stop function 608 called stopMission that has been added by the user to this mission.
- One or more stop conditions 610 are selected from a library of available stop conditions.
- a stop condition is a function that defines a stop condition of the mission.
- a stop condition may, for example, without limitation, define a time value at which the mission stops regardless of whether the individual mission elements completed execution or not.
- graphical user interface 600 depicts stop condition 610 called seqAdvanced that has been added by the user to this mission.
- One or more iteration functions 612 are selected from a library of available iteration functions.
- An iteration function is a function that runs on every iteration of the mission management framework. For example, an iteration function may run in parallel with the mission elements and display a report about available and deployed assets.
- graphical user interface 600 depicts iteration function 612 called defaultlteration that has been added by the user to this mission .
- One or more support functions 614 are selected from a library of available support functions.
- a support function is a function that is called by one or more mission elements.
- a support function may be an algorithm to partition a search area that can be used by different mission elements that perform search related tasks.
- graphical user interface 600 depicts support functions 614
- Mission elements 604 are represented by bars the length of which is proportional to the mission element duration as measured by time scale 616, similar to a Gantt chart .
- graphical user interface 600 in Figure 6 is not meant to imply physical or architectural limitations to the manner in which different advantageous embodiments may be implemented. Other components in addition and/or in place of the ones illustrated may be used. Some components may be unnecessary in some advantageous embodiments. Also, the blocks are presented to illustrate some functional components. One or more of these blocks may be combined and/or divided into
- Graphical user interface 700 is an illustrative example of one implementation of mission planner client 306 in Figure 3 using number of devices 312 in Figure 3.
- Graphical user interface 700 includes mission elements database 702 and library of mission elements 704.
- Mission elements database 702 is an illustrative example of one
- mission elements database 702 includes a number of mission elements, such as inspectTarget 706, observeStructure 708, mixedCoverage 710, monitorAdvance
- Library of mission elements 704 is an illustrative example of mission elements that a user selected from mission elements database 702 and added to the mission using graphical user interface 700. In this illustrative example, the user added five mission elements of initialSurvey 714, inspectTarget 706, jointSearch 718, monitorAdvance 712, mixedCoverage 710. Library of mission elements 704 are represented by bars in this
- the length of which is proportional to the mission element duration as measured by time scale 720 and/or sequence number 722.
- This proportional measurement may be similar to a Gantt chart, for example.
- a user such as number of users 314 in Figure 3, may enter values in sequence times estimates 724 to determine the estimated times at which the mission sequence is advanced in order to represent those mission elements that begin or end on sequence change on a time scale, for example.
- the user selects update 726 to apply the values entered into sequence times estimates 724 to the mission elements display.
- the mission sequence starts at zero and changes to one at sixty seconds.
- Graphical user interface 800 is an illustrative example of one implementation of mission planner client 306 in Figure 3 using number of devices 312 in Figure 3.
- Graphical user interface 800 includes canvas 802 and mission component creator 804.
- Canvas 802 may be an
- Canvas 802 is a
- graphical situational display of the mission environment where a user can drag a cursor using a device, such as a mouse, to select coordinates parameters of mission elements.
- Mission component creator 804 is a dialog box that presents a user, such as number of users 314 in Figure 3, with mission element database 80S and associated code of selected mission element 808.
- database 806 the code for that mission element will be displayed in the dialog box to the right of the listing of mission elements.
- Mission component creator 804 is an example of select window module 418.
- the different, options in select 'window module 418 are the mission elements in mission element database 806. and the different values of select window module 418 are code of selected mission element 808 for each mission element. Code of selected mission element 808 is displayed by popup element module 416.
- Graphical user interface 900 is an illustrative example of one implementation of mission planner client 306 in Figure 3 using number of devices 312 in Figure 3.
- Graphical user interface 900 includes mission planner 902 and component editor 904.
- Mission planner 902 is an
- Component editor 904 may be an illustrative example of a dialog box presented by mission variable define module 414 in Figure 4, for example, allowing a user to change variables in a mission.
- a user can insert multiple instances of the same mission element, for example running at different times or locations, and get unique names.
- the user entered initialSurveyl as Unique Name of this instance of initialSurvey mission element.
- FIG. 10 depicted in accordance with an advantageous embodiment.
- the process in Figure 10 may be implemented by a component such as mission planning system 302 in Figure 3, for example.
- the process begins by receiving a user definition of a mission (operation 1002 ⁇ .
- the user definition may be received using a graphical user interface, such as mission planner client 306 in Figure 3, for example.
- the user definition may generally define a mission, or number of tasks, that needs to be
- the process generates a mission script using the user definition and a number of mission elements (operation 1004) .
- the process may generate a mission script using a mission management framework, such as mission management framework 318 in Figure 3, for example.
- the number of mission elements may define aspects of a mission, or specific tasks in a number of tasks, for examp1e .
- the process assigns the number of mission elements to a number of assets (operation 1006) .
- the assets may be, for example, number of assets 324 in Figure 3.
- the process controls execution of the number of mission elements by the number of assets (operation 1008) .
- the process may generate instructions for the number of assets to execute the mission using the number of mission elements.
- the instructions may be transmitted to the number of assets using a communication system, such as
- the execution control actions may include optional advancement of a mission sequence by the process to skip a mission element, for example, given feedback or messages received from the number of assets.
- the sequence advancement of a mission may be controlled by a user, such as number of users 314 in Figure 3, whereby the user gives this command to the mission management framework and not directly to the number of assets.
- the process monitors the number of assets executing the mission (operation 1010 ⁇ .
- the process receives messages from the number of assets during execution of the mission (operation 1012) .
- the process determines whether a mission reconfiguration is needed (operation 1014) .
- a mission reconfiguration may be needed if an asset is unable to complete a mission element, or needs assistance completing a mission element, for example.
- the process If a determination is made that a mission reconfiguration is needed, the process generates a modified mission using the messages received (operation 1016) . and returns to operation 1010.
- This reconfiguration is performed autonomously by the mission management framework based on the messages received, such as vehicle failures or other changes in asset status, for example.
- a user may also modify the mission by adding a number of mission elements to the mission, removing a number of mission elements that have not yet started from the mission, modifying the parameters of a number of mission
- the process determines whether the mission is complete (operation 1018) . If a determination is made that the mission is not complete, the process returns to operation 1010. If a determination is made that the mission is complete, the process terminates thereafter.
- FIG 11 a flowchart illustrating a process for defining a mission is depicted in accordance with an advantageous embodiment.
- the process in Figure 11 may be implemented by a component such as mission planning system 302 in Figure 3, for example.
- the process begins by adding a number of mission elements to a library of mission elements (operation 1102) .
- the number of mission elements may be selected from a mission elements database, such as mission elements database 319 in Figure 3, by a mission planner server, and saved into a mission elements library, such as mission elements library 516 in Figure 5, for example.
- the process adds a number of start functions to the mission (operation 1104) .
- the start functions may be added from a library of start functions, such as start functions 606 in
- the process then adds a number of stop functions to the mission (operation 1106) .
- the stop functions may be added from a library of stop functions, such as stop functions 608 in Figure 6,
- the process optionally adds a number of stop
- the stop conditions may be added from a library of stop conditions, such as stop conditions 610 in Figure 6.
- the process adds a number of
- the iteration functions may be added from a library of iteration functions, such as iteration functions 612 in Figure ⁇ .
- the process optionally adds a number of support functions to the mission (operation 1112) .
- the support functions may be added from a library of support functions, such as support functions 614 in Figure 6.
- the process optionally save the mission in a storage device (operation 1114) , with the process terminating thereafter.
- the process may execute the mission from the computer memory without saving it, for example.
- each block in the flowchart or block diagrams may represent a module, segment, or portion of computer usable or readable program code, which comprises one or more executable instructions for implementing the specified function or functions.
- the function or functions noted in the block may occur out of the order noted in the figures. For example, in some cases, two blocks shown in succession may be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved .
- the different advantageous embodiments can take the form of an entirely hardware embodiment, an entirely software
- the different embodiments can take the form of a computer program product accessible from a computer usable or computer readable medium providing program code for use by or in connection with a computer or any device or system that executes instructions.
- a computer usable or computer readable medium can generally be any tangible apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
- the computer usable or computer readable medium can be, for example, without limitation an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or a
- readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM) .
- RAM random access memory
- ROM read-only memory
- Optical disks may include compact disk - read only memory (CD-ROM) , compact disk - read/write (CD-R/W) and DVD .
- a computer usable or computer readable medium may contain or score a computer readable or usable program code such that when the computer readable or usable program code is executed on a computer, the execution of this computer readable or usable program code causes the computer to transmit another computer readable or usable program code over a communications link.
- This communications link may use a medium that is, for example without limitation, physical or wireless,
- a data processing system suitable for storing and/or executing computer readable or computer usable program code will include one or more processors coupled directly or indirectly to memory elements through a communications fabric, such as a system bus.
- the memory elements may include local memory
- bulk storage employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some computer readable or computer usable program code to reduce the number of times code may be retrieved from bulk storage during execution of the code.
- I/O devices can be coupled to the system either directly or through intervening I/O controllers. These devices may include, for example, without limitation to
- keyboards touch screen displays, and pointing devices.
- Different communications adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks .
- Non- limiting examples are modems and network adapters are just a few of the currently available types of communications adapters.
- the different advantageous embodiments recognize and take into account that current mission planning systems involve scripting missions offline then running the missions after they are completed offline. Many of these current systems are specific to a single mission type, such as an area search mission for example. The existing solutions are not practical for dynamic applications where missions need to be created and modified on the fly. These current systems are rigid and do not provide for modifying a mission once execution starts. The systems currently available require a user to be able to write code and script the mission manually as well.
- the different advantageous embodiments provide a system for real-time definition, execution, and monitoring of complex missions involving a multitude of assets.
- This system allows operators of autonomous vehicles to easily create, update, and execute complex missions involving multiple casks that need to be executed by varying numbers and types of assets, using a graphical user interface.
- the user can define, update, and execute missions from a remote location.
- multiple clients can be used to allow multiple users to simultaneously monitor and update missions.
- Mission definition is modular using the concept of "mission elements”, and thus a user can monitor mission progress by graphically displaying the mission elements in a real-time Gantt chart format chat shows the dependencies between mission elements, their completion status, and the resources assigned or required to each mission element. This is achieved using feedback from the mission management framework via the mission planner server.
- XML-RPC may be used for rich messaging between clients and server including querying the server library of mission elements, thus allowing easy updates of the library without any software change in the clients .
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Tourism & Hospitality (AREA)
- Operations Research (AREA)
- Marketing (AREA)
- Quality & Reliability (AREA)
- Game Theory and Decision Science (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Data Mining & Analysis (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- User Interface Of Digital Computer (AREA)
- Stored Programmes (AREA)
- Debugging And Monitoring (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- General Factory Administration (AREA)
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2013510105A JP5924740B2 (en) | 2010-05-14 | 2011-04-14 | Real-time mission planning |
EP11722637.3A EP2569697A4 (en) | 2010-05-14 | 2011-04-14 | Real time mission planning |
AU2011253405A AU2011253405B2 (en) | 2010-05-14 | 2011-04-14 | Real time mission planning |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/780,621 US9064222B2 (en) | 2010-05-14 | 2010-05-14 | Real time mission planning |
US12/780,621 | 2010-05-14 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2011142933A2 true WO2011142933A2 (en) | 2011-11-17 |
WO2011142933A3 WO2011142933A3 (en) | 2013-03-14 |
Family
ID=44120986
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2011/032459 WO2011142933A2 (en) | 2010-05-14 | 2011-04-14 | Real time mission planning |
Country Status (5)
Country | Link |
---|---|
US (1) | US9064222B2 (en) |
EP (1) | EP2569697A4 (en) |
JP (1) | JP5924740B2 (en) |
AU (1) | AU2011253405B2 (en) |
WO (1) | WO2011142933A2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015517142A (en) * | 2012-03-22 | 2015-06-18 | イスラエル エアロスペース インダストリーズ リミテッド | Autonomous mission planning and monitoring |
WO2021137910A3 (en) * | 2019-10-03 | 2021-08-12 | Raytheon Company | Computer architecture for resource allocation for course of action activities |
US11205142B2 (en) * | 2017-06-29 | 2021-12-21 | The Boeing Company | Work order generation system |
CN114676471A (en) * | 2022-04-21 | 2022-06-28 | 北京航天飞行控制中心 | Method and device for establishing mission planning model of mars vehicle, electronic equipment and medium |
Families Citing this family (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101157484B1 (en) * | 2010-12-14 | 2012-06-20 | 주식회사 대한항공 | Uav automatic recovering method |
US9442946B1 (en) * | 2013-02-27 | 2016-09-13 | The Boeing Company | Methods and systems of processing contextual information |
US11334069B1 (en) | 2013-04-22 | 2022-05-17 | National Technology & Engineering Solutions Of Sandia, Llc | Systems, methods and computer program products for collaborative agent control |
US10168674B1 (en) * | 2013-04-22 | 2019-01-01 | National Technology & Engineering Solutions Of Sandia, Llc | System and method for operator control of heterogeneous unmanned system teams |
EP2800032A1 (en) * | 2013-05-02 | 2014-11-05 | BAE Systems PLC | Goal-based planning system |
US9567080B2 (en) | 2013-05-02 | 2017-02-14 | Bae Systems Plc | Goal-based planning system |
US20150025927A1 (en) * | 2013-07-16 | 2015-01-22 | Raytheon Company | Mission component evaluation and response architecture |
US9629220B2 (en) * | 2013-08-05 | 2017-04-18 | Peter Panopoulos | Sensor-based controllable LED lighting system with repositionable components and method |
EP2879012A1 (en) * | 2013-11-29 | 2015-06-03 | The Boeing Company | System and method for commanding a payload of an aircraft |
US9423943B2 (en) | 2014-03-07 | 2016-08-23 | Oracle International Corporation | Automatic variable zooming system for a project plan timeline |
US9710571B2 (en) | 2014-03-07 | 2017-07-18 | Oracle International Corporation | Graphical top-down planning system |
US9418348B2 (en) | 2014-05-05 | 2016-08-16 | Oracle International Corporation | Automatic task assignment system |
US10643157B2 (en) | 2015-02-03 | 2020-05-05 | Oracle International Corporation | Task progress update history visualization system |
US10496943B2 (en) | 2015-03-30 | 2019-12-03 | Oracle International Corporation | Visual task assignment system |
US9776717B2 (en) | 2015-10-02 | 2017-10-03 | The Boeing Company | Aerial agricultural management system |
US10319243B2 (en) | 2015-11-30 | 2019-06-11 | At&T Intellectual Property I, L.P. | Computer aided dispatch of drones |
US10296862B1 (en) * | 2016-05-12 | 2019-05-21 | Northrop Grumman Systems Corporation | Time interval assessment of mission plan quality dynamic asset allocation |
US10520953B2 (en) | 2016-09-09 | 2019-12-31 | Walmart Apollo, Llc | Geographic area monitoring systems and methods that balance power usage between multiple unmanned vehicles |
US10423169B2 (en) * | 2016-09-09 | 2019-09-24 | Walmart Apollo, Llc | Geographic area monitoring systems and methods utilizing computational sharing across multiple unmanned vehicles |
CN109996732A (en) | 2016-09-09 | 2019-07-09 | 沃尔玛阿波罗有限责任公司 | The system and method that interchangeably tool system is coupled with unmanned vehicle |
CA3035771A1 (en) | 2016-09-09 | 2018-03-15 | Walmart Apollo, Llc | Geographic area monitoring systems and methods through interchanging tool systems between unmanned vehicles |
US10540898B2 (en) * | 2017-07-21 | 2020-01-21 | General Electric Company | Decision support system for air missioncommander dynamic mission re-planning |
US10872533B1 (en) | 2017-09-29 | 2020-12-22 | DroneUp, LLC | Multiplexed communications of telemetry data, video stream data and voice data among piloted aerial drones via a common software application |
US10769466B2 (en) * | 2018-02-20 | 2020-09-08 | International Business Machines Corporation | Precision aware drone-based object mapping based on spatial pattern recognition |
US11720849B2 (en) | 2018-11-15 | 2023-08-08 | Corverity Corporation | Method and system for managing navigational data for autonomous vehicles |
US11093715B2 (en) | 2019-03-29 | 2021-08-17 | Samsung Electronics Co., Ltd. | Method and system for learning and enabling commands via user demonstration |
US11468881B2 (en) * | 2019-03-29 | 2022-10-11 | Samsung Electronics Co., Ltd. | Method and system for semantic intelligent task learning and adaptive execution |
US11119485B1 (en) * | 2020-10-07 | 2021-09-14 | Accenture Global Solutions Limited | Drone operational advisory engine |
Family Cites Families (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH06344279A (en) | 1993-06-07 | 1994-12-20 | Hitachi Ltd | Remote operation device and method |
US5642467A (en) * | 1995-01-31 | 1997-06-24 | The Penn State Research Foundation | Controller for autonomous device |
IL117792A (en) * | 1995-05-08 | 2003-10-31 | Rafael Armament Dev Authority | Autonomous command and control unit for mobile platform |
US6222547B1 (en) * | 1997-02-07 | 2001-04-24 | California Institute Of Technology | Monitoring and analysis of data in cyberspace |
US5951609A (en) * | 1997-05-29 | 1999-09-14 | Trw Inc. | Method and system for autonomous spacecraft control |
US6317690B1 (en) * | 1999-06-28 | 2001-11-13 | Min-Chung Gia | Path planning, terrain avoidance and situation awareness system for general aviation |
JP2001353678A (en) | 2000-06-12 | 2001-12-25 | Sony Corp | Authoring system and method and storage medium |
US6493609B2 (en) * | 2001-04-27 | 2002-12-10 | Lockheed Martin Corporation | Automatic flight envelope protection for uninhabited air vehicles |
US20030059743A1 (en) * | 2001-08-29 | 2003-03-27 | The Boeing Company | Method and apparatus for automatically generating a terrain model for display during flight simulation |
US7154496B1 (en) * | 2001-10-30 | 2006-12-26 | Lockheed Martin Corporation | Telemetry-based flight vehicle visualization system and method |
US7243008B2 (en) * | 2002-06-11 | 2007-07-10 | Lockheed Martin | Automated intel data radio |
US7343232B2 (en) * | 2003-06-20 | 2008-03-11 | Geneva Aerospace | Vehicle control system including related methods and components |
JP4676784B2 (en) * | 2004-03-02 | 2011-04-27 | 株式会社リコー | Process management apparatus, process management method, and process management program |
US20050216182A1 (en) * | 2004-03-24 | 2005-09-29 | Hussain Talib S | Vehicle routing and path planning |
US8942882B2 (en) * | 2004-07-02 | 2015-01-27 | The Boeing Company | Vehicle health management systems and methods |
JP4460411B2 (en) | 2004-10-01 | 2010-05-12 | 本田技研工業株式会社 | Robot controller |
US7765038B2 (en) | 2005-02-16 | 2010-07-27 | Lockheed Martin Corporation | Mission planning system for vehicles with varying levels of autonomy |
US8078319B2 (en) * | 2005-02-16 | 2011-12-13 | Lockheed Martin Corporation | Hierarchical contingency management system for mission planners |
US7236861B2 (en) * | 2005-02-16 | 2007-06-26 | Lockheed Martin Corporation | Mission planning system with asynchronous request capability |
US7624024B2 (en) | 2005-04-18 | 2009-11-24 | United Parcel Service Of America, Inc. | Systems and methods for dynamically updating a dispatch plan |
US20060271286A1 (en) * | 2005-05-27 | 2006-11-30 | Outland Research, Llc | Image-enhanced vehicle navigation systems and methods |
US7415331B2 (en) * | 2005-07-25 | 2008-08-19 | Lockheed Martin Corporation | System for controlling unmanned vehicles |
US7451023B2 (en) * | 2005-07-25 | 2008-11-11 | Lockheed Martin Corporation | Collaborative system for a team of unmanned vehicles |
US7734386B2 (en) * | 2005-07-25 | 2010-06-08 | Lockheed Martin Corporation | System for intelligently controlling a team of vehicles |
US7844396B2 (en) * | 2005-09-13 | 2010-11-30 | Deere & Company | Method and system for modular data processing for a vehicle control system |
JP2007093045A (en) | 2005-09-27 | 2007-04-12 | Mitsubishi Electric Corp | Command device and command method |
US7813888B2 (en) | 2006-07-24 | 2010-10-12 | The Boeing Company | Autonomous vehicle rapid development testbed systems and methods |
US7894952B2 (en) * | 2006-11-21 | 2011-02-22 | Lockheed Martin Corporation | Methods and apparatus for accessing vehicle electronic systems |
US8620714B2 (en) * | 2006-11-28 | 2013-12-31 | The Boeing Company | Prognostic condition assessment decision aid |
WO2009045580A1 (en) * | 2007-06-15 | 2009-04-09 | Mountaintop Technologies, Inc. | Aviation ground navigation system |
ITRM20070347A1 (en) * | 2007-06-21 | 2008-12-22 | Space Software Italia S P A | METHOD AND SYSTEM FOR THE INTERACTION AND COOPERATION OF SENSORS, ACTUATORS AND ROBOTS |
US8019447B2 (en) * | 2007-09-14 | 2011-09-13 | The Boeing Company | Method and system to control operation of a device using an integrated simulation with a time shift option |
US8713177B2 (en) * | 2008-05-30 | 2014-04-29 | Red Hat, Inc. | Remote management of networked systems using secure modular platform |
WO2010039306A2 (en) * | 2008-06-27 | 2010-04-08 | Raytheon Company | Apparatus and method of controlling an unmanned vehicle |
US8521339B2 (en) * | 2008-09-09 | 2013-08-27 | Aeryon Labs Inc. | Method and system for directing unmanned vehicles |
US8543265B2 (en) * | 2008-10-20 | 2013-09-24 | Honeywell International Inc. | Systems and methods for unmanned aerial vehicle navigation |
US8538673B2 (en) * | 2008-10-31 | 2013-09-17 | Czech Technical University In Prague | System and method for planning/replanning collision free flight plans in real or accelerated time |
IL196276A (en) | 2008-12-30 | 2013-04-30 | Elbit Systems Ltd | Autonomous navigation system and method for a maneuverable platform |
US9127908B2 (en) * | 2009-02-02 | 2015-09-08 | Aero Vironment, Inc. | Multimode unmanned aerial vehicle |
US9046892B2 (en) * | 2009-06-05 | 2015-06-02 | The Boeing Company | Supervision and control of heterogeneous autonomous operations |
US8515609B2 (en) * | 2009-07-06 | 2013-08-20 | Honeywell International Inc. | Flight technical control management for an unmanned aerial vehicle |
US10810519B2 (en) * | 2009-10-05 | 2020-10-20 | The Boeing Company | Hierarchical mission management |
US20110130916A1 (en) * | 2009-12-01 | 2011-06-02 | Ise Corporation | Location Based Vehicle Data Logging and Diagnostic System and Method |
EP2733560A1 (en) * | 2012-11-19 | 2014-05-21 | The Boeing Company | Autonomous mission management |
-
2010
- 2010-05-14 US US12/780,621 patent/US9064222B2/en active Active
-
2011
- 2011-04-14 AU AU2011253405A patent/AU2011253405B2/en active Active
- 2011-04-14 JP JP2013510105A patent/JP5924740B2/en active Active
- 2011-04-14 WO PCT/US2011/032459 patent/WO2011142933A2/en active Application Filing
- 2011-04-14 EP EP11722637.3A patent/EP2569697A4/en not_active Ceased
Non-Patent Citations (1)
Title |
---|
See references of EP2569697A4 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2015517142A (en) * | 2012-03-22 | 2015-06-18 | イスラエル エアロスペース インダストリーズ リミテッド | Autonomous mission planning and monitoring |
US11205142B2 (en) * | 2017-06-29 | 2021-12-21 | The Boeing Company | Work order generation system |
WO2021137910A3 (en) * | 2019-10-03 | 2021-08-12 | Raytheon Company | Computer architecture for resource allocation for course of action activities |
CN114676471A (en) * | 2022-04-21 | 2022-06-28 | 北京航天飞行控制中心 | Method and device for establishing mission planning model of mars vehicle, electronic equipment and medium |
CN114676471B (en) * | 2022-04-21 | 2022-09-13 | 北京航天飞行控制中心 | Method and device for establishing mission planning model of mars vehicle, electronic equipment and medium |
Also Published As
Publication number | Publication date |
---|---|
AU2011253405B2 (en) | 2017-02-02 |
AU2011253405A1 (en) | 2012-11-01 |
JP2013527529A (en) | 2013-06-27 |
EP2569697A4 (en) | 2015-03-25 |
US9064222B2 (en) | 2015-06-23 |
JP5924740B2 (en) | 2016-05-25 |
WO2011142933A3 (en) | 2013-03-14 |
EP2569697A2 (en) | 2013-03-20 |
US20110283285A1 (en) | 2011-11-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9064222B2 (en) | Real time mission planning | |
CN104636643B (en) | Control method and device are presented in system interface content | |
CA3017121C (en) | Systems and methods for dynamic prediction of workflows | |
CN114556305A (en) | Artificial intelligence based process identification, extraction and automation for robotic process automation | |
US20180081642A1 (en) | Connectors framework | |
EP3832462A1 (en) | Dynamic orchestration in a multi-container architecture | |
US20180032913A1 (en) | Machine learning data analysis system and method | |
US20210117299A1 (en) | Data agnostic monitoring service | |
US9087353B2 (en) | Personalized demo environment based on software configuration information | |
US10732804B2 (en) | Device-to-device communication in an industrial automation environment | |
EP3098767A1 (en) | Data collection for assistance in an industrial automation environment | |
US20100145490A1 (en) | Three-dimensional (3d) manufacturing process planning | |
US8935667B2 (en) | Synchronization of prospect information between software providers and resale partners | |
US9542171B2 (en) | Managing an application modification process | |
Iglesias et al. | Model-based personalized visualization system for monitoring evolving industrial cyber-physical system | |
CN115525796A (en) | View generation method and device, electronic equipment and storage medium | |
US20130138690A1 (en) | Automatically identifying reused model artifacts in business process models | |
US11009848B2 (en) | Scalable intelligent query-based interface to navigate and find industrial information on a visual interface | |
Pribeanu | An approach to task modeling for user interface design | |
CN116909582B (en) | Visual large-screen service deployment method and device and electronic equipment | |
CN115033282B (en) | Charging pile operation platform compatible method, device, equipment and storage medium | |
US20230102809A1 (en) | Preconfigured robots for robotic process automation | |
US20230101948A1 (en) | Generation of rpa platform design components for configuring rpa platforms | |
US20220176552A1 (en) | Guided operation by robotic processes | |
CN115981719A (en) | Configuration method, control method, storage medium and electronic device of intelligent system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2011253405 Country of ref document: AU Date of ref document: 20110414 Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2013510105 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REEP | Request for entry into the european phase |
Ref document number: 2011722637 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011722637 Country of ref document: EP |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11722637 Country of ref document: EP Kind code of ref document: A2 |