WO2017165481A1 - Aircrew automation system and method - Google Patents
Aircrew automation system and method Download PDFInfo
- Publication number
- WO2017165481A1 WO2017165481A1 PCT/US2017/023523 US2017023523W WO2017165481A1 WO 2017165481 A1 WO2017165481 A1 WO 2017165481A1 US 2017023523 W US2017023523 W US 2017023523W WO 2017165481 A1 WO2017165481 A1 WO 2017165481A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- aircrew automation
- automation system
- aircraft
- flight
- aircrew
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 230000008447 perception Effects 0.000 claims description 62
- 238000012544 monitoring process Methods 0.000 claims description 40
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 claims description 28
- 238000004891 communication Methods 0.000 claims description 22
- 230000004438 eyesight Effects 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 11
- 230000004044 response Effects 0.000 claims description 11
- 230000001755 vocal effect Effects 0.000 claims description 11
- 230000036541 health Effects 0.000 claims description 9
- 238000009434 installation Methods 0.000 claims description 6
- 230000003416 augmentation Effects 0.000 claims description 5
- 230000004297 night vision Effects 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 238000004393 prognosis Methods 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 abstract description 2
- 230000008569 process Effects 0.000 description 11
- 230000009471 action Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 9
- 239000000446 fuel Substances 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 239000011521 glass Substances 0.000 description 7
- 238000010801 machine learning Methods 0.000 description 7
- 238000013459 approach Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 238000012549 training Methods 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 230000008901 benefit Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000010354 integration Effects 0.000 description 4
- 239000011295 pitch Substances 0.000 description 4
- 238000012552 review Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000010348 incorporation Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000002441 reversible effect Effects 0.000 description 3
- 238000005094 computer simulation Methods 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 238000013481 data capture Methods 0.000 description 2
- 238000013480 data collection Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- HRANPRDGABOKNQ-ORGXEYTDSA-N (1r,3r,3as,3br,7ar,8as,8bs,8cs,10as)-1-acetyl-5-chloro-3-hydroxy-8b,10a-dimethyl-7-oxo-1,2,3,3a,3b,7,7a,8,8a,8b,8c,9,10,10a-tetradecahydrocyclopenta[a]cyclopropa[g]phenanthren-1-yl acetate Chemical compound C1=C(Cl)C2=CC(=O)[C@@H]3C[C@@H]3[C@]2(C)[C@@H]2[C@@H]1[C@@H]1[C@H](O)C[C@@](C(C)=O)(OC(=O)C)[C@@]1(C)CC2 HRANPRDGABOKNQ-ORGXEYTDSA-N 0.000 description 1
- 206010048909 Boredom Diseases 0.000 description 1
- 241000272470 Circus Species 0.000 description 1
- 241000282412 Homo Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000002730 additional effect Effects 0.000 description 1
- 230000002547 anomalous effect Effects 0.000 description 1
- 230000009118 appropriate response Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 229910003460 diamond Inorganic materials 0.000 description 1
- 239000010432 diamond Substances 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000010006 flight Effects 0.000 description 1
- 238000005755 formation reaction Methods 0.000 description 1
- 230000008014 freezing Effects 0.000 description 1
- 238000007710 freezing Methods 0.000 description 1
- 230000004313 glare Effects 0.000 description 1
- 239000000383 hazardous chemical Substances 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 231100000206 health hazard Toxicity 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 229910001092 metal group alloy Inorganic materials 0.000 description 1
- 150000002739 metals Chemical class 0.000 description 1
- 230000000116 mitigating effect Effects 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 230000000414 obstructive effect Effects 0.000 description 1
- 230000010355 oscillation Effects 0.000 description 1
- 230000003534 oscillatory effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 238000000275 quality assurance Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000029305 taxis Effects 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
- G05D1/0088—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64C—AEROPLANES; HELICOPTERS
- B64C13/00—Control systems or transmitting systems for actuating flying-control surfaces, lift-increasing flaps, air brakes, or spoilers
- B64C13/02—Initiating means
- B64C13/16—Initiating means actuated automatically, e.g. responsive to gust detectors
- B64C13/18—Initiating means actuated automatically, e.g. responsive to gust detectors using automatic pilot
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENTS OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D43/00—Arrangements or adaptations of instruments
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENTS OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D45/00—Aircraft indicators or protectors not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64F—GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
- B64F5/00—Designing, manufacturing, assembling, cleaning, maintaining or repairing aircraft, not otherwise provided for; Handling, transporting, testing or inspecting aircraft components, not otherwise provided for
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0416—Control or interface arrangements specially adapted for digitisers
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F7/00—Methods or arrangements for processing data by operating upon the order or content of the data handled
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENTS OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D45/00—Aircraft indicators or protectors not otherwise provided for
- B64D2045/0075—Adaptations for use of electronic flight bags in aircraft; Supports therefor in the cockpit
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENTS OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D45/00—Aircraft indicators or protectors not otherwise provided for
- B64D2045/0085—Devices for aircraft health monitoring, e.g. monitoring flutter or vibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T50/00—Aeronautics or air transport
- Y02T50/40—Weight reduction
Definitions
- the present invention relates to the field of flight control systems, methods, and apparatuses; even more particularly, to a system, method, and apparatus for providing aircraft state monitoring and/or an automated aircrew.
- Centaur can be installed into, and removed from, the DA42 without affecting the original type certificate, thus maintaining the aircraft's original certification.
- Centaur includes mechanical actuation of the primary flight controls and its own avionics suite, and may be used with a pilot in a supervisory role or as a fully unmanned aircraft. For example, Centaur may be flown by an operator in the back seat of the airplane, directing the flight plan on a laptop. While Centaur offers many features, it suffers from certain drawbacks.
- Centaur hardware is not portable to other aircraft, nor is the software plug-and-pay extensible to other capabilities; (2) parts of the Centaur system are invasive and require cutting into existing avionics wiring in a manner very specific to the aircraft (i.e., the DA42); (3) Centaur does not allow the onboard pilot to be the operator and to perform tasks such as directing the flight plan; and (4) Centaur does not acquire knowledge about the aircraft it is operating.
- An example application includes the operation of aircraft where fatigue and boredom can cause a reduction in crew attentiveness, in which case the aircrew automation system reduces risk in a flight operation by alerting the pilot and, in certain instances, assuming control of the aircraft.
- Other example applications exist where the potential for human error currently limits extensive use of aircraft (e.g., low-altitude operations), synchronized operations, unmanned flights, unmanned formations with manned flight lead, and improved debrief capabilities due to comprehensive data logging.
- the present invention is directed to flight control systems, methods, and apparatuses; even more particularly, to a system, method, and apparatus for providing aircraft state monitoring and/or automated aircrew.
- the aircrew automation system is vehicle-agnostic.
- the aircraft state monitoring system is a perception system.
- the aircraft state monitoring system is hardwired to one of more cockpit instruments.
- the aircrew automation system employs an operating system that operates as the middleware to interconnect one or more operational applications or hardware interfaces.
- the aircrew automation system is configured to provide a pilot with continuous aircraft state monitoring and information augmentation with or without taking control of the aircraft.
- the autopilot manager is configured to control an aircrew automation actuation module or an aircraft actuation module.
- the commands from the core platform are generated based at least in part on the state of the aircraft.
- the core platform displays the current aircraft state via a display of the human machine interface.
- the core platform employs an open architecture.
- the core platform includes a flight controller system and/or a mission control system.
- the core platform is configured to communicate one of more commands to a flight control system of the aircraft based at least in part on the flight situation data.
- the core platform is configured to develop, using information collected from one or more systems coupled thereto, an understanding of the aircraft's systems, configurations, and procedures necessary to maintain safe operation.
- the core platform is configured to receive and aggregate the flight situation data to determine a current aircraft state.
- the core platform is configured with plug and play software to enable (1) removal or disability of systems or subsystems or (2) installation or activation of systems or subsystems.
- the flight controller system includes one or more of an autopilot manager, a vehicle manager, a state estimation module, a perception module, or a navigation module.
- the flight situation data includes airspeed, altitude, and vertical speed of the aircraft.
- the hardware interfaces include at least one of a primary actuation interface, a secondary actuation interface, an aircraft state interface, or an HMI interface.
- the human machine interface is a tablet computer.
- the human machine interface enables a pilot to control and communicate with the aircrew automation system.
- the human machine interface includes a tool bar with a plurality of selectable tabs.
- the human machine interface includes a touch screen display.
- the touch screen display is compatible with night-vision goggles.
- the human machine interface is configured to display a plurality of tasks on the touch screen display in a form of a task list. In certain aspects, each of the plurality of tasks is marked as either completed or not completed, for example, based at least in part on a pilot input via a touch screen display. [0036] In certain aspects, the human machine interface is configured to display at least one of checklists, health alerts, predictions of aircraft state, and failure prognosis.
- the human-machine interface employs speech-recognition techniques to recognize a verbal communication from the pilot.
- the human-machine interface is configured to general verbal response to the verbal communication from the pilot.
- the information specific to the aircraft includes whether the aircraft is susceptible to a particular operational error.
- the knowledge acquisition system is configured to determine information specific to the aircraft to yield a vehicle-agnostic aircrew automation system.
- the knowledge acquisition system is configured to determine the layout of the cockpit instruments.
- the operational applications include at least one of a normal flight operation application, an anomaly detection application, a contingency operation application, an intelligence, surveillance, and reconnaissance (“ISR") application, a trend recognition application, or an aerial refueling application.
- a normal flight operation application an anomaly detection application, a contingency operation application, an intelligence, surveillance, and reconnaissance (“ISR") application, a trend recognition application, or an aerial refueling application.
- ISR intelligence, surveillance, and reconnaissance
- the perception system includes a vision system having a camera to visually monitor one of more cockpit instruments of the aircraft.
- the plurality of selectable tabs includes one or more of a route tab, a procedures tab, a calibration tab, and an applications tab.
- the aircrew automation system further comprises an actuation system operatively coupled with the core platform.
- the actuation system comprises a primary actuation system and a secondary actuation system that is separate from the primary actuation system.
- the aircrew automation system is configured to control of the aircraft via an/the actuation system.
- the primary actuation system controls actuation of the aircraft's primary flight controls and/or the secondary actuation system controls actuation of the aircraft's secondary flight controls.
- the aircraft's primary flight controls include one or more stick controls or yoke controls.
- the aircraft's secondary flight controls include one or more switches and/or knobs.
- the primary actuation system includes a frame, an articulating arm coupled at its proximal end to the frame, and a gripper positioned at a distal end of the articulating arm to engage at least one of the aircraft's primary flight controls.
- the primary actuation system includes an articulating arm with a gripper to engage at least one of the aircraft's primary flight controls.
- the gripper is coupled to the articulating arm via a multiple-
- the articulating arm is rotatable and slideably coupled at its proximal end to the frame via a movable base.
- the secondary actuation system includes an XY-plotter defining a Y-axis and an X-axis, a tool to engage at least one of the aircraft's secondary flight controls, and a control system to move the tool along the Y-axis and the X-axis.
- the tool is a multi-tool configured with a switch actuator and a knob actuator, to engage a switch or a knob, respectively.
- Figure la illustrates a block diagram of an example aircrew automation system.
- Figure lb illustrates an example flow of information data between the subsystems of Figure la.
- Figure lc illustrates a block diagram of an example core platform.
- Figure 2 illustrates a diagram of an example core platform architecture.
- Figure 3a illustrates a first example human-machine interface illustrating a route application.
- Figure 3b illustrates a second example human-machine interface illustrating a procedural checklist and aircraft health alert screen.
- Figure 4 illustrates a block diagram of an example perception system.
- Figures 5a and 5b illustrate an example primary actuation system.
- Figure 5c illustrates an example secondary actuation system.
- circuits and “circuitry” refer to physical electronic components (i.e. hardware) and any software and/or firmware ("code”) which may configure the hardware, be executed by the hardware, and or otherwise be associated with the hardware.
- code software and/or firmware
- a particular processor and memory may comprise a first "circuit” when executing a first set of one or more lines of code and may comprise a second "circuit” when executing a second set of one or more lines of code.
- "and/or” means any one or more of the items in the list joined by “and/or”.
- "x and/or y” means any element of the three-element set ⁇ (x), (y), (x, y) ⁇ .
- "x and/or y” means “one or both of x and y”.
- "x, y, and/or z” means any element of the seven-element set ⁇ (x), (y), (z), (x, y), (x, z), (y, z), (x, y, z) ⁇ .
- x, y and/or z means “one or more of x, y and z".
- exemplary means serving as a non-limiting example, instance, or illustration.
- terms “e.g.,” and “for example” set off lists of one or more non-limiting examples, instances, or illustrations.
- the words “about” and “approximately,” when used to modify or describe a value (or range of values), mean reasonably close to that value or range of values.
- the embodiments described herein are not limited to only the recited values and ranges of values, but rather should include reasonably workable deviations.
- circuitry or a device is "operable" to perform a function whenever the circuitry or device comprises the necessary hardware and code (if any is necessary) to perform the function, regardless of whether performance of the function is disabled, or not enabled (e.g., by a user-configurable setting, factory trim, etc.).
- VTOL aircraft refers to a machine capable of flight, including, but not limited to, both traditional runway and vertical takeoff and landing (“VTOL”) aircraft.
- VTOL aircraft may include fixed-wing aircraft (e.g., Harrier jets), rotorcraft (e.g., helicopters), and/or tilt-rotor/tilt-wing aircraft.
- the terms “communicate” and “communicating” refer to (1) transmitting, or otherwise conveying, data from a source to a destination, and/or (2) delivering data to a communications medium, system, channel, network, device, wire, cable, fiber, circuit, and/or link to be conveyed to a destination.
- the term “database” as used herein means an organized body of related data, regardless of the manner in which the data or the organized body thereof is represented.
- the organized body of related data may be in the form of one or more of a table, a map, a grid, a packet, a datagram, a frame, a file, an e-mail, a message, a document, a report, a list, or data presented in any other form.
- a system configured to, inter alia, function as a pilot's assistant (or co-pilot) or flight engineer.
- Such an aircrew automation system may be configured to operate an aircraft from takeoff to landing, automatically executing the necessary flight and flight plan activities, checklists, and procedures at the correct phases of flight, while detecting contingencies and responding to them.
- the pilot e.g., a human pilot or operator
- the aircrew automation system may provide real-time information and/or feedback to the pilot.
- the aircrew automation system may indicate a state of the aircraft relative to the procedure being accomplished.
- the aircrew automation system may be configured to take back control of the aircraft through robotic actuators, if desired.
- the pilot is enabled to perform tasks best suited for humans, such as high-level decision-making and flight plan planning. Tasks best suited for automation, however, may be handled by the aircrew automation system, including, for example, the manipulation of controls, executing checklists, and monitoring aircraft engine and performance data. Additionally, the aircrew automation system may have the capability to access external information that is either currently unavailable to pilots or that which only comes with experience, such as common system failures for a particular aircraft or how air traffic control commonly routes traffic at a particular airport. The aircrew automation system may be configured to operate as an assistant or as the primary pilot, thereby, if so configured, entirely obviating the need for a human operator.
- the aircrew automation system may serve to provide a pilot with continuous aircraft state monitoring and information augmentation, without actually taking control of the aircraft.
- the aircrew automation system may serve as a "second set of eyes" for the pilot, monitoring checklists, instrumentation, engine state, airspace, flight regime, etc.
- an aircrew automation system in accordance with an aspect of the present invention employs a system architecture and knowledge acquisition system that enables rapid non-invasive installation, which facilitates widespread use and enables the aircrew automation system to be quickly adapted for use in a variety of aircraft.
- the aircrew automation system' s data collection and perception systems are not limited to GPS, accelerations, orientation, and heading, as is the case with existing robotic autopilots.
- the aircrew automation system exceeds the capability of existing data collection and perception systems to better capture aircraft performance by employing both standalone sensors, instrument image data capture (e.g., temperature, altitude, radar, flap angles, etc.), and measuring, detecting, or otherwise receiving pilot inputs.
- instrument image data capture e.g., temperature, altitude, radar, flap angles, etc.
- pilot assist systems the aircrew automation system's core platform and design of the primary and secondary flight control actuation systems enables portability across a variety of aircraft.
- the aircrew automation system may be temporarily installed and readily transferred from aircraft to aircraft, without invasive modification to the aircraft.
- the aircrew automation system through its modular design, further reduces the likelihood of designing a single point solution that becomes obsolete as aircraft evolve.
- the aircrew automation system' s combination of subsystems provides a pilot with high-fidelity knowledge of the aircraft' s physical state, and notifies that pilot of any deviations in expected state based on, for example, predictive models. This state awareness may be translated directly into useful information for the pilot, such as alerts to developing emergency conditions, fuel state computation, notification of icing conditions, etc.
- the aircrew automation system may also serve as a digital flight engineer, thereby advising the pilot by monitoring checklists, instrumentation, engine state, airspace, flight regime, etc.
- This ride-along aircrew automation system which may be non-invasively installed in preexisting aircraft, perceives the state of the aircraft visually and via other sensors, derives the aircraft state vector and other aircraft information, and communicates any deviations from expected aircraft state to the pilot or a control tower. While the aircrew automation system may be non-invasively installed (e.g., via a perception system), it may alternatively be invasive. For example, the aircrew automation system may electronically couple with the cockpit instrument panel (e.g., via the reverse side of the instrument panel) via, for example, the aircraft state monitoring system. Alternatively, the aircrew automation system may be integral and permanently installed during fabrication of the aircraft. In conjunction with an actuation system, the aircrew automation system may further take control of the aircraft and autonomously navigate the aircraft.
- FIG. 1 System Level Architecture. To share the duties and workload related to the execution of flight activities, the aircrew automation system 100 should be capable of executing the actions a pilot would perform routinely over the duration of a flight, regardless of the aircraft make, model, or type.
- An example system architecture for an aircrew automation system 100 in accordance with one aspect is shown in Figures la through lc. As illustrated in Figure la, the core platform 102 may operate as a central subsystem that connects the other subsystems via one or more interfaces. The subsystems may communicate with one another through software and/or hardware interfaces 156 using wired and/or wireless communication protocols and hardware.
- Figure lb illustrates an example flow of information (e.g., data) between the various subsystems.
- the aircrew automation system 100 may comprise a core platform 102 operatively coupled with a plurality of subsystems, such as those listed below.
- Each of the plurality of subsystems of the aircrew automation system 100 may be modular, such that the entire aircrew automation system 100 can be substantially ported to another aircraft rapidly.
- the various subsystems may be removably and communicatively coupled to one another via the core platform 102 using one or more software and/or hardware interfaces 156.
- the aircrew automation system 100 may alternatively be integral with the aircraft's system, thereby directly employing all sensors and indicators in the airplane.
- the aircrew automation system 100, or components thereof may be integrated into the aircraft during its design and manufacturing.
- the plurality of subsystems may include, for example, a perception system 106, an actuation system 108, a human machine interface ("HMI") system 104, and a flight control system 116, each of which may be operatively coupled with the core platform 102.
- a perception system 106 may be mitigated or obviated via use of another aircraft state monitoring system.
- the aircrew automation system 100 may couple (e.g. , communicatively or electronically) with the instrument panel, or be otherwise integrated with the aircraft or its systems. As can be expected, however, such integration would likely require a degree of modification to the aircraft or its wiring.
- the aircrew automation system 100 and/or core platform 102 may also comprise, or be operatively coupled to, a knowledge acquisition system 114 and a communication system 122.
- the modular configuration further enables the operator to remove/disable unnecessary systems or modules or to add/install additional systems or modules.
- the aircrew automation system 100 is configured to only provide information to the pilot via the HMI system 104 (i.e. , without the ability to control the aircraft), the actuation system 108 may be removed or disabled to reduce weight, cost, and/or power consumption. Accordingly, depending on the configuration, the aircrew automation system 100 may be configured with fewer or additional module, components, or systems without departing from the spirit and scope of the invention.
- the flight control system 116 derives the aircraft state based on information data from another subsystem (e.g. , perception system 106) and directs another subsystem (e.g. , the actuation system 108) to operate (e.g. , dynamically) in a manner to maintain aircraft stability.
- the flight control system 116 may receive vehicle mode commands and configuration data from the core platform 102, while sending to the core platform 102 status and command information generated by the flight control system 116.
- the core platform may be configured to communicate one of more commands to the flight control system 116 of the aircraft based at least in part on the flight situation data, which may be obtained from the aircraft state monitoring system 112, the perception system 106, or a combination thereof.
- the flight control system 116 may include, or communicate with, existing flight control devices or systems, such as those used in fixed wing aircraft and rotary wing aircraft.
- the communication system 122 enables the aircrew automation system 100 to communicate with other devices (including remote or distant devices) via, for example, a network.
- the communication system 122 may receive communication commands and configuration data from the core platform 102, while sending to the core platform 102 status and response information from the communication system 122.
- Core platform 102 illustrates an architecture diagram of an example core platform 102.
- a core platform 102 may provide, or otherwise serve as, middleware that can be made specific to a particular aircraft or configuration through an initial transition and setup phase.
- the mission control system 110 may provide an operating system 206 that provides services to a set of operational applications 202 and output signals to one or more of a set of hardware interfaces 204 or HMI system 104, while collecting and logging the data necessary to enable those applications.
- the core platform 102 serves as the primary autonomous agent and decisionmaker, which synthesizes inputs from the perception system 106 and HMI system 104 with its acquired knowledge base to determine the overall system state.
- the core platform 102 may process inputs from the various sensor suites and aggregate the resultant information into an understanding of current aircraft state.
- the resultant information may be compared against an aircraft specific file that encompasses aircrew automation system's 100 understanding of pilot intent, system health, and understanding of appropriate aircraft procedures as they relate to the aircrew automation system's 100 state estimation.
- the resultant state knowledge and associated recommendations can be passed to a human pilot via the HMI system 104 or, in certain aspects, to the flight control system 116 and/or actuation system 108 to enable autonomous operation.
- the aircrew automation system 100 may further generate a log of a given flight for later analysis, which may be used to facilitate pilot training that can provide detailed training and operations flight debriefs.
- the logs may be used in connection with, for example, flight operational quality assurance analysis, maintenance analysis, etc.
- the core platform 102 may comprise a mission control system 110 and flight controllers 118, each of which are configured to communicate with one another and the other subsystems via one or more software and/or hardware interfaces 156, which may be a combination of hardware (e.g., permanent or removable connectors) and software.
- the core platform 102 can host various software processes that track the aircraft and procedure states, as well as any modules for trend analytics (predictive warnings) and machine learning routines.
- the aircrew automation system 100 and/or core platform 102 may employ a computer bus and specification (e.g.
- a user may readily add or remove system or subsystems (e.g. , as modules) to the aircrew automation system 100 via the core platform 102 without requiring substantive modification or integration efforts.
- the core platform 102 outputs may be used to provide messages to the HMI system 104.
- the messages may indicate, for example, checklist progress, contingencies to initiate, warnings to raise, etc.
- the core platform 102 may also contain a flight data recorder, for instance to provide performance review capability and to provide robustness against in-flight reset.
- the hardware and various computers may also be ruggedized and share a housing with other devices, such as the perception computer.
- the core platform 102 may be operatively coupled with a global positioning system (“GPS")/inertial navigation system (“INS”) system 154 and power management system (e.g. , 28VDC power).
- GPS global positioning system
- INS intial navigation system
- the core platform 102 may also contain a flight data recorder, for instance to provide performance review capability and to provide robustness against in-flight reset.
- the mission control system 110 generally comprises a mission manager 132, a standard interface 130 (e.g., a STANAG interface), a state awareness manager 158, and other operational components 120 (e.g., hardware and software controllers and/or interfaces), each of which are communicatively coupled to one another via one or more data buses 124.
- the open architecture of the core platform 102 enables the incorporation of additional data received from systems via the data bus 124.
- the mission control system 110 may be coupled with one or more cockpit instruments of the aircraft via the vehicle systems interface to collect flight situation data.
- the mission control system 110 may collect flight situation data through an aircraft state interface via the aircraft state monitoring system 112, which may collect or generate flight situation data via a direct connection to the aircraft and/or the perception system 106.
- the mission control system 110 may be operationally coupled with the secondary actuation system 108b (e.g. , when autonomous operation is desired), the perception system 106, and the HMI system 104, including the human-machine interface 126 (e.g., software and/or hardware that conveys inputs from and displays information to the pilot), and ground station 128.
- the mission control system 110 may communicate with the flight controllers 118 via the mission manager 132.
- the flight controllers 118 may include, for example, an autopilot manager 134 and a vehicle manager 136.
- the vehicle manager 136 may be generally responsible for navigation and determining the location and state of the aircraft.
- the vehicle manager 136 may be coupled with a state estimation module 142, which determines the estimated state of the aircraft using information received from the perception system 106 via a perception module 138 and from the GPS/INS system 154 via a navigation module 140.
- the autopilot manager 134 may be generally responsible for controlling the aircraft's flight based on, for example, information received from the vehicle manager 136 and the mission control system 110.
- the autopilot manager 134 controls, inter alia, the flight control system 152, which may be new or preexisting (and comprises a flight controller 150), as well as the aircrew automation actuation module 144 and the aircraft actuation module 146.
- the aircrew automation actuation module 144 may control the primary actuation system 108 a, while the aircraft actuation module 146 may control the aircraft controls 148 (e.g., various flight surfaces and actuators).
- the flight controller's 118 components may overlap with certain components of the flight control system 116.
- the core platform 102 may exploit certain of the existing aircraft software and/or hardware, thereby obviating the need for additional hardware, such as certain flight controller 118 components and/or a GPS/INS system 154.
- the core platform 102 serves as the central subsystem, or interface, of the aircrew automation system 100, connecting and controlling the remaining subsystems (e.g., as individual applications) in an open architecture.
- the remaining subsystems include, for instance, the flight control system 1 16 (including any flight plan capabilities), the HMI system 104, the actuation systems 108 (e.g., the primary and secondary actuation systems to provide autonomous operation where desired), the perception system 106, knowledge acquisition system 114, and other subsystems 236.
- control of the other aircrew automation system 100 hardware may be provided via separate applications specific to a particular piece of hardware, which enables rapid integration of new systems or other external flight plan support technology.
- the core platform's 102 architecture enables rapid portability and extensibility when transitioning to a new aircraft or incorporating a new flight plan feature/capability.
- an application may be used to enable the aircrew automation system 100 to acquire information specific, or otherwise needed, for that aircraft or to provide the new capability.
- transition and setup can be handled by individual applications that operate within the core platform 102 or other subsystems, representing aircraft- specific functionalities as well as a growing library of capabilities of aircrew automation system 100, which can be exchanged depending on flight plan, aircraft or crew requirements.
- the transition process may be supported by software applications external to the aircrew automation system 100 (such as a procedure editor).
- Aircraft data structure 208 The operating system 206 operates as the middleware, interconnecting the operational applications 202, hardware interfaces 204, and other subsystems, such as the knowledge acquisition system 114.
- the operating system 206 may employ an aircraft data structure 208, which may include a knowledge database 210, a procedure database 212, and a state database 214.
- the aircraft data structure 208 facilitates a vehicle-agnostic aircrew automation system 100 by enabling the core platform 102 to develop a complete understanding of an aircraft's systems, their configuration, and the procedures necessary to maintain safe operation, and all other knowledge and expertise a certified pilot of that aircraft would be expected to have.
- the aircraft data structure 208 may be populated by the knowledge acquisition system 114 (discussed below), which contains necessary information about the aircraft currently being operated (e.g., flight control model, operational procedures, aircraft systems, etc.), data received from internal state sensors, and other subsystems or sensors.
- the aircraft data structure 208 can be populated and adjusted to a specific aircraft during a knowledge acquisition phase (e.g., during initial setup) such that it contains all the information necessary to operate the aircraft.
- the knowledge acquisition system 114 may perform predefined activities in order to determine the layout (e.g. , of the controllers/read outs, such as the cockpit instruments), performance parameters, and other characteristics of the aircraft.
- the predefined activities may include, for example: (1) generation of an aircraft system model, which informs aircrew automation system 100 about which systems are onboard and how they are configured, actuation limits, etc.; (2) procedure codification, which informs aircrew automation system 100 how to operate aircraft in normal and non-normal situations, further including the codification of checklists; (3) an aerodynamic model, which informs the aircrew automation system 100 how to fly the aircraft and what performance to expect for which aircraft configurations; and (4) information about mission operations.
- the core platform 102 can combine this information with data from a set of internal state sensors, which also improve redundancy and system robustness, thereby allowing the aircrew automation system 100 to generate a highly accurate estimate of the aircraft state and system statuses, and to identify deviation from expected behavior.
- the data structure is dynamically updated with real-time data gathered by, inter alia, the aircrew automation system's 100, perception system 106, the HMI system 104, as well as the aircrew automation systems 100 internal state sensing.
- the aircraft data structure 208 for a given aircraft is populated, the aircraft data structure 208 can then be retained in an aircraft library and used for all other aircraft of the same make and model for which aircrew automation system 100 is available.
- the aircraft data structure 208 may be further refined as additional data is generated and/or collected by the aircrew automation system 100.
- Operational applications 202 may provide the aircrew automation system 100 with a plurality of operational applications 202.
- operational applications 202 might include, without limitation, normal flight operation application 216, an anomaly detection application 218, a contingency operation application 220, an intelligence, surveillance, and reconnaissance (“ISR") application 222 ⁇ e.g., ISR orbits), a trend recognition application 238, or other flight plan-specific activity applications 224, such as an aerial refueling application 316.
- ISR intelligence, surveillance, and reconnaissance
- the normal flight operation application 216 enables aircrew automation system
- the normal flight operation application 216 is specific to the continuous execution of normal flight activity, as needed by a particular flight phase.
- the predetermined flight plan may be modified in flight due to unexpected disturbances such as weather, air traffic control commands, air traffic, etc.
- the anomaly detection application 218 employs machine learning techniques to monitor aircraft state, cluster, and classify sensor inputs in order to detect the presence of non- normal situations, and to identify whether a contingency has occurred.
- the anomaly detection application 218 is configured to compare the sensed states against a set of thresholds defined in the operational documentation for the specific aircraft ⁇ e.g., never exceed a predetermined airspeed, engine temperature, etc.).
- the anomaly detection application 218 may also compare the sensed states against additional information available to aircrew automation system 100 and generate alerts or other messages in response to meeting predetermined or dynamically determined thresholds ⁇ e.g., warning thresholds, etc.).
- a contingency operation application 220 executes the necessary predetermined checklists, procedures, and actions specified by the contingency application 220 in order to maintain safe operation of the aircraft or safely divert the flight. Notably, if a departure from expected performance is observed, the pilot can be alerted to a non-normal condition, thereby mitigating or avoiding potential mistakes. If an aircraft is susceptible to a particular operational error (e.g., pilot induced oscillations), the aircrew automation system 100 can identify and mitigate such events. If an anomaly is detected, the contingency operation application 220 informs and interacts with the pilot via the HMI system 104 and ultimately executes the necessary procedure(s) to respond to the anomaly. Finally, the ISR application 222 and other flight plan-specific activity applications 224 may provide instructions, algorithms, or information to carry out operations relevant to a mission.
- the ISR application 222 and other flight plan-specific activity applications 224 may provide instructions, algorithms, or information to carry out operations relevant to a mission.
- the trend recognition application 238 provides trend analysis developed using machine learning based on, for example, the knowledge acquisition system 114.
- the trend recognition application 238 may supply data, or otherwise trigger, the anomaly detection application 218. For example, if the trend recognition application 238 detects an undesirable trend, the trend may be flagged as an anomaly and reported to the anomaly detection application 218.
- Hardware interfaces 204 The various information pertaining to the operational applications 202 are communicated between the primary actuation system 108a, secondary actuation system 108b, perception system 106, aircraft state monitoring system 112, HMI system 104, and other subsystems 236 via, for example, the primary actuation interface 226, secondary actuation interface 228, aircraft state interface 230, HMI interface 232, and other interface 234.
- the HMI system 104 provides a control and communication interface for the pilot (e.g., a human pilot, whether on-board or remote).
- the HMI system 104 is configurable to operate as a flight plan manager that enables the pilot to direct the aircrew automation system 100.
- the HMI system 104 can combine elements of glass cockpits, unmanned aerial vehicle (“UAV”) ground stations, and electronic flight bags (EFB) to enable effective, efficient and latency-tolerant communication between the pilot and aircrew automation system 100.
- UAV unmanned aerial vehicle
- EFB electronic flight bags
- an EFB is an electronic information management device that allows flight crews to perform a variety of functions that were traditionally accomplished by using paper references.
- the HMI system 104 may include a human-machine interface 126, which may be based on a touch screen graphical user interface ("GUI") and/or speech-recognition systems.
- GUI graphical user interface
- the human-machine interface 126 may employ, for example, a tablet computer, a laptop computer, a smart phone, or combination thereof.
- the human-machine interface 126 can be secured near the pilot (e.g. , on the yoke - as checklists often are, or on a knee-strap) depending on pilot preferences.
- the human-machine interface 126 may be removable coupled to the cockpit or, in certain aspect, employ an integrated display within the cockpit (e.g. , an existing display).
- FIG 3a illustrates an example human-machine interface 126 having a single- screen touch interface and speech -recognition system.
- the HMI system 104 serves as a primary channel of communication between the pilot and the aircrew automation system 100, enabling the pilot to command tasks to and receive feedback or instructions from the aircrew automation system 100, to change the allocation of tasks between pilot and aircrew automation system 100, and to select which operational applications 202 are currently enabled for the aircrew automation system 100.
- the HMI system 104 may receive status information from a subsystem via the core platform 102, while sending to the core platform 102 mode commands generated by the HMI system 104 or input by the pilot.
- the pilot may be remote (e.g., on the ground or in another aircraft) or on-board (i.e. , in the aircraft).
- the HMI system 104 may be remotely facilitated over a network via communication system 122.
- Human-machine interface 126 As illustrated in Figures 3a and 3b, the human- machine interface 126 may employ a tablet based GUI and a speech-recognition interface that enables vocal communications.
- An objective of the human-machine interface 126 is to enable the pilot to interact with the core platform 102' s knowledge base in manner akin to the way a pilot interacts with a human flight engineer or copilot.
- the human-machine interface 126 can display the current state of aircrew automation system 100 (its current settings and responsibilities) as well as which operational applications 202 are currently installed, which operational applications are running and, if they are active, which actions the operational applications 202 are taking.
- the human-machine interface 126' s GUI display may also be night-vision goggles compatible such that it is visible regardless of the pilot's eyewear.
- the speech-recognition system may be used to replicate the same types of verbal communications used by human flight crew when running through checklists and communicating on the flight deck. In certain aspects, the speech recognition may be limited to the same standards of codified communications used by pilot teams to minimize the chances of the system failing to recognize commands or changing into inappropriate modes of operations.
- the speech-recognition system may be configured to learn/recognize the speech of a given pilot through a voice training protocol. For example, the pilot may speak a predetermined script such that the speech-recognition system can become trained with the pilot' s dialect.
- the human-machine interface 126 may provide the status and/or details of various operations, including the entire aircrew automation system 100 via the aircrew automation status application 302, the perception system 106 via the perception status application 304, the autopilot via the autopilot status application 306 (where applicable), the GPS/INS system 154 via the GPS status application 308, and any other application or system status information 310.
- the display of the human-machine interface 126 may be customized by the pilot. For example, the pilot may wish to add, reorganize, or remove certain of the display icons and/or operational applications 202, which may be accomplished through a select and drag maneuver or through the aircrew automation settings application 312.
- the human-machine interface 126 may further inform the pilot regarding the aircraft's operating status and to provide the pilot with instructions or advice.
- the human-machine interface 126 may provide a tool bar with various selectable tabs, such as a route tab 328, a procedures tab 330, a calibration tab 332, and an applications tab 334.
- the human- machine interface 126 may display the various operational applications 202 installed on the aircrew automation system 100 (e.g. , the core platform 102), including, for example, a normal flight operation application 216, a contingency operation application 220, an aircrew automation settings application 312, gauge application 314, and aerial refueling application 316.
- Selecting the aircrew automation settings application 312 enables the pilot to change, reallocate, or otherwise edit the settings of the aircrew automation system 100 and/or to install operational applications 202.
- Selecting the gauge application 314 may cause the human- machine interface 126 to display the various operational conditions of the aircraft, including, for example, position, direction, speed, altitude, pitch, yaw, etc.
- the various operational conditions of the aircraft which may be gathered from the perception system 106 or another sensor, may be displayed as alphanumeric characters or as graphical dials (e.g. , in accordance with the pilot's preference settings).
- selecting the aerial refueling application 316 icon may cause the aircrew automation system 100 to perform a predetermined protocol for facilitating or coordinating a mid-air refueling operation.
- the aircrew automation system may coordinate with another aircraft to facilitate refueling and perform the necessary checklists for doing the same (e.g. , ensuring aircraft position, airspeed, fuel hatch opening, etc.).
- Additional mission applications may be included that enable performance of mission operations by the aircrew automation system.
- the human-machine interface 126 may display an area map 326 with an icon 322 representing the current location of the aircraft along a flight path relative to its various waypoints 320. Selecting (e.g., tapping, clicking, etc.) the icon 322 may cause a dialog window 324 to display that provides the various operational conditions of the aircraft.
- the area map 326 may be saved, exported, rotated, or panned using a map control window 318.
- the area map 326 may be saved or exported (e.g., via communication system 122) as a static image or a data set (or database).
- the human-machine interface 126 may display the calibration of the aircraft, whereby the pilot may be further enabled to revise the same.
- the HMI system 104 may provide an intuitive display and interface that includes checklist verification and health alerts from the core platform 102 and predictions of aircraft state (e.g. , fuel consumption and predicted remaining range), as well as failure prognosis and deviation alerts (e.g. , "Left engine EGT is 5 degrees above normal and rising").
- checklist verification and health alerts from the core platform 102 and predictions of aircraft state (e.g. , fuel consumption and predicted remaining range), as well as failure prognosis and deviation alerts (e.g. , "Left engine EGT is 5 degrees above normal and rising").
- aircraft state e.g. , fuel consumption and predicted remaining range
- failure prognosis and deviation alerts e.g. , "Left engine EGT is 5 degrees above normal and rising”
- a function of the HMI system 104 is to facilitate checklist monitoring and/or execution, marking items as complete when the when the perception system 106 perceives their completion and providing warnings to the pilot when items are not completed, as based on information previously imported from, for example, a Pilot's Operating Handbook (POH).
- the aircrew automation system 100 also monitors system health, comparing the current system state to that expected based on the POH and other knowledge sources, and guides appropriate responses to contingencies.
- either the pilot or the core platform 102 can acknowledge checklist actions as they are performed and the HMI system 104 automatically proceeds to the correct checklist as appropriate.
- the HMI system 104 may give visual and auditory alerts to direct the pilot' s attention to unattended checklist items, instruments that are displaying out-of-normal range values, or predicted events as the aircraft proceeds through the flight plan, which can be entered as a series of waypoints (for instance).
- a list of tasks may be provided alongside indicators that indicate whether the task has been completed, is being completed, or needs to be completed (e.g., a "check mark” icon to include complete, an "in progress" icon, and a "to be completed” icon).
- a list of health hazards may be provide, along with one or corresponding icons to indicated one or more operational conditions that are out of range.
- a low fuel indicator may be provided alongside a low fuel icon if fuel is low.
- the HMI system 104 can enable the pilot to limit the activities executed by the aircrew automation system 100, if any.
- the HMI system 104 may define the allocation of tasks between the pilot and aircrew automation system 100, their responsibilities, and the communication of information between the two, thereby functioning as a collaborative teammate of the pilot.
- the aircrew automation system 100 may operate, depending on configuration, in a purely advisory role (i.e., without any control over the aircraft), a fully autonomous role (i.e., controlling the flight control without pilot intervention), or an advisory role with the ability to control flight controllers.
- the HMI system 104 may be further designed to enable a pilot to go through a transitional phase, where the pilot specifies the aspects of flight operation for which the aircrew automation system 100 is responsible. For example, the HMI system 104 may display a list of tasks where the pilot may select whether the aircrew automation system 100 or the pilot is responsible for a given task on the list. The list of tasks may be provided to the HMI system 104 from a procedure editor, which is described below. Once the aircraft data structure 208 has been populated and refined such that the pilot better trusts the aircrew automation system 100, the pilot may allow aircrew automation system 100 to perform additional actions, transitioning the pilot from a primary mode to a supervisory mode (i.e., a fully autonomous role).
- a supervisory mode i.e., a fully autonomous role
- pilot interactions may be at a high, goal-based level, with the HMI system 104 supporting those tasks as well as allowing the operator insight at other levels for troubleshooting.
- all tasks may be performed by the pilot, leaving the aircrew automation system 100 to serve an advisory role.
- Mode awareness A risk when employing any automation system is the potential for mode confusion on the part of the pilot (e.g. , where the pilot neglects a task believing that the automation system will handle the task).
- the HMI system 104 avoids such mode confusion by first generating the correct function and the above-described task allocation between the aircrew automation system 100 and the pilot.
- the HMI system 104 allows the pilot to directly command and configure aircrew automation system 100 via the human-machine interface 126 and displays the information necessary for the pilot to understand what actions the aircrew automation system 100 is taking to ensure mode awareness.
- mode awareness generally refers to a state where the mode of the system matches the operational mode expected by the operator.
- the human-machine interface 126 may display the information necessary to ensure that the pilot is always aware of the mode in which aircrew automation system 100 is operating.
- the HMI system 104 serves as the human interface for individual mission applications (e.g., operational applications 202).
- the aircraft state monitoring system 112 collects, determines, or otherwise perceives the real-time aircraft state.
- the aircraft state monitoring system 112 may perceive the real-time aircraft state through, inter alia, a direct connection (e.g., integral with or otherwise hardwired to the aircraft) to the aircraft, or via perception system 106.
- the aircraft state monitoring system 112 may include a dedicated controller (e.g., processor) or share the controller 402 of the perception system 106.
- the perception system 106 for example, may employ a combination of a vision system, an acoustic system, and identification algorithms to read or comprehend flight situation information displayed by cockpit instruments.
- Example cockpit instruments include, for example, an altimeter, an airspeed indicator, a vertical speed indicator, one or more compass systems (e.g., a magnetic compass), one or more gyroscopic systems (e.g., attitude indicator, heading indicator, turn indicator), one or more flight director systems, one or more navigational systems (e.g., very-high frequency omnidirectional range (VOR), non-directional radio beacon (NDB)), etc.
- the perception system 106 may include a processor and one or more optical sensors (e.g. , three or more lightweight machine vision cameras) trained on the instrument panel to maximizes pixel density, glare robustness, and redundancy.
- the one or more optical sensors may wiredly connect to the perception computer via, for example, Ethernet.
- the one or more optical sensors should be installed with a line of sight with the instrument panel, but so as to be not obstructive to the pilot.
- the flight situation data perceived by the perception system 106 may be encoded and provided to the core platform 102 in real-time.
- the open architecture of the core platform 102 enables the incorporation of additional data received via a data bus 124 to augment the flight situation data generated by the perception system 106.
- the aircraft state monitoring systeml l2 and/or the perception system 106 may receive commands and configuration data from the core platform 102, while sending to the core platform 102 status and flight situation information (e.g., flight situation data) gathered by the perception system 106 or otherwise collected by the aircraft state monitoring systeml 12.
- FIG. 4 illustrates an example perception system 106 operatively coupled with, inter alia, the core platform 102 (which is coupled to other subsystems, such as flight control system 116), the GPS/INS system 154, and any other input systems 412.
- the perception system 106 visually and/or acoustically monitors, inter alia, the cockpit instruments to generate flight situation data that can be used to derive the aircraft state from cockpit layouts, which may range from basic analog aircraft instruments to highly integrated, glass cockpit avionics suites.
- the perception system 106 may also monitor instruments that are specific to aircraft systems such as fuel gauges and radios and provide secondary feedback about the status and positioning of the actuation system 108.
- the perception system 106 may comprise a perception controller
- the perception controller 402 may be, for example, a processor configured to feed flight situation data to (or otherwise instruct) the core platform 102 based upon information received and manipulated information received from the plurality of sensors, the database 404, and external components, such as the GPS/INS system 154 and other input systems 412.
- the perception system 106 may employ a monocular or stereovision system, possibly including motion capture markers, to continuously monitor the state of the aircraft by reading what is displayed on the cockpit instruments. In certain aspects, by comparing information about a scene from two vantage points, 3D information can be extracted by examining the relative positions of objects in the two panels.
- the vision system may be used to accurately monitor instruments ⁇ e.g., glass gauges, physical steam gauges, etc.) and switches, as well as their positions in a variety of lighting conditions and cockpit layouts and sizes.
- Using a stereovision system and/or markers also provides sensing to prevent collisions between any robotic components and the pilot.
- the vision system may employ a suite of high-definition, stereo cameras and/or a
- the system may be capable of recognizing data from all flight instruments and derive the state of switches knobs and gauges that display the state of aircraft specific systems (e.g., remaining fuel). It may also be capable of recognizing the state of the panel with enough resolution to detect minor changes that result from pilot actions.
- Machine vision algorithms on the perception system 106 computer read' the instruments (gauges, lights, wind correction angle panel, individual elements of the primary flight display or multi-function display in a glass cockpit) and mechanical items such as throttle levers, trim settings, switches, and breakers to provide a real-time cockpit state update to the core platform 102.
- the perception system 106 may be capable of deriving the aircraft state from cockpit layouts ranging from basic analog aircraft instruments to highly integrated, "glass cockpit" avionics suites. Through the vision system, the requirement for a data feed from the aircraft is obviated, which permits/increases portability across aircraft. However, when possible, the aircrew automation system 100 may also be coupled to an aircraft' s data feed (e.g. , through a data port). Further, using the application approach described for the core platform 102, different cockpit layouts can be addressed and understood using different underlying operation applications 202.
- the aircrew automation system 100 may employ the gauge application 314 to derive the values displayed on the instruments, whether graphical dial (e.g., analog "steam" gauges or digital representations thereof) or a glass cockpit.
- This approach would also enable the aircrew automation system 100 to run operational applications that monitor, inter alia, weather radars, traffic displays, and terrain maps displayed in the cockpit.
- the process of rapidly learning a new cockpit layout and codifying subtle differences in location and scaling or unit of instruments is addressed by the perception system 106 design.
- the location and scale of instruments and switches can be encoded and verified for a particular aircraft, reducing the real-time task to the extraction of the position of the graphical dial (round dial) or number (glass cockpit), whether graphical dial gauges, CRT display, LCD, etc.
- the piece-wise planar structure of cockpit instrumentation enables the perception system 106 to construe the images (e.g., using Homography methods) and register it against the pre-mapped data generated during the initial knowledge acquisition phase. Accordingly, live imagery can be registered and compared against the previously annotated model, thereby greatly simplifying interpretation of the data.
- Actuation system 108 When desired, an actuation system 108 executes the actions commanded via the core platform 102 to guide the flight and overall operation of the aircraft.
- the aircrew automation system's 100 actuation system 108 executes the actions commanded by the core platform 102 to guide the flight and overall operation of the aircraft without interfering with the activities performed by the pilot.
- the actuation system 108 may receive actuation commands and configuration data from the core platform 102, while sending to the core platform 102 status and response information generated by the actuation system 108.
- Manned aircraft cockpits are designed for the human reach envelope and, therefore, all cockpit controls are reachable by a comparably sized robotic/mechanical manipulator.
- the aircrew automation system 100 may separate the actuation of primary flight controls (stick/yoke, stick, side-stick or collective, rudder pedals, brakes, and throttles) from the actuation of secondary flight controls (e.g., switches, knobs, rockers, fuses, etc.). This approach reduces the likelihood of designing a single point solution that becomes obsolete as aircraft evolve.
- the aircrew automation system 100 may employ a primary actuation system 108a and a secondary actuation system 108b to physically control the actuators in the cockpit.
- the primary actuation system 108a may actuate the primary flight controls, while the secondary actuation system 108b may actuate the secondary flight controls, without obscuring the use of those controls by the pilot.
- the primary actuation system 108a and the secondary actuation system 108b are configured to collectively actuate all standard controls present on today' s flight decks during flight operations.
- the primary actuation system 108a focuses on actuating the primary flight controls (stick/yoke, stick, side-stick or collective, rudder pedals, breaks and throttles), while the secondary actuation system 108b focuses on actuating the controls that are not as easily accessed by the primary actuation system 108a, such as secondary flight controls (e.g., switches, knobs, rockers, fuses, etc.).
- secondary flight controls e.g., switches, knobs, rockers, fuses, etc.
- Primary actuation system 108a The primary actuation system 108a focuses on the set of controls necessary to safely operate the aircraft. As shown in Figures 5a and 5b, primary actuation system 108a include a frame 516 having an articulating arm 502 (e.g., a robotic appendage or "arm") and stick/yoke actuator 510 that actuates the primary flight controls (yoke, stick, side-stick or collective, rudder pedals, breaks and throttles) and other, easy to reach controls.
- the actuators may be one or more of linear (straight line), rotary (circular), or oscillatory actuators, which may be driven through one or more of electrical, pneumatic, and/or hydraulic techniques.
- the frame 516 may be sized and shaped to fit within the seat of a standard aircraft. To that end, the frame's 516 footprint should be about the same size as, or smaller than, an average human's "seated" footprint.
- the actuation system 108 may be fabricated using lightweight metals, metal alloys, and/or composite materials.
- Stick/yoke actuator 510 may couple to and engage the aircraft's existing stick/yoke 514 using a stick/yoke gripper 512.
- the stick/yoke gripper 512 may be sized and shaped such that it is universal and can engage various forms of stick/yokes and/or control wheels.
- the stick/yoke actuator 510 may be configured to move the stick/yoke 514 forward, backward, left, right, and positions therebetween.
- the stick/yoke gripper 512 may further comprise one or more actuators for actuating buttons and/or switches positioned on the stick/yoke 514.
- the actuator-controlled articulating arm 502 may be sized, shaped, and configured to occupy the space typically occupied by a co-pilot's arms, thereby ensuring portability across aircraft.
- the articulating arm 502 may comprise a plurality of arm segments (whether linear, curved, or angled) joined using a plurality of hinged or pivotal joints 506.
- the articulating arm 502 may comprise a gripper 504 at its distal end.
- the gripper 504 may be coupled to the articulating arm 502 via a multiple-DOF connection.
- the base of the articulating arm 502 may be rotatable and slideably coupled to the frame 516 via a movable base 508.
- the articulating arm 502 may be coupled with an upper base 508a, which is slideably coupled with a lower base 508b, which may be secured to the frame 516.
- the upper base 508a may slide relative to the lower base 508b using, for example, a combination of rails and ball bearings.
- the upper base 508a may slide relative to the lower base 508b along both the X- and Y- axis.
- the articulating arm 502 can be equipped with an encoder (e.g., twin 18-bit encoders) for each of its degrees of freedom to ensure exact positioning of the articulating arm 502.
- Internal clutches may be provided at each hinged or pivotal joint 506 such that the articulating arm 502 can be overpowered by the pilot if so desired, without damaging the articulating arm 502.
- the aircrew automation system 100 may determine the position or location of the articulating arm 502 using the encoders.
- the gripper 504 may be configured to couple, or otherwise engage, for example, throttle levers, etc.
- the gripper 504 may also provide force and pressure detection so as to allow the aircrew automation system 100 to estimate how a flight controls actuator is grasped and to adjust the motion to properly throw it. Once the motion is executed, the same feedback may be used to determine if the desired switch configuration has been achieved.
- the articulating arm 502 may be fitted with an electronic device (e.g., a homing device) that enables it to find and hit a target.
- Secondary actuation system 108b Unlike the primary flight controls, which are generally located in the same vicinity across aircraft makes and types, the location of the secondary flight controls (e.g., avionics, switches, knobs, rockers, toggles, covered switches, fuses, etc.) is not as consistent or spatially contained from aircraft to aircraft.
- the secondary flight controls e.g., avionics, switches, knobs, rockers, toggles, covered switches, fuses, etc.
- the secondary actuation system 108b focuses on actuating the controls that are not as easily accessed by the primary actuation system 108a. For example, some switches may even be on an overhead panel directly above the captain's head, making it potentially difficult to manipulate them with a robotic arm (especially in turbulent flight conditions). Accordingly, some actuators may be allocated to the above described primary actuation system 108a, while others may be allocated to a self-contained, secondary actuation system 108b.
- the secondary actuation system 108b may be provided in the form of an adaptable XY-plotter or gantry system mounted directly to the panel of interest and calibrated to the specific panel it is operating.
- the secondary actuation system 108b is preferably universal and resizable.
- An example XY-plotter is illustrated in Figure 5c.
- the XY-plotter may comprise a square frame that serves as the rails 520 of the plotter, a rotatable multi-tool 528 with multiple interfaces (e.g., switch actuator 532 and knob actuator 530) capable of manipulating the controls of interest, and a control system that moves this multi-tool 526 within the frame along a Y-axis set of rails 522 and an X-axis set of rails 524.
- the plotter moves the multi-tool 528 to the location, selects the correct manipulator interface, and manipulates the secondary flight control of interest.
- the multi-tool 528 that can flip binary switches and/or covered switches using a switch actuator 532 and can twist knobs using a knob actuator 530.
- the switch actuator 532 and/or knob actuator 530 may be coupled to the multi-tool 528 via an articulating or rotating member, such as the rotatable switch arm 534.
- the multi-tool 526 may return to a home position (e.g., automatically navigate to a far corner) to prevent obstruction of the panel.
- the multi-tool 526 would be equipped with sensors (e.g., proximity sensors) such that it can move out of the way when it detects the pilot' s hands.
- sensors e.g., proximity sensors
- the location, type, and position of the secondary flight control panel may be encoded. Once a particular secondary flight control panel is encoded, the configuration can be saved to the aircraft data structure 208 and loaded when aircrew automation system 100 is installed in the same aircraft, or the same type of aircraft.
- additional actuators may be provided to actuate controllers that are positioned in, for example, the foot well of the cockpit, such as foot pedals (e.g., brake and/or rudder pedals).
- Knowledge acquisition system 114 gathers and/or generates a knowledge base necessary to enable the aircrew automation system 100 to determine aircraft specific information. This includes knowledge of aircraft performance characteristics, limitations, checklists, and procedures (including emergency procedures), and criteria that define contingencies in the aircraft.
- the data may be derived from a combination of encoded data (e.g. , from manuals, pilot briefings, pilot operating handbook) and data acquired in flight (e.g. , via sensors), which supports off-line machine-learning and trend analysis.
- the date to be encoded may be loaded in .xml format that describes the contents of procedures and the flow of tasks both within and between procedures.
- the knowledge acquisition system 114 may receive operational commands from the core platform 102, while sending to the core platform 102 configuration data and status and response information generated by the knowledge acquisition system 114.
- the operation of the knowledge acquisition system 114 may be generally divided into three processes, including, for example, aircraft system modeling, procedure codification, and aerodynamic modeling.
- the aircraft system modeling process provides the aircrew automation system 100 with information about the available onboard systems and how the onboard systems are configured, actuation limits, etc.
- the procedure codification process provides the aircrew automation system 100 with information about aircraft operation in normal and non-normal situations.
- Procedure codification may include, for example, the codification of checklists.
- aerodynamic modeling process provides the aircrew automation system 100 with information about flying the aircraft and what performance to expect for a given aircraft type and configuration.
- the aircrew automation system 100 can fine-tune its aerodynamic and control models by observing a series of in-flight maneuvers flown by the pilot. This information includes flight dynamics data, operational limitations, procedures, aircraft systems, and layouts as well as other related data. In addition to written information, the aircrew automation system 100 may also codify information based on past events and experience of more experienced pilots. Machine learning enables the knowledge acquisition process to be performed efficiently and quickly.
- the population of aircraft data structure 208 may be accomplished using the
- XML Extensible Markup Language
- a XML data structure may be employed that comprises a set of fields and data trees that, when populated, allow the core platform 102 to configure and operate an aircraft.
- the aircrew automation system 100 may employ natural language interpretation of flight documents and/or a software tool that enables a human to enter the data efficiently and accurately.
- a set of airplane agnostic features may be generated and coded.
- Procedure Editor The aircraft specific information may be gathered during a transition period using, for instance, written documentation (e.g., pilot operating handbook, maintenance manual, etc.) as well as through direct monitoring of aircraft operations.
- the output of this knowledge acquisition process is the aircraft data structure 208, which is described above with regard to the core platform 102. Contained in this aircraft data structure 208 may be operational procedures, available systems and their designs, cockpit layout, and all other information necessary for safe operation of the aircraft.
- an aircrew automation software development kit may allow a software/flight controls engineer to specify, code, and unit-test one aircraft subsystem (e.g., electrical or hydraulic) per day.
- the aircrew automation software development kit can provide tools for turning the flight manual' s procedures into state machines compatible with Matlab State Flow and Simulink, which can then auto-code the procedures in C for inclusion in the core platform 102.
- the aircrew automation software development kit may also generate test code for the unit-level as well as interfaces for testing to the core platform 102. For example, the procedure editor may provide a list of tasks where the pilot may select whether the aircrew automation system 100 or the pilot is responsible for a given task on the list.
- a first step in knowledge acquisition of flight control uses the Athena Vortex Lattice ("AVL") method to generate the mathematical model in the form of non-dimensional stability derivatives that is used and refined during the flight with the pilot.
- the system ID trainer application may be used to perform a sequence of flight maneuvers designed to identify specific stability derivatives.
- the data is automatically processed into updated stability derivatives for use in the controller.
- the controller may employ an auto-tuner.
- the same updated stability derivatives are used in a 6-DOF simulation as a validation step that the controllers perform adequately prior to flight.
- An additional benefit of performing knowledge acquisition of flight control is that it enables the refinement and incorporation of a great deal of formal procedural knowledge. Although the procedures lay out the individual steps, fine detail on how such steps are to be executed may be missing (e.g., how long to wait between steps or how sharply to increase throttle).
- Aircraft performance characteristics that can be measured through on-board data-acquisition units are generally considered proprietary by the aircraft and avionic manufacturers. This information can be utilized for flight simulations, aircraft health monitoring, aircraft development, and much more.
- third parties wanting to utilize the on-board data acquisition are restricted by its proprietary nature. This restriction has only been partially been overcome through the use of stand-alone aircraft sensor suites. These commercially available sensor suites only measure a fraction of the data available through cockpit instrumentation and pilot inputs.
- the aircrew automation system 100 utilizes a variety of sensors to determine the aircraft flight performance characteristics, it effectively reverse engineers the air vehicle performance characteristics.
- the aircrew automation system 100 collects aircraft information through a combination of stand-alone sensors, data capture through images of cockpit instrument, and input controls.
- Example Aspects of the present disclosure may be illustrated through the following example flight plan, which illustrates how aircrew automation system 100 may interact with the pilot, execute a flight plan, execute flight operational tasks, and respond to contingencies during system engagement and takeoff, flight plan engagement, and anomaly detection & handling.
- the present teachings should not be limited to those used in this example.
- the aircrew automation system 100 may report success to the pilot and report, for example, "ready to taxi,” (either audibly or via the human- machine interface 126).
- the pilot calls for a taxi clearance and upon hearing it, the aircrew automation system 100 transcribes the taxi clearance and displays it to the pilot for confirmation.
- the pilot hits the "taxi via clearance” button on the application and aircrew automation system 100 taxis to the assigned runway while the pilot monitors for traffic.
- the pilot verbally commands the aircrew automation system 100 to perform a pre- takeoff check and the system completes all necessary checks, prompting the pilot to manually double-check critical items, such as flight controls.
- the aircrew automation system 100 may monitor the human operator's execution of a checklist, and output "checklist complete" or identify a flight plan or error.
- the pilot Upon receiving further clearance, the pilot then commands the aircrew automation system 100 to guide the aircraft to line-up and wait, and then ultimately takeoff.
- the aircrew automation system 100 pushes the throttles forward via the primary actuation system 108a, visually checks engine and cockpit indicators via the perception system 106, calls out speeds via the HMI system 104, and rotates at the speed appropriate to the current weight, balance, and density altitude.
- the pilot keeps his hand on the stick/yoke 514 to confirm aircrew automation system's 100 inputs and retain his muscle memory.
- the aircrew automation system 100 confirms aircraft performance according to current conditions and reports any deviation from expected climb rate.
- the pilot's workload is reduced by the aircrew automation system 100 during climb, enabling more heads-up time (i.e., eyes forward, not on the instruments) to look for traffic in the busy airspace.
- the aircrew automation system 100 may also provide experienced pilot advice for a given checklist, aircraft, or location. For example, in a particular airport, the aircrew automation system 100 may instruct the human operator with airport specific tips, such as "steep departure angle from this runway.”
- the aircraft arrives at the training area and begins the day's flight plan. During the flight plan, however, the aircraft enters a towering cumulus cloud, where instrument meteorological conditions ("IMC") conditions are at below freezing temperatures.
- IMC instrument meteorological conditions
- the pilot requests and receives clearance from the ground, via an internet relay chat (“IRC") chat window on the human-machine interface 126, to climb to 24,000 feet to get above the weather.
- IRC internet relay chat
- the aircrew automation system 100 request clearance from the ground.
- the aircrew automation system 100 may detect that given the climb, the indicated airspeed is slowly deviating from its modeled airspeed for these pitch and power settings, indicating lower than expected values. This is an indication that the pitot heater has failed and the pitot tubes have iced up.
- the pilot has fewer than 100 hours flying the aircraft and is not aware that pitot heaters in this model are known to be unreliable. The pilot has not yet noticed that the airspeed indicator is trending below nominal.
- the aircrew automation system 100 recognizes that the airspeed data is anomalous to the rest of the flight data and its internal flight dynamics model, and aurally warns the pilot "airspeed indicator fault.” While the pilot recognizes that the airspeed information is currently unreliable, he is unsure as to whether the aircraft is flying faster or slower than the indicator shows.
- the aircrew automation system 100 presents a set of procedural options and highlights the minimum safe altitude for the area (e.g., 8,000ft). The pilot chooses the most conservative option, which results in wings level, pitch, and power descent to a lower altitude (e.g., 10,000ft). The aircrew automation system 100 eases back on the power, pitches slightly down, and commences the descent. While descending through 15,000 feet the pitot tubes again come online. Once stable at 10,000 feet, the aircrew automation system 100 holds the aircraft straight and level while the pilot assesses the situation prior to returning to the flight plan.
- the aircrew automation system 100 presents a set of procedural options and highlights the minimum safe altitude for the area (e.g., 8,000ft). The pilot chooses the most conservative option, which results in wings level, pitch, and power descent to a lower altitude (e.g., 10,000ft). The aircrew automation system 100 eases back on the power, pitches slightly down, and commences the descent. While descending through 15,000 feet the pit
- the aircrew automation system 100 may execute an automatic landing procedure. For example, the aircrew automation system 100 may navigate the aircraft to a predetermined waypoint, where the aircraft may commence its initial descent. During the descent, the aircrew automation system 100 may monitor the flight conditions and locate the runway. Upon final approach, the aircrew automation system 100 may slow the aircraft down and ultimately land the aircraft. If the aircrew automation system 100 determines that landing is not feasible (e.g., an obstruction or unacceptable flight conditions), the aircrew automation system 100 may commence a missed approach routine or other contingency routine. For example, the aircrew automation system 100 may retry landing at the same location or navigate the aircraft to an alternative landing location. An example system for landing an aircraft at an alternative landing location is disclosed by commonly owned U.S. Patent Publication No. 2015/0323932, titled "Autonomous Cargo Delivery System.”
- the aircrew automation system 100 and derivative technologies may be applied across a wide range of aircraft and flight simulators.
- the derived flight performance characteristics from an aircraft flight test can be used to improve the fidelity of flight simulators used to train pilots. Providing flight simulators access to actual aircraft performance data has tremendous value for flight simulator operators.
- Another benefit of aircrew automation system 100 is its ability to synthesize flight performance characteristics when aircraft are modified for special flight plans such as the addition of sensors and antennas that can affect aerodynamic performance and flight handling qualities (e.g., aircraft development).
- the data captured by the aircrew automation system 100 can be used for aircraft health monitoring, using prognostics to sense maintenance needs.
- the aircrew automation system 100 furthers the safety and utility of commercial aviation operations while providing significant savings in human operating costs.
- the aircrew automation system 100 may be applied to long-haul air cargo carriers to increase safety and efficiency as well the cost-savings of this advanced pilot-assist technology.
- the ultimate state machine for example, may serve as a training tool for pilots in-flight, or as a safety system, providing a second set of eyes in what would traditionally be a single -pilot aircraft.
- Portions of the human-machine interface 126 streamline all piloted flight operations, even multi-crew operations.
Abstract
Description
Claims
Priority Applications (8)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2017238122A AU2017238122B2 (en) | 2016-03-22 | 2017-03-22 | Aircrew automation system and method |
EP21176083.0A EP3889038B1 (en) | 2016-03-22 | 2017-03-22 | Aircrew automation system |
RU2018130606A RU2732646C2 (en) | 2016-03-22 | 2017-03-22 | Flight crew automation system |
EP17771033.2A EP3433167B1 (en) | 2016-03-22 | 2017-03-22 | Aircrew automation system |
CN201780023666.0A CN109070997B (en) | 2016-03-22 | 2017-03-22 | Unit automation system and method |
KR1020187029828A KR102269419B1 (en) | 2016-03-22 | 2017-03-22 | Crew automation systems and methods |
JP2018549976A JP6913106B2 (en) | 2016-03-22 | 2017-03-22 | Flight attendant automation systems and methods |
IL261322A IL261322B (en) | 2016-03-22 | 2018-08-22 | Aircrew automation system and method |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662311659P | 2016-03-22 | 2016-03-22 | |
US62/311,659 | 2016-03-22 | ||
US201662383032P | 2016-09-02 | 2016-09-02 | |
US62/383,032 | 2016-09-02 | ||
US15/464,786 | 2017-03-21 | ||
US15/464,786 US10359779B2 (en) | 2016-03-22 | 2017-03-21 | Aircrew automation system and method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017165481A1 true WO2017165481A1 (en) | 2017-09-28 |
Family
ID=59898475
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2017/023523 WO2017165481A1 (en) | 2016-03-22 | 2017-03-22 | Aircrew automation system and method |
Country Status (9)
Country | Link |
---|---|
US (3) | US10359779B2 (en) |
EP (2) | EP3433167B1 (en) |
JP (1) | JP6913106B2 (en) |
KR (1) | KR102269419B1 (en) |
CN (1) | CN109070997B (en) |
AU (1) | AU2017238122B2 (en) |
IL (1) | IL261322B (en) |
RU (1) | RU2732646C2 (en) |
WO (1) | WO2017165481A1 (en) |
Families Citing this family (50)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9557742B2 (en) | 2013-11-27 | 2017-01-31 | Aurora Flight Sciences Corporation | Autonomous cargo delivery system |
FR3023912B1 (en) * | 2014-07-18 | 2020-05-08 | Thales | PERFORMANCE CALCULATION FOR AIRCRAFT |
DE102015013642A1 (en) * | 2015-10-21 | 2017-04-27 | Mbda Deutschland Gmbh | Safety function module for a vehicle, in particular for a missile |
US10509415B2 (en) * | 2017-07-27 | 2019-12-17 | Aurora Flight Sciences Corporation | Aircrew automation system and method with integrated imaging and force sensing modalities |
US10725091B2 (en) * | 2017-08-28 | 2020-07-28 | Teradyne, Inc. | Automated test system having multiple stages |
US10845410B2 (en) | 2017-08-28 | 2020-11-24 | Teradyne, Inc. | Automated test system having orthogonal robots |
GB2568356B (en) * | 2017-09-07 | 2021-12-15 | Borealis Tech Ltd | Improved aircraft ground collision avoidance system |
US10599138B2 (en) | 2017-09-08 | 2020-03-24 | Aurora Flight Sciences Corporation | Autonomous package delivery system |
US11094419B2 (en) * | 2017-09-12 | 2021-08-17 | Duro Health, LLC | Sensor fusion of physiological and machine-interface factors as a biometric |
US10426393B2 (en) | 2017-09-22 | 2019-10-01 | Aurora Flight Sciences Corporation | Systems and methods for monitoring pilot health |
US20190092337A1 (en) | 2017-09-22 | 2019-03-28 | Aurora Flight Sciences Corporation | System for Monitoring an Operator |
DE102017126560A1 (en) | 2017-11-13 | 2019-05-16 | Airbus Defence and Space GmbH | Test system and robot assembly for performing a test |
EP3489140B1 (en) * | 2017-11-27 | 2022-02-23 | Airbus Operations, S.L. | Aircraft system with asisted taxi, take off, and climbing |
US10759517B2 (en) * | 2017-12-12 | 2020-09-01 | The Boeing Company | System and method for modifying the location of water impingement limits on an airfoil |
CA3088256A1 (en) | 2018-01-23 | 2019-08-01 | Platform Aerospace, Inc. | Modular, palletized system for a deployable sensor |
US10875662B2 (en) * | 2018-04-19 | 2020-12-29 | Aurora Flight Sciences Corporation | Method of robot manipulation in a vibration environment |
CN108945397B (en) * | 2018-04-24 | 2021-05-04 | 中国商用飞机有限责任公司北京民用飞机技术研究中心 | Flight driving system |
US10733551B2 (en) * | 2018-05-01 | 2020-08-04 | Honeywell International Inc. | Systems and methods for providing dynamic voice-prompt dialogue for situational awareness onboard an aircraft |
US11262900B1 (en) | 2018-07-30 | 2022-03-01 | The Boeing Company | Graphical user interface in a computer system in an aircraft |
US11305886B1 (en) * | 2018-07-30 | 2022-04-19 | The Boeing Company | Graphical user interface in a computer system in an aircraft |
US11136120B2 (en) | 2018-10-05 | 2021-10-05 | Aurora Flight Sciences Corporation | Ground operations for autonomous object pickup |
US11417220B2 (en) * | 2018-11-27 | 2022-08-16 | Honeywell International Inc. | Systems and methods for providing an integrated flight management display with interactive time-based functionality |
CN109902413B (en) * | 2019-03-07 | 2022-12-09 | 西安天圆光电科技有限公司 | Trimming method for airborne electromechanical frame products |
US11687067B2 (en) * | 2019-04-04 | 2023-06-27 | Honeywell International Inc. | Pattern classification system with smart data collection for environmental control system fault isolation |
GB2583464A (en) * | 2019-04-18 | 2020-11-04 | Airbus Operations Ltd | System and method for landing gear retraction |
US20200377197A1 (en) * | 2019-05-31 | 2020-12-03 | Hamilton Sundstrand Corporation | System and method for performing device analytics |
US11097425B2 (en) | 2019-09-16 | 2021-08-24 | Lockheed Martin Corporation | Automated switch activation system |
US11186384B2 (en) * | 2019-09-19 | 2021-11-30 | Lockheed Martin Corporation | Adaptive flight display for autonomous aircraft operations |
US20210223773A1 (en) * | 2020-01-16 | 2021-07-22 | Honeywell International Inc. | Methods and systems for remote operation of vehicles using hands-free functionality |
US11586196B2 (en) | 2020-01-22 | 2023-02-21 | Honeywell International Inc. | Approach mode activation monitoring function and automatic activation in an event of pilot incapacitation |
CN111552315B (en) * | 2020-05-11 | 2023-07-18 | 中国商用飞机有限责任公司北京民用飞机技术研究中心 | Flight driving method, device, equipment and storage medium |
US20210358314A1 (en) * | 2020-05-15 | 2021-11-18 | Hrl Laboratories, Llc | Neural network-based system for flight condition analysis and communication |
US11655047B2 (en) | 2020-10-21 | 2023-05-23 | Honeywell International, Inc. | System and method for monitoring an electronic checklist and supporting sensors |
US11953519B2 (en) | 2020-10-22 | 2024-04-09 | Teradyne, Inc. | Modular automated test system |
US11899042B2 (en) | 2020-10-22 | 2024-02-13 | Teradyne, Inc. | Automated test system |
US11754596B2 (en) | 2020-10-22 | 2023-09-12 | Teradyne, Inc. | Test site configuration in an automated test system |
US11754622B2 (en) | 2020-10-22 | 2023-09-12 | Teradyne, Inc. | Thermal control system for an automated test system |
US11867749B2 (en) | 2020-10-22 | 2024-01-09 | Teradyne, Inc. | Vision system for an automated test system |
US11971728B1 (en) | 2020-10-23 | 2024-04-30 | Rockwell Collins, Inc. | Platform guidance system incorporating hierarchical modular arbitration architecture |
RU2751377C1 (en) * | 2020-10-29 | 2021-07-13 | Федеральное государственное унитарное предприятие "Государственный научно-исследовательский институт авиационных систем" (ФГУП "ГосНИИАС") | Method for intelligent support of aircraft crew during flight phase |
KR102517897B1 (en) * | 2020-11-26 | 2023-04-04 | 엘아이지넥스원 주식회사 | Method and Apparatus for Flight Management Based on Flight Data Analysis |
RU2767406C1 (en) * | 2020-12-17 | 2022-03-17 | Фонд содействия развитию науки, инноваций и технологий | Method for intellectual support of the crew |
CN113119085B (en) * | 2021-03-24 | 2022-04-19 | 北京航空航天大学 | Helicopter flight driving robot system |
CN113358337B (en) * | 2021-05-27 | 2022-05-10 | 燕山大学 | Loading method and loading device for aircraft wing static strength experiment |
US11427305B1 (en) * | 2021-09-16 | 2022-08-30 | Beta Air, Llc | Methods and systems for flight control for managing actuators for an electric aircraft |
USD953371S1 (en) | 2021-10-22 | 2022-05-31 | BioReference Health, LLC | Display screen or portion thereof with animated graphical user interface |
CN114035425B (en) | 2021-11-05 | 2022-07-26 | 广东工业大学 | Virtual and real state synchronization method and system for unmanned aerial vehicle digital twin system |
US20230260337A1 (en) * | 2022-02-17 | 2023-08-17 | Ge Aviation Systems Llc | Configurable status console within an aircraft environment and method |
CN115236998B (en) * | 2022-06-21 | 2023-07-14 | 江苏锐天智能科技股份有限公司 | System and method for monitoring fuel oil consumption of aircraft in simulated flight |
US20230419271A1 (en) * | 2022-06-24 | 2023-12-28 | Gm Cruise Holdings Llc | Routing field support to vehicles for maintenance |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020004695A1 (en) * | 2000-02-03 | 2002-01-10 | Glenn Matthew H. | Event based aircraft image and data recording system |
US20110149067A1 (en) | 2007-08-17 | 2011-06-23 | George Steven Lewis | System for Optical Recognition, Interpretation, and Digitization of Human Readable Instruments, Annunciators, and Controls |
EP2251851B1 (en) * | 2009-05-14 | 2012-03-14 | Honeywell International Inc. | Aircraft clearance enforcement |
US20150094982A1 (en) * | 2012-03-28 | 2015-04-02 | Dassault Aviation | Method for determining the estimated weight of an aircraft and corresponding system |
US20160019793A1 (en) * | 2014-07-18 | 2016-01-21 | Thales | Processing of the data of a flight plan |
WO2016035002A1 (en) | 2014-09-03 | 2016-03-10 | University Of Malta | A human machine interface device for aircraft |
Family Cites Families (81)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB897627A (en) * | 1958-06-23 | 1962-05-30 | Sperry Gyroscope Co Ltd | Control apparatus for aircraft |
US3510090A (en) * | 1967-09-08 | 1970-05-05 | Honeywell Inc | Automatic altitude control apparatus for aircraft |
US3848833A (en) * | 1972-07-14 | 1974-11-19 | Sperry Rand Corp | Aircraft automatic flight control system |
US4063073A (en) | 1974-11-29 | 1977-12-13 | Strayer Larry G | Computer system to prevent collision between moving objects such as aircraft moving from one sector to another |
CA1050633A (en) * | 1974-12-19 | 1979-03-13 | Ronald J. Miller | Semi-automatic flight control system utilizing limited authority stabilization system |
US4490794A (en) * | 1982-01-18 | 1984-12-25 | Sperry Corporation | Altitude preselect system for aircraft without air data computer |
US5157615A (en) | 1990-01-09 | 1992-10-20 | Ryan International Corporation | Aircraft traffic alert and collision avoidance device |
JPH07115677B2 (en) | 1990-10-30 | 1995-12-13 | 嘉三 藤本 | Flight information recording method and apparatus for aircraft |
RU2128854C1 (en) * | 1996-08-30 | 1999-04-10 | Летно-исследовательский институт им.М.М.Громова | System of crew support in risky situations |
US7131136B2 (en) * | 2002-07-10 | 2006-10-31 | E-Watch, Inc. | Comprehensive multi-media surveillance and response system for aircraft, operations centers, airports and other commercial transports, centers and terminals |
US6480152B2 (en) | 2000-07-20 | 2002-11-12 | American Gnc Corporation | Integrated GPS/IMU method and microsystem thereof |
FR2821452B1 (en) | 2001-02-26 | 2003-06-13 | Eads Airbus Sa | DEVICE FOR MONITORING A PLURALITY OF AIRCRAFT SYSTEMS, PARTICULARLY A TRANSPORT AIRCRAFT |
US20030065428A1 (en) * | 2001-10-01 | 2003-04-03 | Ehud Mendelson | Integrated aircraft early warning system, method for analyzing early warning data, and method for providing early warnings |
US6604044B1 (en) | 2002-02-14 | 2003-08-05 | The Mitre Corporation | Method for generating conflict resolutions for air traffic control of free flight operations |
US6820006B2 (en) | 2002-07-30 | 2004-11-16 | The Aerospace Corporation | Vehicular trajectory collision conflict prediction method |
US6885917B2 (en) * | 2002-11-07 | 2005-04-26 | The Boeing Company | Enhanced flight control systems and methods for a jet powered tri-mode aircraft |
ES2375957T3 (en) | 2002-11-11 | 2012-03-07 | Aeromechanical Services Ltd | AIRCRAFT FLIGHT DATA MANAGEMENT SYSTEM AND CORRESPONDING METHOD. |
US7106219B2 (en) | 2003-11-07 | 2006-09-12 | Pearce James W | Decentralized vehicular traffic status system |
US7460029B2 (en) * | 2003-12-24 | 2008-12-02 | The Boeing Company | Systems and methods for presenting and obtaining flight control information |
US7193729B1 (en) | 2004-04-29 | 2007-03-20 | Yazaki North America, Inc | Instrument cluster with laser beam illumination |
RU2263973C1 (en) * | 2004-05-07 | 2005-11-10 | Федеральное государственное унитарное предприятие Летно-исследовательский институт им. М.М. Громова | Pilotage-and-training complex |
WO2006011141A2 (en) * | 2004-07-25 | 2006-02-02 | Israel Aerospace Industries Ltd. | Method and system for the acquisition of data and for the display of data |
US7437220B2 (en) | 2004-08-17 | 2008-10-14 | Arinc, Inc. | Systems and methods for crew interaction and coordination using portable electronic data storage and display devices |
JP2006151125A (en) | 2004-11-26 | 2006-06-15 | Omron Corp | On-vehicle image processing device |
US7789349B2 (en) * | 2005-05-12 | 2010-09-07 | Aspen Avionics, Inc. | Aircraft vertical speed instrument device with multi-functional design aspects |
US7848698B2 (en) | 2005-07-22 | 2010-12-07 | Appareo Systems Llc | Flight training and synthetic flight simulation system and method |
US7650232B1 (en) | 2005-09-22 | 2010-01-19 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration (Nasa) | Trajectory specification for high capacity air traffic control |
FR2898673B1 (en) * | 2006-03-14 | 2009-04-17 | Thales Sa | METHOD FOR AIDING NAVIGATION OF AN AIRCRAFT WITH FLIGHT PLAN UPDATE |
US8050863B2 (en) | 2006-03-16 | 2011-11-01 | Gray & Company, Inc. | Navigation and control system for autonomous vehicles |
US7624943B2 (en) | 2006-03-22 | 2009-12-01 | The Boeing Company | Multi-mode unmanned and manned vehicle systems and methods |
US7885732B2 (en) * | 2006-10-25 | 2011-02-08 | The Boeing Company | Systems and methods for haptics-enabled teleoperation of vehicles and other devices |
US8013780B2 (en) | 2007-01-25 | 2011-09-06 | Magna Electronics Inc. | Radar sensing system for vehicle |
WO2008139530A1 (en) | 2007-04-27 | 2008-11-20 | Honda Motor Co., Ltd. | Vehicle periphery monitoring device, vehicle periphery monitoring program, and vehicle periphery monitoring method |
US8049658B1 (en) | 2007-05-25 | 2011-11-01 | Lockheed Martin Corporation | Determination of the three-dimensional location of a target viewed by a camera |
US20080316010A1 (en) | 2007-06-23 | 2008-12-25 | Appro Technology Inc. | Recording system and method for capturing images of driving conditions and driving images identification method |
US8255098B2 (en) * | 2007-10-17 | 2012-08-28 | The Boeing Company | Variably manned aircraft |
US8290638B2 (en) | 2008-02-04 | 2012-10-16 | Lockheed Martin Corporation | Apparatus, program product, and methods for updating data on embedded control systems |
US8063798B2 (en) * | 2008-06-03 | 2011-11-22 | Honeywell International Inc. | Methods and apparatus to assist pilots under conditions in which spatial disorientation may be present |
AT507035B1 (en) | 2008-07-15 | 2020-07-15 | Airbus Defence & Space Gmbh | SYSTEM AND METHOD FOR AVOIDING COLLISION |
DE102008036759A1 (en) * | 2008-08-07 | 2010-02-11 | Esg Elektroniksystem- Und Logistik-Gmbh | Apparatus and method for testing systems with visual output |
JP4434296B1 (en) | 2008-09-05 | 2010-03-17 | トヨタ自動車株式会社 | Object detection device |
US8702033B2 (en) * | 2009-01-06 | 2014-04-22 | Ruchit Kumar Regmi | Pilotless aircraft for commercial and military use |
US8195346B1 (en) * | 2009-01-21 | 2012-06-05 | Garmin International, Inc. | Envelope protection for mechanically-controlled aircraft |
FR2941912B1 (en) * | 2009-02-10 | 2011-02-18 | Airbus France | FLIGHT CONTROL SYSTEM AND AIRCRAFT COMPRISING SAME |
US8319665B2 (en) | 2009-02-20 | 2012-11-27 | Appareo Systems, Llc | Adaptive instrument and operator control recognition |
US8319666B2 (en) | 2009-02-20 | 2012-11-27 | Appareo Systems, Llc | Optical image monitoring system and method for vehicles |
US7954965B1 (en) | 2009-04-13 | 2011-06-07 | Yazaki North America, Inc. | Method for multiple gauges in a scanning laser based display device |
US8026827B1 (en) | 2009-05-04 | 2011-09-27 | Yazaki North America, Inc. | Virtual push button switch |
KR101043450B1 (en) | 2009-07-31 | 2011-06-21 | 삼성전기주식회사 | Location and distance mesuring appratus its method usnig camera |
US8306672B2 (en) | 2009-09-09 | 2012-11-06 | GM Global Technology Operations LLC | Vehicular terrain detection system and method |
US8725402B2 (en) | 2009-11-13 | 2014-05-13 | The Boeing Company | Loss of separation avoidance maneuvering |
US9099009B2 (en) | 2009-12-01 | 2015-08-04 | The Boeing Company | Performance-based simulation system for an aircraft |
US8616884B1 (en) | 2009-12-01 | 2013-12-31 | The Boeing Company | Integrated live and simulation environment system for an aircraft |
JP5489708B2 (en) * | 2009-12-28 | 2014-05-14 | ナブテスコ株式会社 | Actuator control system |
US8219267B2 (en) * | 2010-05-27 | 2012-07-10 | Honeywell International Inc. | Wind estimation for an unmanned aerial vehicle |
FR2963119B1 (en) * | 2010-07-20 | 2015-05-01 | Airbus Operations Sas | METHOD AND DEVICE FOR RECALING TARGET ALTITUDE FOR EMERGENCY DESCENT OF AN AIRCRAFT |
US8616883B2 (en) | 2010-12-15 | 2013-12-31 | The Boeing Company | Simulation control system for an integrated live and simulation environment for an aircraft |
FR2972712B1 (en) * | 2011-03-16 | 2014-09-05 | Airbus Operations Sas | MONITORING A FLIGHT CONTROL ACTUATOR OF AN AIRCRAFT |
US9087450B2 (en) * | 2011-05-17 | 2015-07-21 | Innovative Solutions And Support, Inc. | Upgraded flight management system and method of providing the same |
JP2013002927A (en) | 2011-06-15 | 2013-01-07 | Honda Elesys Co Ltd | Obstacle detection apparatus and computer program |
US9424753B2 (en) * | 2011-07-08 | 2016-08-23 | General Electric Company | Simplified user interface for an aircraft |
FR2978736B1 (en) * | 2011-08-01 | 2013-09-27 | Airbus Operations Sas | DEVICE AND METHOD FOR DETERMINING A TRACK STATE, AIRCRAFT COMPRISING SUCH A DEVICE AND A PILOTAGE ASSISTANCE SYSTEM UTILIZING THE TRACK STATE |
US8768534B2 (en) | 2011-11-14 | 2014-07-01 | Arinc Incorporated | Method and apparatus for using electronic flight bag (EFB) to enable flight operations quality assurance (FOQA) |
FR2989500B1 (en) * | 2012-04-12 | 2014-05-23 | Airbus Operations Sas | METHOD, DEVICES AND COMPUTER PROGRAM FOR AIDING THE TROUBLE TOLERANCE ANALYSIS OF AN AIRCRAFT SYSTEM USING REDUCED EVENT GRAPHICS |
US10399695B2 (en) * | 2012-08-07 | 2019-09-03 | C Series Aircraft Limited Partnership | Checklist display system, method and graphical display therefor |
US9002543B2 (en) * | 2012-12-20 | 2015-04-07 | Honeywell International Inc. | Situation aftermath management system and method |
US9052393B2 (en) | 2013-01-18 | 2015-06-09 | Caterpillar Inc. | Object recognition system having radar and camera input |
US9070283B2 (en) * | 2013-01-29 | 2015-06-30 | Honeywell International Inc. | Flight deck display systems and methods for generating in-trail procedure windows including aircraft flight path symbology |
US9014879B2 (en) * | 2013-09-17 | 2015-04-21 | Honeywell International Inc. | Messaging and data entry validation system and method for aircraft |
US9557742B2 (en) | 2013-11-27 | 2017-01-31 | Aurora Flight Sciences Corporation | Autonomous cargo delivery system |
CN105848999A (en) * | 2013-12-26 | 2016-08-10 | 空中客车公司 | Aircraft part with robot arm |
US9260199B2 (en) * | 2014-02-25 | 2016-02-16 | Honeywell International Inc. | Aircraft data processing and transmission system |
US9205931B1 (en) * | 2014-07-16 | 2015-12-08 | Honeywell International Inc. | System and method for increasing pilot awareness during non-normal flight situations |
US9840007B1 (en) * | 2014-08-25 | 2017-12-12 | X Development Llc | Robotic operation libraries |
US9701398B2 (en) * | 2014-11-03 | 2017-07-11 | Douglas Allen SCHULTZ | Method and apparatus for augmented pilot operations of fly-by-wire vehicles |
FR3030448B1 (en) * | 2014-12-18 | 2017-01-06 | Airbus Helicopters | METHOD FOR MANAGING DISCONTINUITIES IN A VEHICLE CONTROL AFTER A CONTROL TRANSITION, AND VEHICLE |
US9764852B2 (en) * | 2015-03-13 | 2017-09-19 | Honeywell International Inc. | Methods and systems for integrating auto pilot functions on a display |
US20160275802A1 (en) | 2015-03-20 | 2016-09-22 | Northrop Grumman Systems Corporation | Unmanned aircraft detection and targeting of other aircraft for collision avoidance |
US20170084183A1 (en) | 2015-09-18 | 2017-03-23 | Robert M. Knox | Automatic aircraft separation assurance |
US9542851B1 (en) * | 2015-11-03 | 2017-01-10 | The Boeing Company | Avionics flight management recommender system |
US20190369615A1 (en) * | 2018-06-05 | 2019-12-05 | Facultad Politecnica-Universidad Nacional del Este | System and method for verifying the presence and vital signs of pilots in a cockpit of an airplane |
-
2017
- 2017-03-21 US US15/464,786 patent/US10359779B2/en active Active
- 2017-03-22 WO PCT/US2017/023523 patent/WO2017165481A1/en active Application Filing
- 2017-03-22 CN CN201780023666.0A patent/CN109070997B/en active Active
- 2017-03-22 JP JP2018549976A patent/JP6913106B2/en active Active
- 2017-03-22 EP EP17771033.2A patent/EP3433167B1/en active Active
- 2017-03-22 RU RU2018130606A patent/RU2732646C2/en active
- 2017-03-22 EP EP21176083.0A patent/EP3889038B1/en active Active
- 2017-03-22 KR KR1020187029828A patent/KR102269419B1/en active IP Right Grant
- 2017-03-22 AU AU2017238122A patent/AU2017238122B2/en active Active
-
2018
- 2018-08-22 IL IL261322A patent/IL261322B/en unknown
-
2019
- 2019-07-08 US US16/504,988 patent/US10642270B2/en active Active
-
2020
- 2020-05-04 US US16/866,458 patent/US11693407B2/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20020004695A1 (en) * | 2000-02-03 | 2002-01-10 | Glenn Matthew H. | Event based aircraft image and data recording system |
US20110149067A1 (en) | 2007-08-17 | 2011-06-23 | George Steven Lewis | System for Optical Recognition, Interpretation, and Digitization of Human Readable Instruments, Annunciators, and Controls |
EP2251851B1 (en) * | 2009-05-14 | 2012-03-14 | Honeywell International Inc. | Aircraft clearance enforcement |
US20150094982A1 (en) * | 2012-03-28 | 2015-04-02 | Dassault Aviation | Method for determining the estimated weight of an aircraft and corresponding system |
US20160019793A1 (en) * | 2014-07-18 | 2016-01-21 | Thales | Processing of the data of a flight plan |
WO2016035002A1 (en) | 2014-09-03 | 2016-03-10 | University Of Malta | A human machine interface device for aircraft |
Non-Patent Citations (1)
Title |
---|
See also references of EP3433167A4 |
Also Published As
Publication number | Publication date |
---|---|
JP2019513610A (en) | 2019-05-30 |
RU2018130606A3 (en) | 2020-07-13 |
RU2018130606A (en) | 2020-04-22 |
EP3433167A4 (en) | 2019-11-20 |
US10359779B2 (en) | 2019-07-23 |
EP3889038A1 (en) | 2021-10-06 |
IL261322B (en) | 2022-05-01 |
US20200301422A1 (en) | 2020-09-24 |
EP3889038B1 (en) | 2023-07-05 |
EP3433167A1 (en) | 2019-01-30 |
RU2732646C2 (en) | 2020-09-21 |
US20200064840A1 (en) | 2020-02-27 |
CN109070997B (en) | 2022-07-22 |
US11693407B2 (en) | 2023-07-04 |
US10642270B2 (en) | 2020-05-05 |
CN109070997A (en) | 2018-12-21 |
AU2017238122A1 (en) | 2018-09-06 |
AU2017238122B2 (en) | 2021-10-28 |
US20170277185A1 (en) | 2017-09-28 |
IL261322A (en) | 2018-10-31 |
EP3433167B1 (en) | 2021-10-13 |
JP6913106B2 (en) | 2021-08-04 |
KR20180118793A (en) | 2018-10-31 |
KR102269419B1 (en) | 2021-06-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11693407B2 (en) | Aircrew automation system and method | |
US11378988B2 (en) | Aircrew automation system and method with integrated imaging and force sensing modalities | |
EP3415416B1 (en) | Aircrew automation system | |
US11181935B2 (en) | System and method for detecting obstacles in aerial systems | |
EP3637212B1 (en) | Adaptable vehicle monitoring system and method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 261322 Country of ref document: IL |
|
ENP | Entry into the national phase |
Ref document number: 2017238122 Country of ref document: AU Date of ref document: 20170322 Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2018549976 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 20187029828 Country of ref document: KR Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2017771033 Country of ref document: EP |
|
ENP | Entry into the national phase |
Ref document number: 2017771033 Country of ref document: EP Effective date: 20181022 |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17771033 Country of ref document: EP Kind code of ref document: A1 |