Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
  • G06N20/20—Ensemble learning
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B64—AIRCRAFT; AVIATION; COSMONAUTICS
  • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
  • B64D47/00—Equipment not otherwise provided for
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
  • G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
  • G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
  • G05B13/0285—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks and fuzzy logic
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/043—Architecture, e.g. interconnection topology based on fuzzy logic, fuzzy membership or fuzzy inference, e.g. adaptive neuro-fuzzy inference systems [ANFIS]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/045—Combinations of networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/0464—Convolutional networks [CNN, ConvNet]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/0475—Generative networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/086—Learning methods using evolutionary algorithms, e.g. genetic algorithms or genetic programming
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/09—Supervised learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/092—Reinforcement learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/094—Adversarial learning
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/12—Computing arrangements based on biological models using genetic models
  • G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N5/00—Computing arrangements using knowledge-based models
  • G06N5/01—Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F40/00—Handling natural language data
  • G06F40/30—Semantic analysis
  • G06F40/35—Discourse or dialogue representation
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
  • G06N20/10—Machine learning using kernel methods, e.g. support vector machines [SVM]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/044—Recurrent networks, e.g. Hopfield networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/047—Probabilistic or stochastic networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods
  • G06N3/084—Backpropagation, e.g. using gradient descent
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N5/00—Computing arrangements using knowledge-based models
  • G06N5/04—Inference or reasoning models
  • G06N5/045—Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N5/00—Computing arrangements using knowledge-based models
  • G06N5/04—Inference or reasoning models
  • G06N5/048—Fuzzy inferencing
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N7/00—Computing arrangements based on specific mathematical models
  • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks

Definitions

• the invention relates to the technical field of man-machine interfaces, and more particularly describes examples of man-machine dialogue systems and methods.
• the risk can be quantified or objectified (e.g. from measurements), for example positioned on a graphic scale.
• the concept of risk can indeed be based on more quantifiable and measurable pillars or factors. In particular, these factors may correspond to intentions for action, or to concerns induced by the context (absence or limit of mastery of constraints).
• the document describes systems and methods for two-way man-machine communication, in particular capable of capturing or receiving or otherwise retrieving the intentions (of action) of the pilot and of translating them into a language understandable by the system (then called " clever ").
• the systems and methods according to the invention make the solutions calculated by the system understandable to humans. As interactions arise, trust in the communication system can grow.
• systems and methods according to the invention are advantageous for the man-machine dialogue can in particular reduce the semantic difference between the level of abstraction of the operator (called high level) and that of the system (called low level).
• a bidirectional communicator for optimizing a dialogue or an interaction (succession of actions and display of information from computers in particular from sensors) between an operator and a complex system .
• the use of one or more universal "approximators" can be advantageous for performing the upward and downward translation functions.
• the adjustment of the upward translator is carried out by supervised learning on the basis of collection of operational knowledge.
• the adjustment of the descending translator can for its part be carried out by reinforced learning using the complex system and the ascending translator previously adjusted.
• the systems and methods for two-way human-machine communication can integrate and use a plurality of translators (eg ascending and / or descending, congruent or competing, arranged in series and / or in parallel. ) between the operator and the complex system.
• a plurality of translators eg ascending and / or descending, congruent or competing, arranged in series and / or in parallel.
• These translators can in particular be used independently (for example according to the operational need, e.g. control of the downward translator, control of the upward translator for an explicability of the response of the system).
• the advanced systems and methods described in this document can be advantageous in many technical fields requiring interactions between humans and machines (eg construction, public works, medicine, economic decision support systems, etc.).
• the adaptation of the systems and methods according to the invention to a new technical field could for example make use of new HLOM (high level abstraction metrics eg intentions) and LLOM (low level operational metrics).
• the methods and systems according to the invention can be used to continuously monitor the synergy of the human / machine pair.
• the methods and systems according to the invention can relate to different phases of preparation, modification, evaluation or progress of the mission, during which the level of complexity is high.
• the methods and systems according to the invention can implement natural interactions, less costly in cognitive resources, and capable of establishing a good level of trust between man and machine.
• the methods and systems according to the invention can be used in any other field for the management of complex systems, for example in particular for the control of autonomous cars, the management of sensor services in airborne surveillance, interactions with a virtual assistant, etc.
• the methods and systems according to the invention can allow the pilot not to exceed the level of risk imposed by the hierarchy (e.g. flight safety and operational risk management).
• the methods and systems according to the invention can allow the pilot to detect any discrepancies between his own mental representation of the situation (his intentions) and that of his digital partner.
• fuzzy logic makes it possible to manipulate data at a high level of abstraction.
• decision trees makes it possible to evaluate and understand the propositions calculated by the machines.
• FIG. 1 illustrates the state of the art of man-machine dialogue
• FIG. 3 illustrates an example of a fuzzy logic decision tree used in one embodiment of the invention
• FIG. 4 illustrates a possible generalization of the invention.
• an "aircraft” can be a drone, or a commercial aircraft, or a freight aircraft, or even a helicopter, whether or not carrying passengers, or any element being susceptible to be remotely controlled (by radio link, satellite, or other), at least partially (intermittently, or periodically, or even opportunistically over time).
• the embodiments of the systems and methods according to the invention aim to provide the pilot with a communication tool making it possible to transmit high-level instructions to a complex system (downward direction, or first direction) and vice versa ( ascending sense, or second sense according to a man-machine axiology), to present in an understandable way the proposed mode of operation.
• the levels of abstraction manipulated by the invention can be various. By convention, we will count N levels of abstraction (cascade or nesting of terms).
• N 5.
• Rasmussen's work in the field of human-machine interaction is known in design science. These works make it possible to classify an object or a function according to a hierarchy established on 5 levels of abstraction. The lowest of these levels designates that of physical forms (technical / design solution). The intermediate levels relate to physical processes (interaction / information processing), general functions and abstract functions (diagrams or modes of operation). Finally, the highest level of abstraction is that of functional goals and intentions. A functional objective is what the information is designed for (eg “I want to detect and protect myself at the same time”).
• An abstract function denotes known or envisaged procedures and schemes (eg electromagnetic waves from radar and jammer interfering with each other).
• a general function refers to information that relies on the technical potential of an object (for example, I want a function that optimizes radar detection and interference by tradeoff).
• a physical process designates a type of interaction and information processing (depending on the context I want to choose between a single radar mode or a single jamming mode is a compromise of the two).
• a physical form relates to the form of information (symbology).
• the method manipulates HLOM, LLOM, MLOM and LLTP data, acronyms which will be developed later.
• a "universal approximator” generally designates a neural network.
• a neural network is indeed capable of imitating practically any process, after adjusting its parameters by learning.
• any sufficiently regular bounded function can be approximated with an arbitrary precision, in a finite domain of the space of its variables, by a neural network comprising a layer of hidden neurons in finite number, all possessing the same function d activation, and a linear output neuron.
• This property is not specific to neural networks: many other families of parameterized functions have this property.
• the specificity of neural networks lies in the “parsimonious” nature of the approximation: for equal precision, neural networks require fewer adjustable parameters than other known approximators.
• the method can comprise one or more algorithms (eg put in competition) among the algorithms comprising: “support vector machines” or “wide margin separators” (in English “Support Vector Machine”, acronym SVM); “boosting"(classifiers); neural networks (in unsupervised learning); decision trees ("Random Forest"), statistical methods such as the mixture model Gaussian; logistic regression; linear discriminant analysis; and genetic algorithms.
• algorithms eg put in competition among the algorithms comprising: “support vector machines” or “wide margin separators” (in English “Support Vector Machine”, acronym SVM); “boosting”(classifiers); neural networks (in unsupervised learning); decision trees (“Random Forest”), statistical methods such as the mixture model Gaussian; logistic regression; linear discriminant analysis; and genetic algorithms.
• Machine learning tasks are generally classified into two broad categories, depending on whether there is a “signal” or learning inputs or “feedback” or “available outputs”.
• supervised learning denotes a situation in which the computer is presented with examples of inputs and examples of outputs (actual or desired). Learning then consists of the identification of an interlacing of rules matching the inputs to the outputs (these rules may or may not be understandable to humans).
• force learning consists in learning the actions to be taken, from experiences, in order to optimize a quantitative reward over time.
• a decision-making behavior (called strategy or policy, which is a function associating with the current state the action to be executed) is determined as being optimal, in that it maximizes the sum of the rewards during time.
• unsupervised learning also called deep learning or deep learning denotes a situation in which no annotation exists (no label, no description, etc.), leaving the learning algorithm alone. to find one or more structures, between inputs and outputs.
• Unsupervised learning can be a goal in itself (uncovering hidden structures in data) or a means to a goal (learning through functionality).
• the human contribution in the machine learning steps can vary.
• machine learning is applied to machine learning itself (reflective).
• the whole learning process can in fact be automated, in particular by using several models and by comparing the results produced by these models.
• humans participate in learning automatic ("Human in the loop"). Developers or curators are responsible for maintaining data clusters: data ingestion, data cleaning, model discovery, etc.
• Machine learning can correspond to hardware architectures which are emulable by computer (e.g. CPU-GPU), but sometimes not (circuits dedicated to learning may exist).
• the method may comprise one or more algorithms from among the algorithms comprising: “support vector machines” or “wide margin separators” (in English “Support Vector Machine”, acronym SVM); “boosting” (classifiers); neural networks (in unsupervised learning); decision trees ("Random Forest"), statistical methods such as the Gaussian mixture model; logistic regression; linear discriminant analysis; and genetic algorithms.
• a neural network according to the invention can be one or more neural networks chosen from neural networks comprising: a) an artificial neural network (in English “feedforward neural network”; b) an acyclic artificial neural network, eg a multi-layered perceptron, thus distinguishing itself from recurrent neural networks; c) a forward propagation neural network; d) a Hopfield neural network (a discrete-time recurrent neural network model whose connection matrix is symmetrical and zero on the diagonal and where the dynamics are asynchronous, a single neuron being updated at each unit of time ); e) a recurrent neural network (made up of interconnected units interacting non-linearly and for which there is at least one cycle in the structure); f) a convolutional neural network (in English "CNN” or “ConvNet” for "Convolutional Neural Networks", a type of feedforward acyclic artificial neural network, by multi
• Fuzzy logic Fuzzy logic is a versatile logic where the truth values of the variables - instead of being true or false - are reals between 0 and 1. In this sense, it extends classical Boolean logic with partial truth values.
• the degree of truth of a fuzzy relation between two or N objects is the degree of belonging of the pair or of the N-uple to the fuzzy set associated with the relation.
• fuzzy logic brings with it the use of words from a dictionary, which can generally be expressed in natural language (advantageous interface in man-machine interfaces)
• FIS Fuzzy Inference System
• Fuzzy inference systems take the inputs and process them according to predefined rules to produce the outputs. Both inputs and outputs have real value, while internal processing is based on fuzzy rules and fuzzy arithmetic.
• An FIS generally uses six steps (determining fuzzy rules, blurring entries by the membership classes, combining fuzzy entries according to these fuzzy rules, determining one or more consequences of the application of the rules, combining the consequences in an output distribution, making said distribution non-fuzzy).
• the method according to the invention can use an ANFIS system, in addition to or in substitution for fuzzy decision trees.
• An adaptive fuzzy inference system or adaptive network based fuzzy inference system is a type of artificial neural network which is based on the Takagi-Sugeno fuzzy inference system. The technique was developed in the early 1990s. Integrating both neural networks and logic fuzzy, it captures the benefits of both approaches in a single frame. Its inference system corresponds to a set of fuzzy IF-THEN rules which have a learning capacity to approach nonlinear functions. An ANFIS is therefore considered to be a universal approximator. To use ANFIS more efficiently and optimally, it is advantageous to use the best parameters obtained by genetic algorithmy.
• a genetic algorithm belongs to the family of evolutionary algorithms, the goal of which is to obtain an approximate solution to an optimization problem when there is no exact method (or the solution is unknown) for it. resolve in a reasonable time. Genetic algorithms take the notion of natural selection and apply it to a population of potential solutions to the given problem.
• a genetic algorithm generally comprises at least three steps (selection, evaluation, generation by selection and / or mutation).
• a base population is determined (e.g. generated or selected).
• the N sets of parameters also called individuals
• the N sets of parameters are initialized, for example to a random value.
• the starting population can also be received from a third party system, or result from a selection from a set of individuals.
• the initialization of the genetic loop can take as input the output of this same genetic loop ".
• a second step the population is evaluated.
• a large number of trajectory calculations are carried out (in general several million or even billions, the larger the number the more the result obtained is accurate but the longer the simulation time is) to each of the N sets of parameters then each of these sets is evaluated with the evaluation function f.
• the best individuals are selected for the next iteration. For example, we can retain the 20% of individuals with the best evaluation. We can also decide to keep 5% of the rest of the population at random, in order to maintain diversity. Different selection modes are possible (eg thresholding, threshold ranges, analytical functions, etc.).
• the k samples with the best score are selected, with k being received or calculated.
• k is predefined.
• only the samples having a score greater than a predefined threshold are kept (for example the k samples having a score greater than 70%, if the score is between 0 and 100%).
• one or more of the thresholds used depend on the iteration of the algorithm (increasing selectivity).
• the selection is by a "biased wheel" method: samples are selected in proportion to their score (more samples with a high score are selected than with a low score).
• the B best scores are kept in the office. All these embodiments can be combined with one another (parametric selection, selection by analytical functions in an algorithmic manner).
• crossover (s) / mutation (s) is carried out.
• This complement can be achieved in different ways.
• Crossbreeding involves creating new individuals from other individuals. For example for two individuals with 4 parameters or "genes" [a, b, c, d] and [a ', b', c ', d'] we can obtain [a, b, c ', d'] and [a ', b', c, d].
• the crosses can for example be done in pairs (eg by randomly choosing two samples ("father", "mother”) which are crossed to obtain two new samples ("children").
• the crossing method is a multiple point crossing. In one embodiment, the crossing method is a one point crossing.
• a mutation consists in randomly modifying an individual of the population, for example by modifying the value of one of the parameters of an individual to a new random value in the defined domain. For example for an individual with 4 genes or parameters [a, b, c, d] becomes [a, e, c, d]. Mutations can be made by random selection d one gene and replacement with another gene For example, in one embodiment, the mutation rate can be set at m% (between 0.01 and 2%), and the gene change follows a uniform law.
• a subsequent step it is determined whether the loop is continued (eg looped back, reiterated), or else stopped. At this stage, it is a question of deciding if an optimum has been found and in this case considering the individual with the best function. evaluation as the optimum set of parameters. Otherwise, calculations continue. In general, an optimum is found if the average of the evaluation function of all individuals is close to the best evaluation function of an individual. An alternative may be to simply consider a given value from which the population score is deemed satisfactory. The best set of parameters is therefore determined.
• the best set of parameters is validated (or not) and the performance is quantified.
• we have an optimal set of parameters best individual found in the 5th sub-step.
• machine learning includes a GFT fuzzy logic genetic decision tree.
• machine learning comprises the implementation of a genetic algorithm (2211), which generates the adjustment of the GFTs used in the up and down translators, namely the adjustment of the membership functions and of the fuzzy rule bases of each FIS making up the GFT, by breaking down membership functions and rule bases into a plurality of associated genes, then mixing them randomly and / or randomly replacing one or more genes with others.
• a genetic algorithm (2211) which generates the adjustment of the GFTs used in the up and down translators, namely the adjustment of the membership functions and of the fuzzy rule bases of each FIS making up the GFT, by breaking down membership functions and rule bases into a plurality of associated genes, then mixing them randomly and / or randomly replacing one or more genes with others.
• a genetic algorithm (2211)
• fuzzy logic in an FIS (Fuzzy Inference System) makes it possible to develop control orders as a function of inputs. These orders are developed by “linear” laws (a minimum), depending on “control parameters”.
• the step of determining orders for the FMS / AP can be guided or governed or framed by a fuzzy logic control assembly.
• machine learning can make it possible to best adjust (or optimize) the control parameters of the FIS, so that the mission success score is as high as possible (calculated by a fitness function).
• the FIS or GFT (FIS organized in a tree) intervene in certain aspects as would a pilot knowing his aircraft: the pilot ends up knowing the orders to be developed in order to have an optimal result with regard to the mission.
• a tree set of FIS can therefore be used to develop all of the parameters associated with the unit elements.
• control algorithm can be based on a neural network.
• learning is carried out by gradient backpropagation, so as to optimize the weightings of the different neurons involved.
• the output of each neuron in the terminal layer can correspond respectively to the orders of orientation and of speed of each unit element (like what is proposed for the algorithm of type GFT).
• a black box between one or more inputs and one or more outputs, designates a process for which access to intermediate data and / or calculation rules is not possible, or, where appropriate, cannot be. directly intelligible to man.
• a white box designates a process for which the intermediate data and / or calculation rules are accessible and directly intelligible to humans.
• the method uses one or more genetic algorithms.
• a genetic algorithm iteratively manipulates a set of vectors of real variables using mutation, selection and crossing operators.
• a genetic algorithm iteratively manipulates a set of vectors of real variables using operators.
• a mutation step is performed by adding a random value drawn from within a distribution which is generally normal. Selection is carried out by a deterministic choice of the best individuals, or by recombination, according to the value scale of an objective function. The use of a recombination operator generally makes it possible to avoid being trapped in local optima.
• Decision tree learning refers to a method based on the use of a decision tree as a predictive model.
• a decision tree can be used to explicitly represent the decisions made and the processes that lead to them. It is a supervised learning technique: we use a set of data for which we know the value of the target variable in order to build the tree (so-called labeled data), then we extrapolate the results to the set of data test.
• the tree is generally constructed by separating all of the data into subsets as a function of the value of an input characteristic. This process is repeated on each subset obtained recursively, so it is a recursive partitioning.
• Certain techniques improve the quality or reliability of the prediction by building several decision trees from the data (eg bagging, classification by forests of random trees, tree boosting classification and regression, classification by rotation of decision tree forests, etc.).
• Decision trees are sometimes combined with each other or with other learning techniques. Procedures for aggregating the performance of the different models used (such as decisions by consensus) are put in place to obtain maximum performance, while controlling the level of complexity of the models used.
• the decision trees correspond to a white box model: if a certain situation is observed on a model, it can be explained using Boolean logic, unlike black box models like neural networks, whose explanation of the results is difficult to understand.
• Decision trees have many advantages: little data preparation (no normalization, empty values to delete, or dummy variable); management of numerical values and categories.
• evolutionary algorithms can be used to avoid separations leading to local optima.
• the method uses a CMA-ES algorithm (acronym for “Covariance Matrix Adaptation Evolution Strategies”), or one of its variants (e.g. hierarchical evolution strategies Meta-ES or Nested-ED).
• CMA-ES acronym for “Covariance Matrix Adaptation Evolution Strategies”
• one of its variants e.g. hierarchical evolution strategies Meta-ES or Nested-ED.
• the CMA-ES algorithm is based in particular on the adaptation, during the iterations, of the variance-covariance matrix of the multi-normal distribution used for the mutation.
• CMA-ES type algorithms are advantageous, in particular for non-convex, non-separable, malformed, multimodal or noisy functions.
• Black box optimization studies have shown that CMA-ES is effective in "harsh conditions" or large search spaces.
• CMA-ES algorithms have also been extended to multi-objective optimization problems (MO-CMA-ES).
• a CMA-ES algorithm comprises in particular a principle of maximum likelihood (the covariance matrix is updated incrementally so that the likelihood of the previous search steps is increased).
• the CMA-ES algorithm can be replaced by a distribution estimation algorithm or "Cross-Entropy Method”.
• the CMA-ES algorithm can be replaced by a “DownhHI simplex method 'or“ Surrogate-based methods ”, by“ BFGS ”or“ NEWUOA ”methods or even“ Multilevel Coordinate Search (MCS ) ”.
• a black box does not display its content (or its accessible content is not intelligible).
• the evaluation can have a cost. For simple BNs, it is possible to sample randomly, deterministically or according to predefined patterns. It is also possible to intensify the exploration around a specific point or to search for borders.
• a method for a bidirectional man-machine dialogue comprising the steps consisting in: translating, according to a so-called downward translation, so-called general commands from a predefined semantic repository into data of 'inputs that can be manipulated by the machine, by one or more universal approximators, for example one or more neural networks and / or fuzzy logic decision trees; and, translate, according to a so-called upward translation, raw output data determined by the machine, into data expressed in said predefined semantic repository, by one or more white boxes comprising one or more fuzzy logic decision trees.
• the method can be learned / optimized so that the upward translation resulting from a given command is the closest to that naturally recorded (communication histories, etc.).
• the commands can be abstract or general.
• a black box, between one or more inputs and one or more outputs, designates a process for which access to intermediate data and / or calculation rules is not possible, or, where applicable, cannot be directly intelligible to the man.
• a white box designates a process in which the intermediate data and / or calculation rules are accessible and directly intelligible to humans.
• a universal approximator can be in a white box or in a black box.
• the downward translation comprises at least one or more black boxes. These black boxes can be arranged in series and / or in parallel.
• the ascending translation (or ascending or ascending) can also use one or more white boxes.
• the upward translation starts from the technical parameters of the output of the system of systems to arrive at proposals with a high level of abstraction; this translation is performed using fuzzy logic decision trees.
• the method uses a genetic optimization algorithm.
• a genetic optimization algorithm iteratively manipulates a set of vectors of real variables using mutation and selection operators.
• a mutation step is performed by adding a random value drawn from within a distribution which is generally normal. Selection is made by a deterministic choice of the best individuals, or by recombination, according to the value scale of an objective function. The use of a recombination operator generally helps to avoid being trapped in local optima.
• a fuzzy logic decision tree captures one or more business logic.
• the method further comprises the steps of: receiving the output data from the machine in response to the input data; compare the input high-level abstraction data captured by the pilot's HMI with the translated high-level abstraction data. This step qualifies or constitutes a cycle ⁇ capture, downward translation, upward translation, restitution ⁇ . Millions of cycles can be repeated. Analog rendering can be part of it (interface and analog rendering, i.e. to the edges of human cognition).
• the method further comprises the step of selecting output data from among several, by filtering and / or thresholding, or in particular by traversing the upward translation made up of fuzzy logic decision trees.
• GFT fuzzy logic decision trees
• the method further comprises the step of controlling at least one black box by at least one white box, a white box.
• white comprising one or more fuzzy logic GFT decision trees.
• the method indeed manipulates a white box and a black box.
• the step of controlling a network of descending black boxes by an ascending white box comprises the step of optimizing said black box by automatic learning.
• the method indeed manipulates a white box (upward translation) against N black boxes (several black boxes i.e. a “network”, according to an arrangement, or a “plurality of” black boxes).
• the white box can be composed of a plurality of white boxes, in finite number, which white boxes control the optimization of several black boxes.
• the white box (s) (upward translation) "encode” the business expertise specific to an application area. These white boxes are therefore “settled” (in the sense of a manual operation).
• a box is said to be black in the sense that the weightings and the intermediate states in neural networks are only rarely intelligible directly to the non-expert.
• the black box or boxes “encode” (by learning on a mass of data observing the activities of numerous pilots in various aircraft) the downward translation.
• BNs allow the injection of great computing power. Human judgments are embedded in the machine, but in ways that are generally not accessible or interpretable. BNs do not encode business expertise (vertical application, field of application).
• the method can therefore be “asymmetric” in that the downward translation in black box (left) is optimized by - relies on - the upward translation (right), which is carefully regulated by humans, and which encodes business expertise.
• the asymmetry can be reversed.
• the translation operations can be "learned” (massive data) or “adjusted” (manual frugal adjustment, simple cases). In reality, all intermediate situations can be observed on the continuum. Automation can be more or less extensive and sometimes (pragmatically) exceed human processing capacities.
• the invention provides the BN / BB architecture described in the document, and its various options (eg modifications of the graphs in GFTs, BNs in series and / or in parallel, networks of approximators).
• the method further comprises the step of selecting a network of universal approximators from among several, by machine learning.
• the method indeed handles N white boxes and M black boxes.
• the algorithms can be put in competition, by clusters or entire networks. The machine part can therefore be pushed to its limits. For example, current data may argue for using a more efficient network than the default network.
• the step of optimizing the graph of fuzzy inference systems FIS of a GFT is performed by machine learning.
• a universal approximator is a parameterized function and / or a neural network and / or a CMA-ES network.
• a neural network is a parsimonious approximator.
• the use of a CMA-ES network is particularly advantageous.
• automatic learning comprises the implementation of a genetic algorithm, which determines the adjustment of the fuzzy decision trees GFT used in the up and / or down translators, in particular the adjustment of the functions of. membership and fuzzy rule bases of each FIS fuzzy inference system composing the GFT fuzzy decision tree.
• said adjustment is carried out by breaking down the membership functions and the rule bases into a plurality of associated genes, then mixing them randomly and / or randomly replacing one or more genes with others.
• the method further comprises the step of optimizing by genetic algorithm the structure of a fuzzy logic decision tree.
• the method further comprises the step of updating current data relating to the aircraft or its environment, said data independently modifying the data of the white boxes and / or black boxes.
• the data refresh rate may vary depending on the application. Refreshment can be extremely slow, possibly years (satellite, Martian robots) but can conversely be extremely fast (e.g. road traffic, intensity of actions).
• the current data is raw data (radar, mount, cloud) and / or space partition data (peaceful zone, supply zone), etc.); a subset of which is sent to the downward translation.
• the method further comprises the step of accessing one or more intermediate values manipulated in the fuzzy logic GFT decision trees.
• the method further comprises the step of displaying one or more intermediate values manipulated in the GFT fuzzy logic decision trees.
• one or more of the white boxes of the upward translation are displayed on demand in a man-machine interface.
• the white boxes can indeed be “unfoldable” ie displayable on request (display of intermediate values, root cause, etc.).
• a fuzzy inference system can be "open”, ie displayed or otherwise rendered (for example to see the rules used, display LLOMs, display one or more levels of dialogue abstraction, etc.).
• the upward translation is regulated by supervised learning (eg the coefficients of the neural networks are adjusted by considering a large number of data eg a large quantity of descending-ascending-judgment cycles).
• supervised learning eg the coefficients of the neural networks are adjusted by considering a large number of data eg a large quantity of descending-ascending-judgment cycles.
• the top-down translation is adjusted or tuned by reinforcement learning relying on the tuned or learned up-translator.
• the coefficients of the neural networks and / or of the GFTs can in particular be adjusted by considering a large number of data (a large number of descending-ascending-judgment cycles).
• learning can lead to a reduction in the number of states, for example to serve to filter very early and not only at the very end (in addition to a thresholding).
• the black box optimization of top-down translation then takes on its full meaning.
• the descending black boxes are put in competition.
• one or more of the intermediate calculation results, information relating to the root causes and / or the calculation context of one or more of the steps of the method are displayed in a man-machine interface.
• the display may allow you to press a "why" or “explain” button, the actuation of which may trigger the display of one or more of the applied fuzzy logic rules.
• Unstructured data e.g. images, videos
• a neural network can transform unstructured data into structured data
• It is also possible to "plug" into intermediate layers of a neural network unstructured data but in a reduced number compared to the input of the neural network).
• automatic learning is performed online.
• the machine learning is performed offline.
• the training data can be historical data.
• machine learning is performed online. This is because machine learning can be done incrementally or online.
• the model is known (stabilized weights in neural networks and / or GFT) and embedded, it is possible to continue learning in data flow (to improve the existing model, without starting from scratch).
• Offline machine learning learns from a complete data set while online learning can continue to learn (“learning transfer”), on-board, without having to re-ingest the starting data.
• the learning can be carried out beforehand (basic, preliminary) then personalized on the data specific to a company or to a particular pilot.
• fuzzy logic uses words from a finite dictionary in natural language, i.e. carrying semantics.
• fuzzy logic implies intelligibility for the pilot.
• the use of fuzzy logic presupposes the use of natural language words (carrying semantics) and fuzzy logic operators, but not necessarily formed sentences.
• the method according to the invention does not manipulate semantics, but syntactic forms or rules IF ... THEN ...
• the method according to the invention comprises a TAL module (descending or ascending).
• the lexicon / lexical field is predefined.
• the machine proposals are expressed while respecting the semantic field of the pilot (in order to make it understandable). If necessary, the outputs of the machine are forced on the words of the dictionary used.
• machine learning comprises one or more algorithms selected from algorithms comprising: support vector machines or wide margin separators; classifiers; neural networks; decision trees and / or stages of statistical methods such as the Gaussian mixture model, logistic regression, linear discriminant analysis and / or genetic algorithms.
• one or more data processing operations are governed by a certified FMS avionics flight management system internalizing predefined constraints.
• a computer program product comprising code instructions for performing one or more of the process steps, when said program is executed on a computer.
• a system for a bidirectional man-machine dialogue comprising - hardware memory and / or calculation resources, accessed locally and / or remotely, configured to: translate, according to a so-called downward translation, of the so-called commands.
• translate according to a so-called downward translation, of the so-called commands.
• one or more universal approximators for example one or more neural networks and / or fuzzy logic decision trees
• translate according to a so-called upward translation, raw output data determined by the machine, into data expressed in said predefined semantic repository, by one or more white boxes comprising one or more fuzzy logic decision trees.
• the method further comprises one or more neural networks configured for machine learning, said one or more neural networks being chosen from neural networks comprising: an artificial neural network; an acyclic artificial neural network; a recurrent neural network; a forward propagation neural network; a convolutional neural network; - a network of generative antagonist neurons; said one or more neural networks being software emulated and / or being physical circuits.
• FIG. 1 illustrates the state of the art.
• UMIs 15 eg speech-to-text, command by speech, automatic language processing, physical sensors and / or actuators, control by gestures, via tactile interfaces, possibly from data representations in axiological, 2D, 3D, radar view, etc.
• man 1 provides inputs (quantified and quantifiable) to complex systems 2.
• the complex system 2 After processing the information, the complex system 2 produces solutions according to different levels of abstraction 12 (e.g. a point on a map, a plot on a map).
• levels of abstraction 12 e.g. a point on a map, a plot on a map.
• FIG. 2 illustrates some general principles of the invention.
• the invention consists in particular in implementing a descending 110 and ascending 130 translation for an improved dialogue between man 1 and machine 2 (e.g. system of systems).
• the downward translation in order to pass from commands at a high level of abstraction 16 expressed by the pilot to technical parameters of low level 11 required by the system of systems, comprises one or more "black" boxes or universal approximators (GFT and / or RN and / or CMA-ES), congruent or concurrent (series-parallel). These boxes are black in the sense that the weightings in neural networks are only rarely intelligible directly to the non-expert), these black boxes having learned
• the upward translation starts from the low level operational parameters of the output of the system of systems 2 to arrive at high level proposals. of abstraction; this translation is performed using fuzzy logic decision trees.
• High level abstraction data (HLOM) 100 supplied by pilot 1 are translated 110 into a set of low level abstraction data 11 ("low level abstraction technical data” or LLTP) that can be manipulated. by machine system 2.
• HLOM High level abstraction data
• LLTP low level abstraction technical data
• HLOM data refers to sets of metrics at a high level of abstraction (intentions), often non-quantifiable, or "fuzzy” (in the sense of fuzzy logic). These data are expressed in the lexical field (or “ontology”) of the operator, making it possible to command an intention for a system (eg: “find a safe and efficient solution”, “detect small boats in heavy seas”, etc).
• Examples of data at a high level of abstraction include (for example) objectives, expressed in formal language (UML, symbology, etc.) and / or in natural language, ie predefined keywords or expressions or sentences such as "Optimize fuel consumption”, “reduce noise on the ground”,
• LLOM data refers to quantifiable operational metrics, provided by the system, allowing humans to interpret and judge a service rendered by the intelligent system. They are used by operations when developing their judgment and understanding the service provided.
• Examples of data at a low level of abstraction include (for example) values (numerical and / or symbolic): roll value, angle of incidence, pressure value, opening of a solenoid valve, active state or inactive equipment, etc. These data are possibly associated with technical functions (for example trajectory control).
• different N intermediate objects can be used, ie between the HLOMs and LLOMs.
• MLOMs Metal Level Operational Metrics
• N can be restricted (few objects or levels) or high (very large number of abstraction sublevels); the process steps may remain essentially the same.
• all or part of the data handled can be a function of the "context" 120, i.e. current data concerning the aircraft. Context can be perceived, measured, informed or otherwise constructed from sensor measurements.
• the context can be the flight context (takeoff phases, cruise) but also according to any division of time and / or space (eg approach phase in a given mission, priority level given among a predefined plurality, etc.)
• the “context” thus determined can influence or control, directly or indirectly, or otherwise modify the data handled (selection, different levels of priorities, etc.).
• the declared "intentions" of pilot 1 are high level abstraction factors (HLOM) and relate to the conduct of the mission as a whole.
• the pilot declares intentions 100, for example via “instructions”. These instructions are broken down into “tasks” to be carried out which are generally combined (linked and implemented simultaneously). For example, the pilot can seek to optimize his fuel consumption while respecting the noise constraints on the ground.
• the low level abstraction parameters (LLOM) 120 relating to the tasks to be performed are technical, quantifiable, measurable and contribute to the implementation of calculation algorithms (often constraint solvers). They are linked to a specific task (optimized trajectory, reaction to constraints, action plan, etc.).
• Operational intentions are thus translated 110 into intelligible technical parameters 11 by a decision support system 2.
• the recommendations of the decision support system are expressed 130 in the semantic reference (intentions) of the human.
• the use of known words and / or expressions advantageously makes it possible to contribute to the explicability of the man-machine dialogue (in particular in the machine-man sense).
• the TBB technical blocks (acronym for "Technical Building Block") designate the functions performed by the complex system making it possible to produce one or more "services”.
• each of the ascending and descending directions may be the subject of particular and advantageous treatments, which are described below in more detail.
• the top-down translation 110 performs an ontology translation (descent into abstraction) which must absorb the "transfer function" of the system. Apart from very simple cases, a person cannot properly set this translator.
• machine learning steps for the top-down translation can be based on an already-tuned bottom-up translator.
• the ascending part serves as a frame of reference or a fixed point to determine or stabilize the descending part.
• the downlink translator is trained in or by a learning process (eg, reinforcement learning) which includes in the chain (processing) the upward translator already set.
• a learning process eg, reinforcement learning
• the objective of optimization by reinforcement is to reduce the difference between, on the one hand, the instruction / intention (expressed in HLOM); and on the other hand, the characterization of the solution deduced from the setpoint (also in HLOM) obtained by crossing the adjusted ascending translator.
• the machine learning process of the top-down translation can advantageously rely on an already-tuned bottom-up translator.
• the efficiency and robustness of the learned model may depend on the variety and coverage of the input data, here learning scenarios.
• the top-down translation operates an ontology translation, i.e. an abstraction descent (and must also absorb the transfer function of the system, ie the elements that do not fall under the NLP).
• the recommendations of the decision support system are expressed 130 in the semantic repository or lexical field of the operator.
• the use of natural language words and / or expressions allows the pilot to understand the calculations of the machine.
• the upward translation ensures an increase in abstraction within the same (operational) ontology.
• the upward translation can be noted. It is advantageously in a "white box" to allow the operator to understand the service provided with a different level of abstraction.
• the upward translation can also be in a black box, if the previous level of explainability is not desired or required (for example by noting only the output or the result of the upward translation. can advantageously be adapted to the reasoning of the operational to judge the solution (precise, relevant, unambiguous information, ).
• one or more multi-criteria decision trees such as a GFT can be used.
• the technical choice having been made on universal approximators adjusted by learning, different learning methods can be implemented.
• the ascending translator may be set by supervised learning and the descending translator may be tuned by enhanced learning relying on the ascending translator set.
• the upward translator is adjusted in a supervised manner, on the basis of an operational data collection consisting in relating, a context, a solution, a vector of LLOM (calculated or provided) on this solution, and a vector of HLOM provided by the operational.
• the learning base is tagged and serves as a reference in a supervised learning process aimed at adjusting the backward translator.
• the learning loops in order to make the descending translator converge so that the "distance" between the high-level operational metrics desired and supplied at the input of the system and the high-level operational metrics observed on the response of the latter is minimum on a large number of samples.
• the machine learning of the downlink translator can be carried out in such a way as to make it "converge” (in the sense of stabilizing the hyper-parameters associated with the chosen universal approximator) towards the ascending semantics.
• the decision support 2) must be minimal or at least can be minimized on a large number of samples.
• the distance can be an edit distance or any other measurement system (metric).
• FIG. 3 illustrates an example of a fuzzy logic decision tree used in one embodiment of the invention.
• the method uses one or more fuzzy decision trees (fuzzy logic), for example 110, 111, 112, etc.
• a GFT includes one or more FIS ("Fuzzy Inference System"), for example the tree 110 includes FIS 301, 311, 312, 321, 322, 323, 324 etc.
• FIS Fuzzy Inference System
• the user indicates to the machine a destination point 301, to which the machine responds with two possibly antagonistic objectives: an ecological journey 311 and a economic toll-free travel 312.
• Each of the branches actually corresponds to different possibilities, common or mutually exclusive.
• the type of fuel has both an ecological and an economic impact (e.g. diesel).
• the driving style, mountain passages, etc. also come into play.
• Facts (e.g. 331) and / or fuzzy rules (e.g. 312) are handled in / by tree 110.
• the system and method according to the invention can arbitrate, i.e. select between one and more predefined trees, depending on current conditions (for example depending on whether the car is entering or leaving a motorway network). Even more, the processes and systems can learn, i.e. modify the structure of fuzzy decision trees (for example a tree 111 will be derived from a tree 110) by meta-learning.
• the pilot formulates a compromise, captured by a man-machine interface, between three objectives which can be at least partially antagonistic: the efficiency of the flight (result achieved or expected versus a given objective), the endurance of the aircraft (eg representative of the level of material stress) and the safety of the flight or the mission.
• Some compromises may not make sense, but there are a number of solutions (almost continuous). Being totally efficient, for example, would be to the detriment of flight safety (by taking reckless risks).
• the pilot may prioritize the endurance of his aircraft (called to fulfill other missions) first, then flight safety, over the effectiveness of the mission.
• the number of intentions will depend (in particular) on the field, on the operational needs and on the context of use. Those intentions can be independent or keep an antagonistic relationship, partially decoupled, linear or non-linear proportional, etc.
• FIG. 4 illustrates a possible generalization of the invention.
• a machine-to-human translation white box 410 can set, or adjust, or otherwise optimize 421 the parameters of a human-to-machine translation black box 420.
• the black box 420 can include one or more white boxes 421, which can symmetrically control one or more black boxes 422 within the white box 410.
• the white boxes and the black boxes can be arranged in very different ways: e.g. in series, in parallel, in competition, in congruence, etc. Arrangement schemes or patterns can follow graphs, be hierarchical, formed in a "fractal" way (recurrence of a pattern), etc. Part of the calculation concerning one or more parameters of which it is advantageous if they are accessible to human intelligence could thus be the subject of one or more white boxes (of which the perimeter is sufficient, ie a few calculation steps in upstream and / or downstream), while calculations which may escape intelligibility (eg millions of correlations, etc.) may remain inaccessible ie in black box (s).
• the perimeters can be scalable, even adaptive: a black box can be "open” to a white box (at least partially) and vice versa.
• Cross-learning can be carried out on different parts of the calculation steps and / or of the hyper-parameters and / or of the inputs / outputs of the black or white boxes.
• the man-machine interaction according to the invention 15 can be variously sophisticated.
• a method according to the invention can comprise one or more feedback loops (eg downstream feedback on the upstream, feedforward, etc.).
• a feedback loop can be “closed”, that is to say inaccessible to human control (it is executed by the machine). It can be “open” (eg display step in a man-machine interface, validation or any other human confirmation system).
• Different embodiments can lead to different implementations by closing, respectively by opening, one or more open loops, respectively closed.
• the method according to the invention can invoke only open feedback loops (ie the pilot intervenes at all stages), or else only closed feedback loops (eg total automation), or else a combination of two (the human contribution being variable or configurable).
• the process (which can be "artificial intelligence") can be interpreted as "transparent", in the sense of controllable.
• the display may concern intermediate calculation results, information relating to root causes, and / or the calculation context. In this way, the process can be considered “explicable”.
• the system comprises avionic flight management means of the Flight Management System type and / or non-avionic means of the Electronic Flight Bag (or "electronic bag") type and / or augmented reality means. and / or virtual.
• the AR means comprise in particular systems of the FIUD ("Flead Up Display” type head-up display) type and the VR means in particular include systems of the EVS ("Enhanced Vision System”) or SVS ("Synthetic Vision System”) type. ").
• the individual display means can comprise an opaque virtual reality headset or a semi-transparent augmented reality headset or a configurable transparency headset, projectors (pico-projectors for example, or video projectors to project the simulation scenes) or even a combination of such devices.
• the individual display headset can be a virtual reality headset (VR or VR in English), or an augmented reality headset (AR or AR in English) or high aim, etc.
• the helmet can therefore be a "head-mounted display", a "wearable computer”, “glasses", a head-mounted display, etc.
• the information displayed can be entirely virtual (displayed in the individual helmet), entirely real (for example projected onto the flat surfaces available in the actual cockpit environment) or a combination of the two (partly a virtual display superimposed or merged with the reality and partly a real display via projectors).
• the device comprises means for selecting one or more portions of the virtual display.
• the pointing of the man-machine interfaces (HMI) or of portions of these interfaces or information can be accessible via different devices, for example a pointing device of the “mouse” type or a designation based on a manual pointing; via acquisition interfaces (button, wheel, joystick, keyboard, remote control, motion sensors, microphone, etc.), via combined interfaces (touch screen, force feedback control, gloves or glove, etc.).
• HMI man-machine interfaces
• the man-machine interfaces can in fact include one or more selection interfaces (menus, pointers, etc.), graphical interfaces, voice interfaces, gesture and position interface.
• the information and menus are selected using a designation system (for example by pointer, via a mouse and / or a trackpad and / or a joystick, by voice command, etc.) completed by if necessary by a detection of the direction of the gaze incorporated in the semi-transparent helmet.
• the selection of these screens can be performed by one or more head movements.
• a device may include a communication bus to which a central processing unit or microprocessor (CPU, acronym for “Central Processing Unit”) is connected, which processor can be "multi-core” or “many-core” read-only memory (ROM, acronym for "Read On ly Memory” in English) which may include the programs necessary for the implementation of the invention; a random access memory or cache memory (RAM, acronym for "Random Access Memory” in English) comprising registers suitable for storing variables and parameters created and modified during the execution of the aforesaid programs; and a communication interface or I / O (I / O acronym for “Input / ouput”) suitable for transmitting and receiving data.
• CPU central processing unit
• ROM read-only memory
• RAM Random Access Memory
• the corresponding program (that is to say the sequence of instructions) can be stored in or on a storage medium removable (for example an SD card, or mass storage such as a hard disk eg an SSD) or non-removable, volatile or non-volatile, this storage medium being partially or totally readable by a computer or a processor.
• a storage medium for example an SD card, or mass storage such as a hard disk eg an SSD
• non-removable, volatile or non-volatile this storage medium being partially or totally readable by a computer or a processor.
• Reference to a computer program which, when executed, performs any of the functions described above, is not limited to an application program running on a single host computer. Rather, the terms computer program and software are used herein in a general sense to refer to any type of computer code (eg, application software, firmware, firmware, or any other form of computer code).
• the computing means or resources can in particular be distributed (“Cloud computing”), possibly with or according to peer-to-peer and / or virtualization technologies.
• Software code can be executed on any suitable processor (e.g., microprocessor) or processor core, or a set of processors, whether provided in a single computing device or distributed among multiple computing devices (e.g. example such as possibly accessible in the environment of the device).
• Security technologies crypto-processors, possibly biometric authentication, encryption, smart card, etc. can be used.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Artificial Intelligence (AREA)
• Software Systems (AREA)
• Health & Medical Sciences (AREA)
• General Physics & Mathematics (AREA)
• Evolutionary Computation (AREA)
• General Engineering & Computer Science (AREA)
• Mathematical Physics (AREA)
• Life Sciences & Earth Sciences (AREA)
• Computational Linguistics (AREA)
• Computing Systems (AREA)
• Data Mining & Analysis (AREA)
• Biophysics (AREA)
• General Health & Medical Sciences (AREA)
• Biomedical Technology (AREA)
• Molecular Biology (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Bioinformatics & Computational Biology (AREA)
• Evolutionary Biology (AREA)
• Fuzzy Systems (AREA)
• Automation & Control Theory (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Medical Informatics (AREA)
• Physiology (AREA)
• Aviation & Aerospace Engineering (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Computational Mathematics (AREA)
• Mathematical Analysis (AREA)
• Mathematical Optimization (AREA)
• Pure & Applied Mathematics (AREA)
• Genetics & Genomics (AREA)
• Management, Administration, Business Operations System, And Electronic Commerce (AREA)
• Machine Translation (AREA)
• Information Transfer Between Computers (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=74045455

Family Applications (1)

Country Status (4)

Families Citing this family (3)

Family Cites Families (15)

• 2020
  • 2020-06-04 FR FR2005823A patent/FR3111210B1/fr active Active
• 2021
  • 2021-06-01 US US17/928,251 patent/US20230312132A1/en active Pending
  • 2021-06-01 EP EP21728090.8A patent/EP4162402A1/de active Pending
  • 2021-06-01 WO PCT/EP2021/064608 patent/WO2021245049A1/fr not_active Ceased

Also Published As

Similar Documents

Legal Events