RU2791840C2 - Decision-making system in a multi-agent environment - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: technical result consists in the adaptive configuration of the multi-agent system depending on the received data of external influence from the system sensors or devices. The technical result is achieved due to the receipt by the computing system of external influence data from sensors or devices of the system; formation of internal images of the system based on the received data of external influence, and the internal images of the system is stored in the internal images library; selection of operating internal images corresponding to the current state of the system at the present time; calculation of the priority matrix for active internal images; making a decision based on the execution of a plurality of processes, and the input data for this stage are the current internal images and the priority matrix; execution of the decision made by the computer system; formation of the experience of the system based on the decisions made.

EFFECT: adaptive configuration of the multi-agent system depending on the received data of external influence from the system sensors or devices.

24 cl, 5 dwg

Description

FIELD OF TECHNOLOGY

Implementations of the invention generally relate to the field of computing systems, and more specifically to systems designed to make decisions without human intervention.

BACKGROUND OF THE INVENTION

Currently, multi-agent (multi-purpose, multi-agent) systems are of great relevance and are widely used in various fields, such as robotics, security, medicine, logistics, business, marketing, etc. The decision-making system is the key element and basis of all functioning to create a "Strong" Artificial Intelligence (Strong AI / AGI). Thinking is the continuous decision-making at different levels in relation to different goals or objectives. One of the goals of the invention is the implementation of a system capable of making an aggregate decision, taking into account the relevance of the goals. In this case, the solution must be optimal, giving the best development of the system both at the current time and in the future.

Modern attempts to create AI are generally reduced to the development of individual fragments of intelligence, each of which solves a single-purpose problem suitable for formalization. However, there are many complexities when assembling a general/entire multitasking system. If we compare the process of human thinking with AI, then a person is able to set certain goals for the future and outline plans for their implementation. A person is capable of making different decisions at any given time. However, not all decisions are available to conscious control and understanding. That is, decisions can be made and executed independently of human consciousness / in parallel, which does not always look like a logical decision on the part of AI.

Thinking has an analytical and prognostic function, respectively, thinking about the past and about the future. Thinking is also responsible for managing the current moment (operational decisions and their implementation). The task of ideal thinking is to find optimal solutions for the current moment of time, taking into account the future development of events.

The main task of the decision-making system is to adequately respond both to external influences, choosing the right decision at every moment of time, and to internal changes occurring in the system itself as such. At the same time, the system must steadily strive to achieve one or more of its competing goals. The goals or objectives of the system have a hierarchical structure. Plans are formed from the goals of different levels. The decision-making system mainly seeks to achieve the set goals and maintain / maintain the equilibrium state with the greatest efficiency.

Specific goals are set at the system design stage, based on a hierarchy of goals. The goals or objectives of the system can in some sense be compared with human needs (Maslow's Hierarchy of Needs). Each need is a separate goal. The relationship of goals to each other is the most difficult problem that can be solved, for example, through a system of priorities in decision-making. The result of which is a line or a matrix of states (depending on the complexity of the system being designed). In the context of the present description, in addition to everything, solutions are proposed for the formation of multi-level hierarchical goals and system needs for making optimal decisions.

Each goal is reflected in the status line (matrix) and is implemented by one separate process. A process or a separate task capable of realizing one goal can be made in the form of a neural network of various types or in the form of a universal analytical process.

DISCLOSURE OF THE INVENTION

Embodiments of the present invention describe decision mechanisms.

The autonomous decision-making system includes: an external influence receiving unit (102) configured to receive external influence data from sensors or devices of the system; an external impact processing unit (104) configured to generate internal images (IN) of the system based on the received external impact data; the VO library (106), which stores (in the process of learning) and corrects the VO of the system received from the block for processing external influences; a block of the current state of the system (108), which, based on data from the block of knowledge (118), selects the active VO corresponding to the current state of the system at the present time; a state supervisor (110) configured to calculate a priority matrix for active VOs; a decision block (112) associated with the state supervisor (110) and configured to make an optimal decision based on the execution of a plurality of processes, the input of the decision block being the active VO and the priority matrix; execution unit (114), configured to implement the decision; an operational learning unit (116) configured to generate a system experience based on the decisions made; and a common learning unit (120) configured to train the VO library (106) and the knowledge library (118).

The decision-making method is implemented by the computer system and includes such steps as: obtaining by the computer system data of external influence from sensors or devices of the system; formation of internal images (IO) of the system, based on the received data of external influence, and the IO of the system is stored in the VO library; selection of operating VOs corresponding to the current state of the system at the present time; calculation of the priority matrix for active VOs; making an optimal decision based on the execution of a set of processes, and the input data for this stage are the current VO and the priority matrix; execution by the computer system of the decision made; formation of the experience of the system based on the decisions made.

The computer-readable, non-volatile storage medium contains executable instructions that, when executed by a computer system, cause the computer system to perform a decision-making method, including steps such as: receiving by the computing system external influence data from sensors or devices of the system; formation of internal images (IO) of the system, based on the received data of external influence, and the IO of the system is stored in the VO library; selection of operating VOs corresponding to the current state of the system at the present time; calculation of a matrix of priorities for active VOs; making an optimal decision based on the execution of a set of processes, and the input data for this stage are the current VO and the priority matrix; execution by the computer system of the decision made; formation of the experience of the system based on the decisions made.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood by the following detailed description and the accompanying drawings of various embodiments of the invention. However, the drawings are not to be used to limit the invention to specific embodiments, but are only intended to explain and understand the essence of the present invention.

On FIG. 1 is an exemplary block diagram of a decision making system.

On FIG. 2 shows a block diagram of the functioning of the decision-making system.

On FIG. 3 is a block diagram of one of the possible implementations of the present invention, which uses cloud storage.

On FIG. 4 is a flow diagram of a decision making process in accordance with some embodiments of the present invention.

On FIG. 5 is a block diagram of a computer system that can be used to implement variants of decision methods.

DETAILED DESCRIPTION

The present document describes a computing system for decision making in a multi-agent environment and a corresponding decision making method. This system can find application in various fields of science and technology, in addition to the specific example of the implementation of the claimed invention described below. The decision-making system is a universal mechanism for making optimal decisions and can be used (or be a specific implementation of the system as such), for example, in financial or economic systems, for example, for profit / income optimization (stock broker), marketing systems, complex diagnostic systems of various orientation (for example, an automatic life support system, etc.).

In addition, the decision-making system or implemented method can be used in automatic/automated control systems for vehicles, robots (autopilots, drones (UAVs), etc.). Most often, autopilots are used to pilot aircraft (because there are the fewest obstacles), but modern technologies are gradually introducing autonomous functions into everyday vehicles. Let us consider in more detail one of the possible automated control systems for vehicles (TC).

Many modern cars are equipped with all the necessary equipment for autonomous movement and path planning, at least based on global positioning systems. With the help of wireless communication and positioning systems, modern vehicles are able to form a multi-agent system. The concept of autonomous road traffic implies the use of the principles of well-known multi-agent systems (MAS). Cars, as agents, can exchange messages with a navigation device located, for example, at an intersection and consisting of a computer and a wireless transmitter. When approaching an intersection, the vehicle establishes contact with such a navigation device.

The main element of the MAC is the universal agent. Therefore, to create a transport system based on MAS, it is necessary to select at least the following agents: agent-driver and agent-manager. A driver agent is a human-driven or autonomous vehicle. Moreover, in the case of a vehicle controlled by a person, the agent-driver can give hints to the driver to achieve the goal (sound, visual prompts, etc.). An agent manager is a system that can be installed, for example, at a traffic light (or at an intersection) and manages driver agents.

The main goal for the driver agent is to reach the destination in the shortest possible time, while using the shortest route, if possible, in order to use less fuel for the trip. The main goal of the agent-manager is to control the agent-drivers approaching the intersection, rebuild them and create a non-stop and conflict-free passage of vehicles at the intersection in the shortest possible time. To achieve the set goals, agents must coordinate their goals, behavior strategies, coordinate their actions and resolve conflicts.

Driver agents and manager agents can exchange knowledge gained using some special language and following the established rules/protocols of the system. MAC should find the optimal solution to the problem without external intervention. In this case, the optimal solution is understood as the solution for which the least amount of time and fuel will be spent.

Thus, the set of basic characteristics of an agent may include, for example, such properties as: 1) activity - the agent-manager must be able to organize and implement passage at the intersection without conflicts and without stops, in the minimum time; 2) reactivity - driver agents must perceive all the necessary information from the environment. At the same time, the environment for the driver agent includes: road signs, road markings, other driver agents, various traffic obstacles, weather phenomena (rain, fog), etc. 3) autonomy - agent-drivers and agent-managers must achieve their goals without human intervention; and 4) sociability - in order to achieve their goals, agent-drivers must actively exchange information among themselves using specially established communication protocols.

Next, as an example, consider an economic multi-agent stock market management system. The open nature of the modern information society and the global market economy leads to the active development of scientific and technological progress, increasing the level of competition in the markets, expanding access to the resources of the global Internet, which stimulates the development of the Internet economy, of which the electronic securities market is a part.

The growth of globalization processes puts forward new requirements for innovative management activities, in particular in the context of the fact that in the real securities market the main role is played by human traders (brokers, traders, etc.), and in the electronic market they correspond to intelligent agents who can accept make decisions, join groups and learn from the functioning of the market, that is, to perform almost all the actions that real people perform. In this regard, there is a need for intelligent methods that can continuously acquire new knowledge and change their structure and functions, developing and adapting to the tasks being solved and environmental conditions.

To achieve the designated management goals in the distributed interaction of various participants, multi-agent systems can be used. The basis of the multi-agent approach is the concept of an agent that functions as an independent algorithm/program or an adaptive element of an artificial intelligence system.

Unlike the traditional method, when a certain strictly defined algorithm is searched for, which allows finding the best solution to the problem, in multi-agent systems, the solution is the result of the interaction of a certain set of independent modules - agents or processes. The main feature in this case is the dynamics and unpredictability of the decision-making process. In practice, this means that solution optimization is achieved through numerous interactions that are almost impossible to trace. The agents of the system are assigned goals that they must achieve, while they do not define strict scenarios for their achievement.

In addition, a multi-agent decision-making system makes it possible to take into account the behavior of individual agents, as a result of which it becomes possible to identify rather subtle features of the collective behavior of system elements, take into account their interconnections with each other and with the environment. Such a system can be designed, for example, to: 1) search for data on financial instruments; 2) monitoring the status of securities and risk management; 3) support for decision-making on the purchase and sale of financial assets. And in each case there will be agents.

The above examples of the use of multi-agent systems are united by a common concept and goal - making the optimal decision at a given time for a specific task. The autonomous decision system described in the present invention has a structure that can be optimally designed for any of the above exemplary purpose or situation. In this case, only the set of agents or processes will change in accordance with the tasks assigned to the system.

On FIG. 1 is a block diagram of a decision making system. In an exemplary embodiment, the system (100) contains: a block for obtaining external influences (BB) (102), the output of which is connected to the input of the block for processing external influences (104), the output of which is connected to the input of the library of internal images (IB) (106), the output of which connected to the input of the block of the current state of the system (108), the second input of which is connected to the knowledge library (118); the output of the current state block of the system (108) is connected to the input of the state supervisor (110) and to the input of the decision block (112), moreover, the state supervisor (110) and the decision block (112) have two-way communication and their respective outputs are connected to the inputs of the executive block (114), which has two-way communication with the operational learning block (116), having additional two-way communication with the state supervisor (110); in this case, the knowledge library (118) and the VO library (106) have a two-way connection with a common learning block (120), the input of which is connected to the output of the state supervisor (110).

The interaction between the mentioned blocks of the system is implemented by transmitting data packets over the communication network. This system (100) implements several formalized single-purpose processes (tasks, agents), which leads to the adoption of an optimal decision by the system, which will be discussed in more detail below. Attention should be paid to the fact that, in principle, the construction of system (100) uses a block/modular approach. The system blocks are autonomous, which makes it possible to replace each of them for a specific application task, as well as the ability to exclude some blocks and add new blocks. Each process/agent of the system has its own goal, which is set at the design stage. Simple processes are assembled for a specific task.

External influences (IE) on the system are recorded by the block for receiving external influences (102), which contains a plurality of sensors and/or system devices. Examples of said sensors and/or devices can be at least known as: video camera, microphone, data input devices (keyboard, mouse, etc.), touch sensor, motion sensor, temperature sensor, humidity sensor, smoke sensor, light sensor, gas analyzer, seismic sensor, etc. It will be clear to a person skilled in the art that for each particular system, its own individual set of sensors and/or devices for recording external influences will be used.

The data received by the block for receiving external influences (102) is sent to the block for processing external influences (104), which can be implemented, for example, by an artificial neural network. In another alternative, a plurality of neural networks trained by known methods can be used, each of which is trained to form specific internal images inherent in a specific external influence (IE), according to its essence. In another alternative, instead of a neural network, an analytical system or other mechanism for processing the IU can be used if the IU is "simple" and no complex mechanism is required to form the internal image (IN). The neural network identifies the data coming to its input and applies a set of functional transformations to them to form internal images (IN) of the system (obtaining internal information).

For example, in one of the alternative embodiments of the invention, the resulting video sequence can be divided into sequences of frames identified by the trained neural network model as related to one event (for example, a car passing through an intersection, a train moving through a checkpoint, etc.).

In another alternative, for example, if photographs are used as data of external influences (102), for example, obtained using a camera equipped with a presence sensor, then the data can be identified by a trained neural network model, for example, by identifying an area of interest in the image and identifying and classifying an object contained within the boundaries of the detected region of interest. It should be noted that the above examples are not limiting the application of the present invention. For each kind/type of external influences (102), when data is received by the system, specialized tools can be applied that are specific to the processing of such data and inherent in the corresponding data type. The claimed invention is a scalable/expandable/adaptive and versatile decision tool.

In some implementations, a neural network (NN) may be represented by a recurrent neural network. In other implementations, the neural network may be represented by a convolutional neural network, which in various implementations of the system may include any suitable number of convolutional layers, ReLU layers, pooling layers, and/or any other layers. The order of the layers, the number of layers, the number of filters, or any other parameter of the convolutional neural network can be adjusted, for example, using empirical data. In other implementations, but not limited to, fully connected neural network models, deep learning models, etc., may be used, depending on the specific task.

However, the above options do not limit the use of other neural network models to obtain internal system images (IS). For example, a specific NN model may depend on external influences (IE). It is also worth considering that in addition to one NN model, for example, an ensemble of various kinds of NNs trained by known methods can be used to obtain consolidated information from external influences in their diversity. Let us assume that the system is exposed to many heterogeneous impacts and each type of impact is subject to processing in order to isolate signs of external influences (IE) significant for the system and their subsequent consolidation to ensure the operation of the proposed system.

An ensemble of NNs may include, but are not limited to, for example, fully connected NNs, recurrent NNs, forward pass NNs, autoencoders, deconvolutional NNs, and other kinds of neural network models.

The specified ensemble of NS can be combined, for example, based on the types of external influences (IE) on the decision-making system. This ensemble can be expanded / narrowed depending on external influences for a specific implementation task of the system. All data received in such an implementation of the decision-making system is subject to concatenation by known methods for further processing in the framework of ensuring the operation of the said declared decision-making system.

Internal images (IO) received by the external influences processing unit (104) enter the VO library (106), where they are stored and constantly updated based on the current state of the system (associative memory/database of the system). To help identify VOs within a system, for example, metadata assigned to VOs and other attributes can be used, but not limited to.

Each VO of the system has its own identifier and a set of associations (associative/feedback). In the context of this invention, an association is the relationship of VO (significance or priority) to a specific goal or need of the system. In general, in the context of the present invention, it can be said that in the aggregate VO contains the relation of the "image" of the object to the goals of the system to be achieved. Each VO from the VO library (106) is capable of influencing the current state of the system, that is, the state of the system at the present time. Each VO has its own significance value (or priority), which differs for different macrostates of the system, taking into account the correction factors. During the operation of the system and internal algorithms for training the system, VOs are filled with content: associations (changes in the state of the system) and cause-and-effect relationships, patterns (system reactions, decision-making).

As the indicated values of the significance of the VO, for example, the coefficients (K) specified in the system can be used, on the basis of which the indicated priorities can be adjusted (for example, in the range from 0 to 1). The indicated coefficients of significance can be obtained on the basis of the expert experience of the system or, for example, empirically, or in the process of learning (accumulation of experience) in the system. Examples of such coefficients can be, for example, but not limited to, the coefficient of insignificance of VO (KNZ), the coefficient of importance of VO (KV), the coefficient of significance of VO (KZ), the coefficient of insignificance of VO (KMZ). For example, in one embodiment, you can take that if K is less than the KHZ having a value of 0.1 (or some other value), then such a VO is not relevant to the goal being solved at the present time and, therefore, such a VO can be discarded. If K is greater than KV and has a value of 0.85 or more, then the larger K, the more important it is for a specific goal and decision making at the current time. However, it is worth considering that a maximum K greater than the KV (for example, 1.0 or a value very close to 1.0) leads to the possibility of suppressing other targets for the present time (survival). As an example of the maximum K and its influence on the automatic piloting system, one can assume the decision to make an emergency landing in the event of a complete exhaustion of the fuel supply; collision with an obstacle, which gives a greater opportunity for the survival of the system (an autonomous vehicle decides to avoid a collision in the snow, rather than ram an oncoming vehicle when a collision is inevitable). Also, the given CV equal to 0.5, or another value, with K greater than the CV, indicates the mandatory participation of the VO for making a specific decision. Further, it can be assumed that when K is larger than the KMZ (that is, K is in the correct range of 0.1 - 0.5), it is of little importance for a particular solution and has a small priority. At the same time, in this case, the more K, the greater the priority of the VO.

The purpose of these coefficients is to determine whether the VO belongs to the decision-making / inconsistency or inapplicability of the VO to a specific task, the decision being made and the goal as such. Thus, an insignificant or insignificant VO can be ignored by the system, which reduces the resource costs of the system. However, it is worth considering that an insignificant VO may have a higher priority for another decision at another point in time.

In one embodiment of the system, the VO library (106) may be configured to learn itself, for example, based on data received from the state supervisor (110) via a common learning block (120). Alternatively, the common learning block (120) is for mutual learning of the VO library (106) and for learning of the knowledge library (118). The general learning block (120) learns/teaches to recognize VOs, describe VOs and form appropriate context/content for them.

In one implementation, the context, for example, can be represented in textual form, for example, using methods of distributive semantics. In another, alternative implementation, the context can be represented, for example, as metadata. In yet another alternative implementation, the context may be presented in a combined form. This statement is not intended to limit the ways and methods of context formation. In any implementation option, the context formation method can be selected, for example, based on expert experience and knowledge, depending on the essence of the VO (set of heterogeneous VO) and its attributes inherent in a particular VO. At the same time, if the declared decision-making system has many heterogeneous VOs, then each group of homogeneous VOs is characterized by the formation of a context in a similar way.

It should be noted that in the VO library (106) internal images are subjected to clustering. The clustering algorithm can be a standard statistical procedure that implements the collection of data containing information about the sample of VO and analysis of VO. Next, the process of ordering/grouping similar VOs into relatively homogeneous clusters is performed. In this case, such classical clustering algorithms as k-means, EM algorithm, hierarchical clustering, etc. can be applied.

VO from the library of images (106) enter the block of the current state of the system (108), which, based on the data from the knowledge library (118), the patterns of clusters, converts the VO into ΔVO (delta VO), corresponding to the current state of the system. In this block (106) the current picture of the surrounding world (system RAM) is formed. That is, only the currently operating VOs with the highest priority, not lower than the SC value, are selected.

The received ΔBOs are further sent to the state supervisor (110), as well as to the decision block (112). The state supervisor (110) is responsible for determining, managing and correcting the current state of the system (100), which influences the system's subsequent adoption of an optimal decision. The state supervisor (110) separates the active VOs into significant and insignificant for each determined subsequent process, taking into account the coefficients of significance. Thus, in the current state, for each VO, the significance/priority of the VO itself is formed when acting on the system (100).

In one additional embodiment of the system, the state supervisor (110) may include a VO value corrector (not shown in the drawings). The VO values corrector is designed to recalculate the VO significance coefficients at each cycle of the system operation by the methods described in detail in this description.

For further processing, taking into account the significance and the priority matrix, only those VOs are selected that exceed the established threshold of significance, which can be preset or selected empirically. For example, the threshold may be selected from 0 to 1 (eg, 0.5 or other value), as detailed above with respect to the VO significance factors. The state or priority matrix reflects not only the current one, but also links the concepts of usefulness/value, which can be expressed through the concept of "energy". Energy in the context of the present invention is a certain metric. For example, the energy value may be predefined by a range of values, depending on the particular implementation of the decision system. Energy can have both positive and negative values (for example, -1, 0, 1 or another value from a predefined range). In this case, a negative energy value means the wrong "path" of the decision system. The decision-making system should tend to the optimal equilibrium energy state, that is, to the fundamental state. In this case, one can take into account Friston's free energy principle, which is designed to formalize the decision-making process by a method similar in some sense to the principle of least action in classical mechanics, which describes the evolution of physical systems. In general, the basic principle of any automatic system is its desire to achieve an equilibrium state, similar to the principles of the human psyche. The state supervisor (110) may be implemented as a standard computational algorithm or program. In another alternative, the state supervisor (110) may be implemented as a trainable defined neural network model that is trained to recalculate and determine relationships between priority matrix values at different points in time.

The current state of the system is determined in the state supervisor (110) by a set of priorities (priority matrix). The microstates of the system change if actions occur within the current set of VOs. The macrostate of the system can change only in the event of a new VO or significant internal changes in the system. At the same time, it should be taken into account that the transition to another macrostate of the system is possible only with a sufficient amount of energy, depending on changes in priorities.

As noted above, the received ΔBOs are also fed to the input of the decision block (112), which directly interacts with the state supervisor (110) through mutual feedback to generate a set of processes, each of which has its own priority, which will be discussed in more detail below. .

The optimal solution obtained in the decision block (112) for the current time is sent further to the execution unit (114), the result of which is a certain system action. Under the action in the context of this description is understood a single executive program/algorithm transmitted to the output of the system. The created information in the system, as well as the information available in the form of the current state, are processed, resulting in the response of the system in accordance with the correct optimal solution. It is important to note that a specific action is correct and optimal for the system only at the current moment in time and only for such a system configuration.

Data on the decision made from the execution unit (114) is additionally fed to the operational learning unit (116), which is connected by two-way communication with the state supervisor (110). The operational learning unit (116) is configured to generate an experience.

To understand the principle of the system, let's consider it in more detail with the disclosure of the processes / agents of the system. On FIG. 2 is a block diagram of the operation of the decision making system (200). The operation of the decision block (112) corresponding to FIG. 1 is based on the processing, execution and analysis of a plurality of processes generated by the system.

Each individual process is some subtask that is part of the overall task/goal of the system. Separation into separate processes is the only way to build an applied system. Simple processes are easy to formalize and automate. The system of priorities for parallel processes allows you to switch to the most significant process at the moment and, accordingly, the task, which allows the system to make the only correct optimal decision.

The main characteristic of a simple process is a single objective function. It is on this principle that the division into processes occurs. One goal - one process (or agent). Each highlighted process is assigned a priority cell in a row or status matrix. Each task is reflected in the status line (matrix) and is implemented by one separate process.

A process (or a separate task) can be implemented as a neural network of various types (recurrent neural network, convolutional neural network, etc.) or as a universal analytical process. As mentioned earlier, a process can be a universal agent. A universal agent or an intelligent agent in the context of the present description is a software and hardware tool that operates autonomously in some environment (physical or informational) and solves a specific task in accordance with the goals set for the system.

As shown in FIG. 2, all VO systems are stored in the VO library (106). The VO library stores all new VOs, old VOs, data on VOs of change processes, as well as VO clustering patterns. At the same time, filtering of old VOs can be provided in the system to reduce processing costs. So, for example, when a new VO identical to the old one arrives, the old VO can be deleted and, for example, only data can remain that the VO has arrived identical to the old one. The new VO can be assigned a context/metadata similar to the VO stroma and a timestamp about the arrival of the old VO and, for example, the number of old VOs that arrived. This data can be added, for example, to the metadata of a new incoming VO as historical data. As stated earlier, the VOs in the library (106) can be divided into clusters based on similarity. Internal images are input to the block of the current state of the system (108). One output of the mentioned block (108) goes to the state supervisor (110), from where, based on the priority matrix and the current state of the system, ΔVO of the target state of the system are selected/formed. Then ΔVO fall into the goals filter (212), which filters the goals and objectives of the system according to their degree of importance for the current moment and builds operational and tactical plans for the decision-making system. The goal filter may contain a thinking subblock (not shown in Figure 2) that is responsible for developing plans and strategies.

The thinking sub-block performs an analysis of the previous experience of the system (analysis of the system's reactions to external influences) and modeling of the future goals and objectives of the system. The operation of the system is simulated in the background for past and future tenses. That is, the reactions and actions of the system (past or possible in the future) to external influences are modeled/predicted. Forecasting can be based on, but not limited to, extrapolation methods, correlation and factor analysis methods, causal forecasting methods. Modeling future actions of the system, taking into account existing experience. Modeling can be performed, but not limited to, for example, based on the Lotka-Volterra, Bohoeffer-van der Pol modeling approaches, the Malthus model, financial and economic mathematical models (for example, calculating the optimal use of limited resources; optimizing profit from the size of capital investments, etc., in depending on the specific implementation of the decision-making system). Simulation results are analyzed and corrected.

In fact, the sub-block of thinking implements the internal activity of the system as a reaction to possible emerging situations (input data). The thinking sub-box helps prioritize goals and create plans and strategies, but the sub-box itself does not make the final decisions on how the system will proceed.

The process of work of the thinking sub-block can be reduced to modeling or analyzing the sequence of actions. At the same time, the result of the sub-block of thinking is not a specific action, but a plan, a strategy of actions subordinate to a global/general task. Goal - a task for the action at the current moment. A plan is a direction of action in the future. The plan sets a vector (set) of goals (goal states). At the same time, the sub-block of thinking indirectly takes part in the training of the system, since atypical / non-standard situations are analyzed in it, after which such situations are translated into standard ones - that is, experience is formed. The generated experience can be dynamically clustered by known methods (for example, the k-means method or another dynamic clustering method).

After the goals filter (212), the input of each process (or agent) is filtered in accordance with the current goals and plans, the operating ΔVO. In this context, Δ-delta is a metric of the difference between one VO and another (at the previous point in time). Each agent process is simple and fast, as well as versatile, capable of solving different problems with appropriate training, since it will be used many times throughout the system to solve any particular tasks or processes.

The division of a specific task of creating an application system into simple parallel processes is carried out at the stage of designing and creating a specific application system. Before operation, the system is pre-configured and tested, for example manually, to verify that the process division is correct. To test the work, automated testing methods can also be used, for example, with manual validation. The goal of a separate process is to find the most valuable solution within its local problem. Moreover, each process should be as fast as possible.

It should also be noted that when the system agent processes are formed, they need to be trained, for example, by standard deep learning methods, to the required working level. At the same time, in the process of system operation in the background, endless additional training in real time (self-improvement) will be implemented.

Each individual process or agent lends itself to formalization. For each process, the right decision can be made for it. The results of each executed system process are stored in the process results block (214) along with the decision priority. However, the individual decisions for each process need to be integrated into the overall solution, where each process will have a current real-time value/priority.

The overall optimal solution for the system is determined in the decision integrator (216), including taking into account priorities, and represents the union and correlation of the set of correct solutions obtained from individual processes to achieve the goal. Further, the obtained optimal solution enters the execution unit of the system (114), which initiates the execution of the corresponding action by the system.

The data stored in the process results block (214) is also input to the state supervisor (110), which, based on past decisions, corrects the current priority matrix. Updated data from the state supervisor is sent to the VO library (106) to change the content of the VO in a timely manner based on the system processes.

Next, consider one cycle of the decision-making system (100). In response to a change in external influences, two types of reactions are provided in the system. Different reactions correspond to the following situations: (1) the emergence of a new VO and (2) a change in the significance/state of actual VOs (in response to some action in the surrounding world). One cycle of the system is approximately 1 second, or another value, within the requirements of a particular application.

The first situation is when there is no new VO or any fundamental changes in existing VOs. In this case, the system calculates a slight change in the actions, on the basis of which the correction of the output decisions of the system is performed. In this case, the macrostate of the system does not change, and oscillations are calculated for the microstates of the system in the state supervisor (110). This move is quick and easy.

Based on the calculated fluctuations in the state supervisor (110), the goal filter (212) located in the decision block (112), or rather its thinking subblock, continues to work in the background. The thinking of the system is relevant for the current state (corresponds to the macrostate of the system) and changes in real time due to external influences on the system. The thinking subblock implements information processing in the system space.

The second situation is the emergence of a new VO for the system. A new VO is characterized by its content (a set of associations and causal relationships, attributes), which can affect both the microstates and the macrostate of the system. The mentioned states in this case are recalculated. State calculations/recalculations can be based on the Markov chain approach, mathematical methods of fuzzy logic (for example, an adaptive neuro-fuzzy inference system described in the universal form of approximators, etc.).

A Markov chain is a sequence of random events with a finite or countable number of outcomes, where the probability of each event occurring depends only on the state reached in the previous event. The Markov chain is characterized by the fact that with a fixed present, the future is independent of the past. Each individual process is in one of n states at any given time. Moreover, if it is in state number i, then it will go to state j with probability pij. Matrix P=|| pij || is called the transition matrix. The Markov chain can also be represented as a graph in which the vertices are the states of the process, and the edges are the transitions between states, and the probability of transition from i to j is written on the edge from i to j, that is, pij.

Returning to the second situation, if the macrostate of the system changes, then the block of the current state of the system (108), which reflects the picture of the world, is loaded with internal images (IO) corresponding to this state. For the past situation, the newly loaded VOs were insignificant, and for the current state they become active. The microstate of the system is also recalculated accordingly. The new/current microstate becomes some starting/starting point for further analysis of new microstates.

If the new VO is not "strong" enough to change the macrostate of the system, then only the microstates change, and the changes can be quite dramatic.

In each cycle of the system, the current optimal solution is calculated. It is calculated based on the state in which the given measure began. That is, based on the priority matrix, which is formed by the state supervisor (110) based on the results of the previous cycle. The new priority matrix from the current cycle will only be used in the next cycle. In a similar way, in each new cycle, the matrix of the significance of internal images for the next time moment is calculated.

On FIG. 3 is a block diagram of one of the possible implementations of the present invention, which uses cloud storage. Illustrated is the transfer of data between the decision system (100) and the cloud (302). It is important to note that the VO library (106) and the knowledge library (118) can be located in the cloud. Such a well-known approach to the organization of distributed storage of libraries in some implementations can reduce system resources for local data storage and allow the decision system (100) to access the specified libraries from anywhere in the world faster. However, this approach to organizing library storage is not a priority, but only one of the possible implementation options. The mentioned storage approach requires constant access to the network, which is not always possible if the system is implemented as a local system and does not require network access, or has restrictions on network access (for example, an autonomous system for automatic piloting and decision-making of a submarine, spacecraft and etc.). On the other hand, in the perspective of improving Internet access technologies and technology, a distributed cloud data storage approach may be possible as a base for a decision-making system with some specific implementation options.

On FIG. 4 is a flowchart of an exemplary decision making process 400. Method 400 and/or each of its individual functions, programs, subroutines, or operations may be executed by one or more processors of a computer system (in particular, computer system 500 depicted in FIG. 5), implementing method 400. In some embodiments, method 400 may be performed using a single processing thread. In an alternative embodiment, method 400 may be executed by two or more processing threads, each of which performs one or more separate functions, programs, subroutines, or operations related to the method. In one embodiment, the processing threads implementing method 400 may be synchronized (eg, using semaphores, critical sections, and/or other thread synchronization mechanisms). In an alternative embodiment, the processing threads implementing method 400 may execute asynchronously with respect to each other. Therefore, although in FIG. 4 and associated descriptions, the steps of method 400 are performed in a specific order, in various embodiments of the method, at least some of the described steps may be performed in parallel and/or in an arbitrary order.

Method 400, in one exemplary embodiment, comprises the following steps:

Step 402. Receipt by the computing system of data of external influence from sensors or devices of the system.

Step 404. Formation of internal images (VO) of the system, based on the received data of external influence, while the VO is stored in the VO library.

Step 406. Selection of active VOs corresponding to the current state of the system at the present time.

Step 408: Compute a priority matrix for active VOs.

Step 410: Making an optimal decision based on the execution of a plurality of processes, the input to this step being the live VOs and a priority matrix.

Step 412: Execution of the decision made by the computing system.

Step 414. Formation of the experience of the system based on the decisions made.

It should be noted that the decision making method 400 can be extended and refined by all of the particular implementations that have already been described above for the implementations of the decision making system 100.

On FIG. 5 is a block diagram of a computer system in accordance with some embodiments of the present invention. The computer system may be connected (eg, over a network) to other computer systems on a local area network, an intranet, an extranet, or the Internet. The computer system may operate as a server in a client-server network environment. A computer system may be a personal computer (PC), a tablet computer, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, or any device capable of executing a set of instructions (sequential or otherwise) that specifies actions to be performed by that system. device. In addition, while only one computer system is illustrated, the term "computer" should also be understood as any collection of computers that individually or collectively execute a set (or multiple sets) of instructions to perform any one or more of the methods described herein. A supercomputer, for example, running Linux, can also act as a computer.

An exemplary computer system 500 consists of a data processing device 502, random access memory 504 (e.g., read only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous dynamic random access memory (SDRAM)) , and storage devices 508 that communicate with each other via bus 522.

The data processing device 502 is one or more general purpose processing devices such as a microprocessor, a central processing unit, and the like. Processing device 502 may be a full instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing a combination of instruction sets.

The data processing device 502 may also be one or more special purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, and the like. Data processor 502 is configured to execute instructions 530 to implement decision method 400 (FIG. 4) as well as to perform any of the operations described above.

The computer system 500 may further include a network interface 506, a visual display device 512 (eg, a liquid crystal display), an alphanumeric input device 514 (eg, a keyboard), a cursor control device 516, and an stimulus acceptor 518. In one embodiment, the device 512 display, the alphanumeric input device 514, and the cursor control device 516 may be combined into a single component or device (eg, a touch-sensitive liquid crystal display).

An stimulus receiving device 518 is one or more devices or sensors for receiving an external stimulus. A video camera, a microphone, a touch sensor, a motion sensor, a temperature sensor, a humidity sensor, a smoke sensor, a light sensor, etc. can act as a device for receiving external influences.

Storage device 508 may include a computer-readable storage medium 510 that stores instructions 530 embodying any one or more of the techniques or functions described herein (method 400). The instructions 530 may also reside wholly or at least partially in RAM 504 and/or on data processor 502 while being executed by computer system 500. RAM 504 and data processor 502 are also computer-readable storage media. In some implementations, instructions 530 may additionally be transmitted or received over network 520 via network interface device 506.

Although in the illustrative examples the computer-readable storage medium 410 is represented in the singular, the term "machine-readable medium" should be understood as including one or more media (for example, a centralized or distributed database and/or its associated caches and servers). ) that store one or more instruction sets. The term "machine-readable medium" should also be understood to include any medium capable of storing, encoding, or carrying a set of instructions for execution by a machine and causing the machine to perform any one or more of the techniques of the present invention. Therefore, the term "computer-readable medium" should include, but is not limited to, solid-state storage devices, optical and magnetic storage media.

While the steps of the methods described herein are shown and described in a particular order, the order of steps of each method can be changed so that certain steps can be performed in reverse order, or so that certain steps can be performed at least partially simultaneously with other steps. . In some implementations, instructions or sub-operations of individual operations may be intermittent and/or interleaved.

It should be understood that the above description is illustrative and not restrictive. Many other embodiments will become apparent to those skilled in the art upon reading and understanding the above description. Therefore, the scope of the invention is determined by reference to the appended claims as well as to the full scope of equivalents against which such claims give rise to claims.

In the above description, numerous details are set forth. However, one skilled in the art will appreciate that aspects of the present invention may be practiced without these specific details. In some cases, to avoid obscuring the present invention, well-known structures and devices are presented in block diagram form rather than in detail.

It should be noted that in the absence of other specific indications, as became apparent from the following discussion, throughout the description, such terms in the discussion as "obtaining", "determining", "selection", "storage", "analysis", etc. ., refers to the activities and processes of a computer system or similar electronic computing device that manipulates data and converts data represented as physical (electronic) quantities in the registers and memory of a computer system into other data similarly represented as physical quantities in memory or registers of a computer system or other such devices for storing, transmitting or displaying information.

The present invention also relates to a device for performing the operations described in this document. This device may be specially designed for the desired purpose or may include a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored on a computer-readable storage medium, such as any type of disk, including floppy disks, optical disks, compact disks and magnetic optical disks, Read Only Memory (ROM), Random Access Memory (RAM), Programmable Read Only Memory (RAM). EPROM), electronically programmable read-only memory (EEPROM), magnetic or optical cards, or any type of media suitable for storing electronic instructions, each connected to a computer system bus.

The algorithms presented in this document are not inherently associated with a specific computer or other device. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may be more convenient to create a more specialized device to perform the required method steps. The required structure for a variety of such systems will be as indicated in the description. In addition, aspects of the present invention are not described with reference to a specific programming language. It should be borne in mind that various programming languages can be used to implement the provisions of the present invention, as described in this document.

Embodiments of the present invention may be in the form of a software product or software including a computer-readable medium with instructions stored thereon, which may be used to program a computer system (or other electronic devices) to perform the process in accordance with the present invention. A computer-readable medium includes any mechanism for storing or transmitting information in a form that is readable by a machine (eg, computer). For example, computer-readable (e.g., computer-readable) media includes computer-readable (e.g., computer-readable) storage media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory).

The words "example" or "given as an example" in this document means an example, case or illustration. Any aspect or solution described herein as "example" or "given as an example" should not necessarily be construed as preferable or advantageous over other aspects or solutions. Rather, the use of the words "example" or "exemplary" is intended to present concepts from a practical point of view. As used in this application, the term "or" is intended to mean an inclusive "or" and not an exclusive "or". In addition, the use of the terms "implementation", "single implementation", "exemplary implementation" or "single implementation" throughout the text does not mean the same implementation or implementation, unless they are described as such. In addition, the terms "first", "second", "third", "fourth", etc., used in this document are intended to refer to various elements and do not necessarily have an ordinal value in accordance with their numerical designation.

While many changes and modifications to the invention will no doubt become apparent to one of ordinary skill in the art, upon reading the foregoing description it should be understood that any particular embodiment shown and described by way of illustration should not be construed in any way. as limiting.

Therefore, references to the details of the various embodiments are not intended to limit the scope of the claims, which by themselves contain only the features considered as a disclosure of the invention.

Claims (40)

1. Autonomous decision-making system, containing: an external influence receiving unit (102) configured to receive external influence data from sensors or devices of the system; an external impact processing unit (104) configured to generate internal images (IN) of the system based on the received external impact data; the VO library (106), which stores and corrects the VO of the system received from the external influence processing unit; a block of the current state of the system (108), which, based on data from the block of knowledge (118), selects the active VO corresponding to the current state of the system at the present time; a state supervisor (110) configured to calculate a priority matrix for active VOs; a decision block (112) associated with the state supervisor (110) and configured to make a decision based on the execution of a plurality of processes, the input of the decision block being the active VO and the priority matrix; execution unit (114), configured to implement the decision; an operational learning unit (116) configured to generate a system experience based on the decisions made; And a common learning unit (120) configured to train the VO library (106) and the knowledge library (118). 2. The system according to claim 1, in which the block for obtaining IE is implemented by a neural network that performs VO recognition. 3. The system according to claim 1, in which the internal images (IO) in the IO library are divided into groups of homogeneous VO corresponding to a homogeneous context. 4. The system according to claim 1, in which the internal images in the VO library are divided into clusters by similarity. 5. The system of claim 1, wherein the state supervisor (110) is further configured to change the macro state and micro states of the system based on external influences. 6. The system according to claim 4, in which, when a new VO appears, the state supervisor (110) recalculates the states. 7. The system of claim 5, wherein Markov chains are used to calculate and recalculate states. 8. The system of claim 1, wherein the decision block includes a goal filter (212), a process results block (214), and a decision integrator (216). 9. The system of claim 7, wherein the goal filter (212) comprises a thinking subblock. 10. The system of claim 1, wherein the decision block (112) implements a plurality of parallel processes. 11. The system of claim 1, wherein each process of the plurality of processes is implemented by a universal agent or universal analytic process. 12. The system of claim 10, further configured to train universal agents. 13. The system of claim 1, wherein the process results block (214) has feedback to the state supervisor (110). 14. A method for automated decision-making implemented by a computer system, including steps such as: receipt by the computing system of data of external influence from sensors or devices of the system; formation of internal images (IO) of the system based on the received data of external influence, and the IO of the system is stored in the VO library; selection of operating VOs corresponding to the current state of the system at the present time; calculation of the priority matrix for active VOs; making a decision based on the execution of a plurality of processes, and the input data for this stage are the current VO and the priority matrix; execution by the computer system of the decision made; And formation of the experience of the system based on the decisions made. 15. The method according to claim 14, in which the received external influence data is processed by a neural network that performs VO recognition. 16. The method according to claim 14, in which the internal images (IR) are divided into groups of homogeneous IRs corresponding to a homogeneous context. 17. The method of claim 14, wherein the internal patterns are clustered by similarity. 18. The method of claim 14 is further configured to change the macrostate and microstates of the system based on external influences. 19. The method of claim 18, wherein when a new VO appears, the states are recalculated. 20. The method of claim 19, wherein Markov chains are used to calculate and recalculate the states. 21. The method of claim 14, configured to implement a plurality of parallel decision processes. 22. The method of claim 14, wherein each of the plurality of processes is implemented by a generic agent or generic analytic process. 23. The method of claim 22, configured to train universal agents. 24. A computer-readable, non-volatile storage medium containing executable instructions that, when executed by a computer system, cause the computer system to execute the decision-making method of any one of claims. 14-23.

Images