DE102021132399A1 - Method, system and computer program product for autonomously constructing and/or designing at least one component for an entity - Google Patents

Abstract

The invention relates to a method for autonomously constructing and/or designing at least one component (10) of an entity (20), comprising:- generating natural and/or sensor-based data (250) in an input module (200), the data ( 250) include text messages, voice messages, images, diagrams, and/or video sequences relating to the component (10) and/or the entity (20) generated by a natural person and/or sensors (500) and/or from a database (520);- transmitting the natural and/or sensor-based data (250) to a transformer module (300), the transformer module (300) using artificial intelligence algorithms;- transforming the natural and/or sensor-based data (250) in encoded data (350) as states (si), parameters (pi), parameter values and/or properties (ei) of the component (10) so that they can be processed by a learning reinforcement module (400) using a reinforcement learning algorithm ;- Forwarding the encoded data (350) to the learning reinforcement module (400) and calculating modeled values for the states (si), parameters (pi), parameter values and/or properties (ei) of the component (10) to generate output data ( 470);- outputting the output data (470) by an output module (800).

Description

The invention relates to a method, a system and a computer program product for autonomously constructing and/or designing at least one component for an entity.

In mechanical engineering and in automotive engineering, a large number of components with mechanical, mechatronic, electrical, electronic, hydraulic, chemical, etc. components are required, which are subject to constant further development due to new model series, environmental specifications, etc., which is time-consuming and costly is. Examples are components for an electric motor or a braking system of a motor vehicle.

The choice or design of a component for an entity such as a motor vehicle or a production facility depends on selected boundary conditions and parameters. In the case of a mechanical or mechatronic component, these are, for example, a relative speed between two moving components, an operating temperature, the service life, the material of the component, certain mechanical loads, etc. A component can be constructed or designed in such a way that various parameters such as the dimension , the shape, the materials or the electrical resistance can be changed while adhering to selected boundary conditions. Typically, designing a component or building a more complex entity is done based on the expertise and experience of experts such as engineers. However, this is associated with a considerable expenditure of time and thus costs. Optimization algorithms are therefore increasingly being used, but these in turn often have to be programmed for a specific task and are therefore generally only suitable for this specific task. Universal use is often not possible and therefore limits the area of application.

The WO 2019/183568 A1 discloses the use of deep reinforcement learning techniques to train a model that selects a robot action to perform a robot task. Natural language input can instruct the robot to perform a specific task.

The US 2021/0019642 A1 discloses a method that generates actions and/or policies for an environment based on an agent neural network, the environment comprising a device and/or a software component. A neural network for processing natural language is provided in order to be able to take speech inputs into account.

The KR 2019061488 A discloses a programming system based on speech recognition using artificial intelligence. A user's voice is recognized and through speech input, the programming results can be improved and optimized using artificial intelligence algorithms.

The object on which the invention is based is now to create a method, a system and a computer program product for autonomously constructing and/or designing at least one component of an entity, which is characterized by high reliability, security and accuracy and is easy to implement leaves.

According to the present invention, a method, a system and a computer program product are proposed by which an optimized and autonomous construction of at least one component of an entity is made possible, so that the construction process or design process takes less time, saves costs and at the same time an optimized solution for a construction task/design task can be found.

This object is achieved according to the invention with regard to a method by the features of patent claim 1, with regard to a system by the features of patent claim 11, and with regard to a computer program product by the features of patent claim 15. The further claims relate to preferred embodiments of the invention.

• - Generieren von natürlichen und/oder sensorbasierten Daten in einem Eingabemodul, wobei die Daten Textnachrichten, Sprachnachrichten, Bilder, Diagramme, und/oder Videosequenzen in Bezug auf die Komponente und/oder die Entität umfassen, die von einer natürlichen Person und/oder Sensoren erzeugt werden und/oder aus einer Datenbank abgerufen werden;
• - Übermitteln der natürlichen und/oder sensorbasierten Daten an ein Transformermodul, wobei das Transformermodul Algorithmen der künstlichen Intelligenz verwendet;
• - Transformieren der natürlichen und/oder sensorbasierten Daten in codierte Daten als Zustände s_i, Parameter p_i, Parameterwerte und/oder Eigenschaften e_i der Komponente, so dass sie von einem Lernverstärkungsmodul bearbeitbar sind, das einen Algorithmus des verstärkenden Lernens verwendet;
• - Weitergeben der codierten Daten an das Lernverstärkungsmodul und Berechnen von modellierten Werten für die Zustände s_i, Parameter p_i, Parameterwerte und/oder Eigenschaften e_i der Komponente zur Erzeugung von Ausgabedaten;
• - Ausgeben der Ausgabedaten durch ein Ausgabemodul.

According to a first aspect, the invention provides a method for autonomously constructing and/or designing at least one component of an entity. The procedure comprises the following procedural steps:

• - Generate natural and/or sensor-based data in an input module, the data comprising text messages, voice messages, images, diagrams, and/or video sequences relating to the component and/or entity generated by a natural person and/or sensors and/or retrieved from a database;
• - Transmitting the natural and/or sensor-based data to a transformer module, the transformer module using artificial intelligence algorithms;
• - Transforming the natural and/or sensor-based data into encoded data as states s _i , parameters p _i , parameter values and/or properties e _i of the component such that they are operable by a learning reinforcement module using a reinforcement learning algorithm;
• - passing the coded data to the learning reinforcement module and calculating modeled values for the states s _i , parameters p _i , parameter values and/or properties e _i of the component to generate output data;
• - Outputting the output data by an output module.

A further development provides that the transformer module has at least one encoder and at least one decoder, which are trained by at least one training data set and an attention algorithm in order to transform natural and/or sensor-based data into coded data for the learning reinforcement module.

Advantageously, the encoder and the decoder use neural networks and/or recurrent neural networks and/or convolutional neural networks.

In an advantageous embodiment, the reinforcement learning algorithm of the learning reinforcement module is designed as a Markov decision process or as temporal difference learning (TD learning) or as Q-learning or as SARSA or as a Monte Carlo simulation or as an actor-critic.

In particular, a parameter p _i and/or a parameter value and/or a property e _{i of} the component represents a dimension, a material, a shape, a characteristic value, a measured value, a safety factor, a thermal resistance, an electrical resistance, a conductivity, a Strength, fracture behavior, color, density, plastic deformation, stress, compression, flexural resistance, material property, surface property or heat treatment.

In one embodiment, the learning reinforcement module includes at least a learning reinforcement agent, an action module, an environment module, a state module, and a reward module.

In particular, it is provided that a state s _i of the component is determined by the state module, a state s _i being defined by parameters p _i and/or parameter values and/or a property e _i of the component.

A further development provides that a calculation function f _i and/or an action a _i is selected based on a guideline for the state _si for modifying at least one parameter p _i and/or a parameter value and/or a property e _i der component of the learning reinforcement agent; wherein a new state s _i+1 is calculated by the environment module based on the modified value for the parameter p _i and/or the parameter value and/or the property e _i ; wherein the new state s _i+1 is compared with a target state s _t and a deviation Δ for a comparison result is calculated in the state module; wherein a reward r _i is determined by the reward module for the comparison result; and wherein the states s _i and/or actions a _i are changed until the target state s _t is reached, which is output in the form of output data.

It is advantageously provided that the transformer module and/or the learning reinforcement module has/have one or more technical interfaces and protocols for access to a cloud computing infrastructure or is/are integrated in a cloud computing infrastructure.

A further development provides that the output module has an output transformer module that transforms the output data into result data, with the result data being available as construction plans, images and graphics, video sequences, acoustic sequences, holograms, key figures, diagrams, mathematical formulas and/or power Point presentations, etc. are trained.

According to a second aspect, the invention provides a system for autonomously constructing and/or designing at least one component of an entity. The system includes an input module configured to generate natural and/or sensor-based data, the natural and/or sensor-based data comprising text messages, voice messages, images, diagrams, video sequences related to the component and/or the entity generated by a natural person and/or sensors can be generated and/or can be retrieved from a database; a transformer module that uses artificial intelligence algorithms and is designed to transform the natural and/or sensor-based data into encoded data as states s _i , parameter p _i , parameter values and/or properties e _{i of} the component in such a way that the encoded data from a Learning reinforcement module that uses a reinforcement learning algorithm; the learning reinforcement module configured to provide modeled values for the states s _i , parameters p _i , calculate parameter values and/or properties _of the component to generate output data; and an output module configured to output the output data.

A further development provides that the transformer module has at least one encoder and at least one decoder, which are designed to be trained by at least one training data set and an attention algorithm in order to transform natural and/or sensor-based data into coded data for the learning reinforcement module.

Advantageously, the encoder and/or the decoder use neural networks and/or recurrent neural networks and/or folded neural networks (convolutional neural networks).

In one embodiment it is provided that the learning reinforcement module has at least one learning reinforcement agent, an action module, an environment module, a state module and a reward module.

According to a third aspect, the invention provides a computer program product comprising executable program code configured, when executed, to carry out the method according to the first aspect.

The invention is explained in more detail below with reference to exemplary embodiments illustrated in the drawing.

• 1 ein Blockdiagramm zur Erläuterung eines Ausführungsbeispiels eines erfindungsgemäßen Systems;
• 2 ein Blockdiagramm eines Lernverstärkungsmoduls;
• 3 ein Flussdiagramm zur Erläuterung der einzelnen Verfahrensschritte eines erfindungsgemäßen Verfahrens;
• 4 ein Blockdiagramm eines Computerprogrammprodukt gemäß einer Ausführungsform des dritten Aspekts der Erfindung.

It shows:

• 1 a block diagram to explain an embodiment of a system according to the invention;
• 2 a block diagram of a learning reinforcement module;
• 3 a flowchart to explain the individual steps of a method according to the invention;
• 4 a block diagram of a computer program product according to an embodiment of the third aspect of the invention.

Additional features, aspects and advantages of the invention or embodiments thereof will become apparent from the detailed description coupled with the claims.

1 shows a system 100 according to the invention for autonomously constructing and/or designing at least one component 10 of an entity 20. An example of a mechanical component is a spring component, which is characterized by parameters such as a spring constant, number of turns, etc. Another example is a bearing such as a plain or roller bearing. In a plain bearing, the parts that move relative to each other touch or are separated from each other by a sliding film. In a rolling bearing, rolling elements such as balls or rollers that perform a rolling motion are located between an inner ring and an outer ring. The components of a bearing can be described by parameters such as dimensions, shape, the selected material, etc., the parameter values assigned to the parameters and other properties.

Within the scope of the invention, however, a component 10 can also be an electrical, electronic, mechatronic, hydraulic, chemical, biological or hybrid component.

In general, a component 10 can thus be described by parameters, parameter values and properties. Entity 20 can be a vehicle such as a motor vehicle, an autonomously driving vehicle, an agricultural vehicle such as a combine harvester, a robot in production or in service and maintenance facilities, or a watercraft or a flying object such as a drone act. However, Entity 20 can also be a component or a system such as a brake system, an electric motor or an internal combustion engine for a motor vehicle, or an energy production system such as a photovoltaic system or a wind turbine, or a production system for manufacturing a product, such as a production plant for the manufacture of a motor vehicle. In particular, the entity 20 can be embodied as a scientific analysis instrument such as a spectrometer, a seismograph, etc. or as a medical device for diagnosing and supporting bodily functions.

However, the entity 20 can also be model calculations for scientific studies, climate models, economic models such as stock trading, development projects such as drug development or product development, administrative tasks, route planning, software developments, for example for autonomous driving, etc.

The system 100 according to the invention is based on methods of reinforcement learning and comprises an input module 200, a transformer module 300, a learning reinforcement module 400 and an output module 800. The transformer module 300 has at least one encoder 320 and one decoder 340. In particular, a plurality of encoders 320 and decoders are connected in series. Like in the 2 As illustrated, the learning reinforcement module 400 includes a reinforcement learning agent (LV) 410, an action module 420, an environment module 430, a state module 440, and a reward module 450.

The input module 200, the transformer module 300, the learning reinforcement module 400 and the output module 500 can each be provided with a processor and/or a memory unit.

In the context of the invention, a “processor” can be understood to mean, for example, a machine or an electronic circuit. A processor can in particular be a central processing unit (CPU), a microprocessor or a microcontroller, for example an application-specific integrated circuit or a digital signal processor, possibly in combination with a memory unit for storing program instructions, etc . A processor can also be understood to mean a virtualized processor, a virtual machine or a soft CPU. It can also be a programmable processor, for example, which is equipped with configuration steps for executing the mentioned method according to the invention or is configured with configuration steps in such a way that the programmable processor has the inventive features of the method, the component, the modules, or other aspects and/or or realized partial aspects of the invention. In particular, the processor can contain highly parallel computing units and powerful graphics modules.

In the context of the invention, a “memory unit” or “memory module” and the like can mean, for example, a volatile memory in the form of a random-access memory (RAM) or a permanent memory such as a hard disk or a data carrier or e.g. B. be understood as a removable memory module. However, the storage module can also be a cloud-based storage solution.

In connection with the invention, a “module” can be understood to mean, for example, a processor and/or a memory unit for storing program instructions. For example, the processor is specially set up to execute the program instructions in such a way that the processor and/or the control unit executes functions in order to implement or realize the method according to the invention or a step of the method according to the invention.

In addition, the transformer module 300 and/or the learning reinforcement module 400 can be connected to sensors 500 which transmit data from the entity 20 and/or the component 10 to the learning reinforcement module 400, in particular to the environment module 430. The sensors 500 can be used as clocks, counters, pressure sensors, piezo sensors, speed sensors, acceleration sensors, speed sensors, capacitive sensors, inductive sensors, temperature sensors, imaging cameras in the visible range, UV cameras in the ultraviolet range, IR cameras in the infrared range, LIDAR (Light detection and ranging) systems with optical distance and speed measurement, stereoscopic optical camera systems, ultrasonic systems, and / or radar systems are formed.

The data determined by the sensors 500 can be images and/or data about physical, chemical, biometric and/or physiological processes and states.

Furthermore, the transformer module 300 and/or the learning reinforcement module 400 can be connected to at least one database 520 in which historical data in the form of images, graphics, time series, parameters, etc. are stored. For example, target variables and target values that define a safety standard can be stored in the database 520 . "Database" means both a storage algorithm and the hardware in the form of a storage unit. In particular, the database 520 can be in the form of a cloud computing infrastructure 700 .

In connection with the invention, “data” is understood to mean both raw data and already processed data from the measurement results of the sensors 500 and/or data stored in the database 520 and/or a cloud computing infrastructure 700 .

The data recorded by the sensors 500 are forwarded to the transformer module 300 and/or the learning reinforcement module 400 by means of communication connections such as a CAN bus system (Controller Area Network). However, wireless connections can also be provided. A wireless communication connection is designed in particular as a mobile radio connection and/or a near-field communication connection such as ^Bluetooth® , Ethernet, NFC (near field communication) or Wi-Fi®.

The input module 200 is designed in particular as a user interface for inputting and generating natural and/or sensor-based data 250 in the form of text messages and/or voice messages and/or images and graphics forms. A keyboard, a microphone, a camera and/or a display designed as a touchscreen are provided for this purpose.

The output module 800 is designed to output result data 850 such as construction plans or data about parameters to be selected, such as a material, a dimension, etc., parameter values and other properties. The input module 200 and the output module 800 can be integrated in a hardware device such as a computer, a tablet, a smartphone, etc.

The transformer module 300 and/or the learning reinforcement module 400 can be designed as independent computer units or as a cloud-based solution. The database 520 can also be integrated into the cloud computing infrastructure 700 .

It can be provided that in particular the input module 200 and/or the sensors 500 and/or the output module 800 are equipped with mobile radio modules of the 5G standard. 5G is the fifth-generation mobile communications standard and, compared to the 4G mobile communications standard, is characterized by higher data rates of up to 10 Gbit/sec, the use of higher frequency ranges such as 2100, 2600 or 3600 megahertz, increased frequency capacity and thus increased data throughput and real-time data transmission because up to one million devices per square kilometer can be addressed simultaneously. The latency times are a few milliseconds to less than 1 ms, so that real-time transmission of data and calculation results is possible. The data recorded by the sensors 500 can be sent in real time to the transformer module 300 and/or learning reinforcement module 400 integrated in the cloud computing infrastructure 700, where a simulation of the parameters and properties of at least one component 10 of the entity 20 is carried out.

This speed of data transmission is required if cloud-based solutions are to be used to process the data. Cloud-based solutions offer the advantage of high and therefore fast computing power. In order to protect the connection to the cloud computing infrastructure 700 by means of a mobile radio connection, cryptographic encryption methods are provided in particular. However, cryptographic encryption methods can also be used for the connection between the sensors 500 and the transformer module 300 and/or the learning reinforcement module 400 and for the connections between the input module 200 and the transformer module 300 and/or the connection between the transformer module 300 and the learning reinforcement module 400 and/or the connection between the learning reinforcement module 400 and the output module 800 may be provided.

1. a) Minimieren der Komponentenquerschnittsfläche, um sicherzustellen, dass die Zugspannung nicht höher als 150 MPa ist und die Zugkraft zwischen 1000 und 5000 N liegt
2. b) Maximieren der Biegespannung
3. c) Festlegen, dass die Länge der Komponente im Bereich von 1, 2 - 1,8 m liegt
4. d) Festlegen, dass bei der Optimierung der Komponente der Durchmesser die Größe von 30 mm nicht überschreitet

The data 250 generated by the input module 200 in the form of text messages and/or voice messages and/or images and graphics are forwarded to the encoder 320 of the transformer module 200 for further processing. The natural and/or sensor-based data 250 can be generated by entering text messages, which can be, for example, as follows:

1. a) Minimizing the component cross-sectional area to ensure that the tensile stress is not higher than 150 MPa and the tensile force is between 1000 and 5000 N
2. b) maximizing the bending stress
3. c) Determine that the length of the component is in the range of 1.2 - 1.8 m
4. d) Determine that when optimizing the component, the diameter does not exceed the size of 30 mm

The natural and/or sensor-based data 250 are transformed in the transformer module 200 in such a way that they can be forwarded to the learning reinforcement module 400 as encoded data 350 .

1. a) p < 150 MPa und F₂ - F₁ = 5000 - 1000 = 4000 Nm A ≤ F₂ - F₁ / p
2. b) p > 400 MPa
3. c) 1,2 m < I <1,8 m
4. d) d < 30 mm

This encoded data 350 may then look like this for the example text messages entered:

1. a) p < 150 MPa and F ₂ - F ₁ = 5000 - 1000 = 4000 Nm A ≤ F ₂ - F ₁ / p
2. b) p > 400 MPa
3. c) 1.2m<I<1.8m
4. d) d<30mm

The transformer module 300 thus translates natural and/or sensor-based data 250 such as text messages and voice messages into physical parameters and parameter values. In addition, the data 250 can also be images, video sequences and graphics from which the transformer module 300 derives physical and other parameters, parameter values and properties.

To do this, the transformer module 300 uses the encoder 320 and the decoder 340 to translate a sequence of characters, such as speech signals or text messages, into another sequence of characters. To do this, the encoder 320 and the decoder 340 use artificial intelligence algorithms such as deep learning and machine image analysis to convert the natural and/or sensorba edited data 250. Neural networks, recurrent neural networks and/or folded neural networks are advantageously used for this. In particular, attention algorithms are used that imitate cognitive attention. In particular, attentional algorithms use folded neural networks. An example of an attention algorithm is Open AI's GPT 2/3. The attention algorithm classifies the important parts of the natural and/or sensor-based data 250 and hides the less important ones. By training with a training data set and the attention algorithm, the transformer module 300 can learn which pieces of data are meaningful for a given context and which are not.

A neural network consists of neurons that are arranged in several layers and connected to each other in different ways. A neuron is able to receive information from outside or from another neuron at its input, to evaluate the information in a certain way and to pass it on to another neuron in a changed form at the neuron output or to output it as the end result. Hidden neurons are located between the input neurons and output neurons. Depending on the network type, there can be several layers of hidden neurons. They ensure the forwarding and processing of the information. Finally, output neurons deliver a result and output it to the outside world. The arrangement and linking of the neurons results in different types of neural networks such as feedforward networks, recurrent networks or convolutional neural networks. The networks can be trained through unsupervised or supervised learning.

The convolutional neural network is a special form of an artificial neural network. It has multiple layers of convolution and is well suited for machine learning and artificial intelligence (AI) applications in the field of pattern recognition. The functionality of a convolutional neural network is to a certain extent modeled on biological processes and the structure is comparable to the visual cortex of the brain. The training of a convolutional neural network usually takes place under supervision. Conventional neural networks consist of fully or partially meshed neurons in several levels. However, these structures reach their limits when processing images, since there would have to be a number of inputs corresponding to the number of pixels. The convolutional neural network is made up of different layers and is based on the basic principle of a partly locally meshed neural feedforward network. The individual layers of the CNN are the convolutional layer, the pooling layer and the fully meshed layer. The convolutional layer is the actual convolution layer and is able to recognize and extract individual features in the input data. In the case of pattern and image recognition, these can be features such as lines, edges or specific shapes. The input data is processed in the form of tensors such as a matrix or vectors. The pooling layer, also known as the subsampling layer, compresses and reduces the resolution of the recognized features using appropriate filter functions. The reduced data volume increases the calculation speed.

The convolutional neural network (CNN) therefore offers numerous advantages over conventional non-convoluted neural networks. It is suitable for machine learning and artificial intelligence applications with large amounts of input data, such as in pattern and image recognition. The network works reliably and is not sensitive to distortions or other optical changes. CNN can process images captured under different lighting conditions and from different perspectives. It still recognizes the typical features of an image. Since the CNN is divided into several local, partially meshed layers, it requires significantly less storage space than fully meshed neural networks. The convolutional layers drastically reduce storage requirements. The training time of the convolutional neural network is also greatly reduced. With the use of modern graphics processors, CNNs can be trained very efficiently.

The data 350 encoded by the transformer module 300 is provided to the learning reinforcement module 400 . The data 350 serves as input for the action module 420, the environment module 430, the state module 440 and/or the reward module 450.

In the learning reinforcement module 400, the reinforcement learning agent (LV) 410 selects a particular state _si ∈ S from a set of available states for at least one action a _i ∈ A from a set of available actions. For the selected action a _i , the LV agent 410 receives a reward that can be positive, neutral, or negative. The LV agent 410 receives the states s _i ∈S from the state module 440 , which contains the encoded data 350 of the transformer module 300 . In addition, further data from different sensors 500 and/or the database 520 can be processed and statuses s _i ∈S can be assigned to these processed data, which the LV agent 410 can access.

The LV agent 410 selects at least one action _{a i} ∈ A from a set of available actions for a given state _si ∈ S from a set of available states. The choice of the selected action a _i is based on a strategy or guideline. For the selected action a _i the agent 410 receives a reward r _i ∈ ℝ from the reward module 450. The LV agent 410 receives the states s _i ∈ S from the state module 440. The strategy is based on of received rewards r _i adjusted. The strategy specifies which action a _i ∈ A should be selected from the set of available actions for a given state s _i ∈ S from the set of available states. This creates a new state s _i+1 for which the LV agent 410 receives a reward r _i . A strategy thus defines the assignment between a state _si and an action a _i , so that the strategy specifies the choice of action a _i to be carried out for a state _si . The goal of the LV agent 410 is to maximize the rewards r _i , r _i+1 ,..., r _i+n achieved.

In the action module 420, the actions a _i selected by the LV agent 410 are carried out. An action a _i is used to adjust a parameter p _i and/or a parameter value and/or a property e _i of the component 10 that is to be constructed or designed. The action a _i is preferably one of the actions A(+), A(0) and A(-). A positive action A(+) is an action that, for example, increases the value for a parameter p _i , a neutral action A(0) is an action in which the value of the parameter p _i remains the same , while for a negative action A(-) the value of the parameter p _i is decreased.

The environment module 430 calculates the states s _i ∈ S based on the selected action a _i and taking into account previously defined constraints. The boundary conditions can also be economic aspects such as the cost structure, the energy costs, the environmental balance, the availability or the delivery situation.

A state s _i ∈ S is thus defined by the selection of specific parameters p _i , parameter values and/or properties e _i and is therefore characterized by measured and/or calculated values of selected parameters p _i and/or properties e _i . The properties e _i can be, for example, a safety coefficient, the energy consumption during production, climate compatibility, etc.

In the reward module 450, a reward r _i is assigned to the degree of deviation Δ between the calculated value for the state s _i and the target value of a target state s _t . Since the degree of deviation Δ depends on the selection of the respective action A(+), A(0), A(-), a matrix or a database of the respective selected action A(+), A(0) , A(-) assigned the reward r _i . A reward r _i preferably has the values +1 and -1, where a small or positive deviation Δ between the calculated state s _i and the target state s _t is rewarded with +1 and is thus amplified, while a significant negative deviation Δ is rewarded with -1 and is therefore evaluated negatively. However, it is also conceivable that values >1 and values <1 are used. The final state is reached when the calculated states s _i are equal to or greater than the target state s _t .

A Markov decision process is preferably used as the algorithm for the LV agent 410 . However, provision can also be made for using a temporal difference learning (TD learning) algorithm. The LV agent 410 with a TD learning algorithm does the adaptation of the actions A(+), A(0), A(-) not only when it receives the reward, but after each action a _i on basis an estimated expected reward. Algorithms such as Q-Learning and SARSA are also conceivable, or Actor-Critic or Monte Carlo simulations. With the algorithm, dynamic programming and adaptation of the strategy through iteration processes is possible.

In addition, the LV agent 410 and/or the action module 420 and/or the environment module 430 and/or the state module 440 and/or the reward module 450 contains calculation methods and algorithms f _i for mathematical regression methods or physical model calculations that establish a correlation between selected parameters describe p _i and a target state s _t . The mathematical functions f _t can be statistical methods such as mean values, minimum and maximum values, lookup tables, models for expected values, linear regression methods or Gaussian processes, Fast Fourier transformations, integral and differential calculations, Markov methods, probability methods such as Monte Carlo methods, temporal difference learning, but also extended Kalman filters, radial basis functions, data fields, convergent neural networks, deep neural networks, artificial neural networks and/or feedback neural networks. Based on the actions a _i and the rewards r _i , the LV agent selects 410 and/or action module 420 and/or environment module 430 and/or state module 440 and/or bypass module 430 selects one or more of these calculation functions f _i for a state s _i .

A second cycle for constructing or designing the at least one component 10 of the entity 20 now begins. Here, the LV agent 410 can perform a different action a _i+1 and/or a different calculation function f _i+1 and/or a different parameter p _i+1 selected according to the defined strategy or guideline. The result is in turn fed to the status module 440 and the result of the comparison is evaluated in the reward module 450 . The LV agent 410 repeats the design process for all provided actions a _i , a _i+1 ,..., a _i+n , calculation functions f _i , f _i+1 ,..., f _i+n and Parameters p _i , p _i+1 ,..., p _i+n until there is the greatest possible correspondence between a calculated state s _i , s _i+1 ,..., s _i+n and a target state s _t is reached. The end state of the construction process or the simulation process of the features of the component 10 is preferably reached when the deviation Δ is in the range of +/-5%. The LV agent 410 thus optimizes its behavior and thus the strategy according to which an action a _i is selected until the calculated states _si , _si+1 , . . . , _si+n converge. The optimized result of the construction or simulation of features of the respective component 10 of the entity can be output in the form of output data 470 on the output module 800 .

The output data 470 can be output directly from the output module 800 . However, it can also be provided that the output module 800 has an output transformer module 820 which translates the output data 470 calculated by the learning reinforcement agent 400 into a data format in the form of result data 850 which is accessible to a user. The result data 850 can be, for example, construction plans, images and graphics, video sequences, acoustic sequences, holograms, key figures, diagrams, mathematical formulas, Power Point presentations, etc. For this purpose, the output transformer module 820 can likewise have at least one encoder and one decoder or encoders and decoders connected in series.

In particular, it can be provided that the calculation results are stored in the form of statuses, actions, rewards and strategies in the cloud computing infrastructure 700 and are each available via the Internet. The LV agent 410, the action module 420, the environment module 430, the status module 440 and the reward module 450 have the necessary technical interfaces and protocols for access to the cloud computing infrastructure 700 the access options and the access speeds to already calculated states, actions, rewards and strategies are simplified.

Since a quick calculation result is often sought, the calculation speed is decisive for the large number of calculation operations in the transformer module 300 and the learning reinforcement module 400 . In order to enable the arithmetic operations to be carried out in real time, the use of the cloud computing infrastructure 700 is therefore advantageous since this can ensure rapid calculation. A 5G cellular connection is again advantageous for the communication of the input module 200 with the cloud computing infrastructure 700 since data can be transmitted in real time in this way.

The functioning of the system 100 according to the invention is explained by the following example. For example, a user writes or speaks the following instruction: "Design a component where the maximum allowable mechanical stress is 150 MPa with a tolerance of +/- 5% and the maximum allowable bending moment is 1800 Nm." This input data 250 in form a natural text message are forwarded to the transformer module 300. The transformer module 300 translates this instruction into the encoded data 350: $$\text{p}>{\text{150 MPa and M}}_{\text{b}}<\text{180 Nm}$$

This data 350 is now forwarded to the learning reinforcement module 400 . A reward is formulated in the rewards module 450 as follows: $$150\text{MPa}-5\%<\text{p}<150\text{MPa}+5\%$$

The state module 440 may include the states "bending moment, length, elastic modulus, diameter, stress, etc.". These states are changed by selecting an appropriate action such as increase A(+), remain the same A(0) and decrease A(-) until a simulation and thus construction of the component 10 is achieved that satisfies the boundary conditions entered.

In 3 the method steps for autonomously constructing and/or designing at least one component 10 for an entity 20 using the system 100 according to the invention are shown.

In a step S10, natural and/or sensor-based data 250 are generated in an input module, the data 250 text direct, voice messages, images, diagrams, and/or video sequences relating to the component 10 and/or the entity 20, which are generated by a natural person and/or sensors 500 and/or retrieved from a database 520.

In a step S20, the natural and/or sensor-based data 250 is transmitted to a transformer module 300, with the transformer module 300 using artificial intelligence algorithms.

In a step S30, the natural and/or sensor-based data 250 are transformed into encoded data 350 as states s _i , parameters p _i , parameter values and/or properties e _i of the component 10 such that they can be processed by a learning reinforcement module 400 that has a Reinforcement learning algorithm used.

In a step S40 the encoded data 350 is forwarded to the learning reinforcement module 400 and modeled values for the states s _i , parameters p _i , parameter values and/or properties e _i of the component 10 for generating output data 470 are calculated.

The output data 470 is output by an output module 800 in a step S50.

4 Figure 12 schematically illustrates a computer program product 900 comprising executable program code 950 configured to perform the method according to the first aspect of the present invention when executed.

With the method and the system 100 according to the present invention, a component 10 for an entity 20 can thus be reliably constructed or designed without the learning reinforcement module 400 having to be specifically programmed for a certain construction task or design task. Instead, natural language messages and text messages or sensor data such as images can be used, which are transformed by the transformer module 300 into encoded data formats 350 suitable for the learning reinforcement module 400 . The transformer module 300 thus represents an interface in the form of an HMI (human machine interface), which enables data entered in natural language or also image formats to be translated into encoded data 350 for the learning reinforcement module 400 . The learning reinforcement module 400 can then reliably find optimized states s _i by selecting appropriate actions a _i for the encoded data 350 in order to construct or design one or more components 10 for an entity 20 reliably and autonomously. By using a transformer module 300 in conjunction with a learning reinforcement agent 410 with a reinforcement learning algorithm, it is possible to perform construction or design processes autonomously and self-optimizing.

As a result, optimally constructed and designed mechanical components for selected purposes can be provided in a short time and at reduced costs. In particular, boundary conditions that go beyond the shape can also be taken into account for the component to be designed. A boundary condition can relate, for example, to the use of recycling material, climate-neutral production and/or the cost structure.

In a further development, the system 100 itself can become an inventor of components 10 if it inventively solves a task set for it by means of the natural and/or sensor-based data 250, since the component created by the system 100 meets the criteria of novelty and inventive activity compared to the previously known state of the art. Within the scope of this invention, the system 100 is therefore also referred to as ^PERL® as an abbreviation for “Porsche Engineering Reinforcement Learning”. The present invention is thus an example of artificial intelligence having the ability to become the inventor of a technical invention.

Reference List

10

component

20

entity

100

system

200

input module

250

Data

300

transformer module

320

encoders

340

decoder

350

encoded data

400

learning reinforcement module

410

Learning Reinforcement Agent

420

action module

430

environment module

440

state module

450

reward module

470

output data

500

sensors

520

Database

700

cloud computing infrastructure

800

output module

820

Output Transformer Module

850

results data

900

computer program product

950

program code

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• WO 2019/183568 A1 [0004]
• US 2021/0019642 A1 [0005]
• KR 2019061488 A [0006]

Claims (15)

A method for autonomously constructing and/or designing at least one component (10) of an entity (20), comprising: - generating (S10) natural and/or sensor-based data (250) in an input module (200), the data ( 250) include text messages, voice messages, images, diagrams, and/or video sequences related to the component (10) and/or the entity (20), generated by and/or from a natural person and/or sensors (500). retrieved from a database (520); - Transmission (S20) of the natural and/or sensor-based data (250) to a transformer module (300), the transformer module (300) using artificial intelligence algorithms; - transforming (S30) the natural and/or sensor-based data (250) into encoded data (350) as states (s _i ), parameters (p _i ), parameter values and/or properties (e _i ) of the component (10), see above that they are processable by a learning reinforcement module (400) using a reinforcement learning algorithm; - forwarding (S40) the encoded data (350) to the learning reinforcement module (400) and calculating modeled values for the states (s _i ), parameters (p _i ), parameter values and/or properties (e _i ) of the component (10) for generating output data (470); - Outputting (S50) the output data (470) by an output module (800). procedure after claim 1 , wherein the transformer module (300) has at least one encoder (320) and at least one decoder (340), which are trained by at least one training data set and an attention algorithm to convert natural and/or sensor-based data (250) into encoded data (350) for transform the learning reinforcement module (400). procedure after claim 1 or 2 , wherein the encoder (320) and the decoder (340) use neural networks and/or recurrent neural networks and/or convolutional neural networks. Procedure according to one of Claims 1 until 3 , wherein the algorithm of the reinforcement learning of the learning reinforcement module (400) is designed as a Markov decision process or as Temporal Difference Learning (TD-Learning) or as Q-Learning or as SARSA or as Monte Carlo simulation or as Actor-Critic. Procedure according to one of Claims 1 until 4 , wherein at least one parameter (p _i ) and/or a parameter value and/or a property (e _i ) of the component (10) is a dimension, a material, a shape, a characteristic value, a measured value, a safety factor, a thermal resistance, represents electrical resistance, conductivity, strength, fracture behavior, color, density, plastic deformation, stress, compression, flexural resistance, material property, surface property, or heat treatment. Procedure according to one of Claims 1 until 5 , wherein the learning reinforcement module (400) comprises at least one learning reinforcement agent (410), an action module (420), an environment module (430), a state module (440) and a reward module (450). procedure after claim 6 , wherein a state (s _i ) of the component (10) is determined by the state module (440), wherein a state (s _i ) by parameters (p _i ) and/or parameter values and/or a property (e _i ) of the component (10) is defined. procedure after claim 7 , wherein a calculation function (f _i ) and/or an action (a _i ) is selected based on a policy for the state (s _i ) for modifying at least one parameter (p _i ) and/or a parameter value and/or a property (e _i ) the component (10) of the learning reinforcement agent (410); wherein a new state (s _i+1 ) is calculated by the environment module (430) based on the modified value for the parameter (p _i ) and/or the parameter value and/or the property (e _i ); wherein the new state (s _i+1 ) is compared to a target state (s _t ) and a deviation (Δ) for a comparison result is calculated in the state module (440); wherein a reward (r _i ) is determined by the reward module (450) for the comparison result; and wherein the states (s _i ) and/or actions (a _i ) are changed until the target state (s _t ) is reached, which is output in the form of output data (470). Procedure according to one of Claims 1 until 8th , wherein the transformer module (300) and/or the learning reinforcement module (400) has/have one or more technical interfaces and protocols for access to a cloud computing infrastructure (700) or in a cloud computing infrastructure (700) is/are integrated. Procedure according to one of Claims 1 until 9 , wherein the output module (800) has an output transformer module (820) which transforms the output data (470) into result data (850), the result data (850) as construction plans, images and graphics, video sequences, acoustic sequences, holograms, key figures , Diagrams, mathematical formulas and/or Power Point presentations, etc. are formed. System (100) for autonomously constructing and/or designing at least one component (10) of an entity (20), comprising: an input module (300) which is designed to generate natural and/or sensor-based data (250), the natural and/or sensor-based data (250) include text messages and voice messages, images, diagrams, video sequences relating to the component (10) and/or the entity (20) that can be generated by a natural person and/or sensors (500). and/or can be retrieved from a database (520); a transformer module (300) that uses artificial intelligence algorithms and is configured to convert the natural and/or sensor-based data (250) into encoded data (350) as states (s _i ), parameters (p _i ), parameter values and/or properties (e _i ) transforming the component (10) such that the encoded data (350) is processable by a learning reinforcement module (400) using a reinforcement learning algorithm; the learning reinforcement module (400) configured to calculate modeled values for the states (s _i ), parameters (p _i ), parameter values and/or properties (e _i ) of the component (10) to generate output data (470); and an output module (800) which is designed to output the output data (470). system (100) after claim 11 , wherein the transformer module (300) has at least one encoder (320) and at least one decoder (340), which are designed to be trained by at least one training data set and an attention algorithm in order to convert natural and/or sensor-based data (250) into encoded data (350) for the learning reinforcement module (400). system (100) after claim 11 or 12 , wherein the encoder (320) and/or the decoder (340) use neural networks and/or recurrent neural networks and/or convolutional neural networks. System (100) according to one of Claims 11 until 13 , wherein the learning reinforcement module (400) comprises at least one learning reinforcement agent (410), an action module (420), an environment module (430), a state module (440) and a reward module (450). Computer program product (900), comprising an executable program code (950), which is configured so that when it is executed it performs the method according to any one of Claims 1 until 10 executes

Images