Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/004—Artificial life, i.e. computing arrangements simulating life
  • G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/04—Architecture, e.g. interconnection topology
  • G06N3/045—Combinations of networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N3/00—Computing arrangements based on biological models
  • G06N3/02—Neural networks
  • G06N3/08—Learning methods

Definitions

• This specification relates to controlling agents using neural network based multimodal language models.
• Machine learning models receive an input and generate an output, e.g., a predicted output, based on the received input.
• Some machine learning models are parametric models and generate the output based on the received input and on values of the parameters of the model.
• Neural networks are machine learning models that employ one or more layers of nonlinear units to predict an output for a received input.
• Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as input to the next layer in the network, i.e., the next hidden layer or the output layer.
• Each layer of the network generates an output from a received input in accordance with current values of a respective set of parameters.
• This specification describes systems and methods, implemented as computer programs on one or more computers in one or more locations, that can be used for controlling an agent to act in an environment to perform a task, and for training or evaluating an action selection system used by such an agent, and for obtaining training data for training such an action selection system.
• the action selection system may, but need not be, based on a neural network.
• the task can be one of a plurality of different tasks that the agent can be trained, or instructed, to perform.
• an agent configured to select actions to perform one or more tasks in an environment, the agent comprising a system as described herein.
• the agent includes one or more observation capture subsystems to capture observations of the environment, e.g. still or moving images of a real-world environment.
• a digital assistant device comprising a system as described herein.
• the digital assistant device can include a user interface to enable a user to request assistance and to output information.
• Some implementations of the described systems provide a robust and generalizable approach to determining whether or not a task has been achieved. For example, whilst it is possible to train a success detector for a particular task such a model often has difficulties when the environmental conditions change from those observed in the training data, or if the task changes. This can arise, for example, from natural visual variations in the environment e.g. due to lighting changes, or from changes camera position, or from differences between particular objects, or from distractions in the environment.
• the described techniques are robust to such changes, and also to variations in language used to describe a task (e.g. “lift a rubber duck” or “lift a toy duck object”), and can handle ambiguities, e.g.
• Implementations of the described systems can be used to detect task success based on object state or agent behavior, and across a wide range of tasks and environmental conditions.
• the robustness and generalizability of the described techniques facilitates the training, evaluation, and use, of agent action selection systems that have generalized action selection policies.
• FIG. 8 is an example multi-modal language model neural network.
• the agent 600 includes one or more observation capture subsystems 610 to capture observations of the environment, in particular for the action selection system 110 and for the language model neural network 120.
• the environment 104 may be a real-world environment and the observation capture subsystem(s) 610 can, e.g., capture a still or moving image of the real -world environment 104.
• the agent can use the language model interface 630 to interface with the language model neural network 120.
• the language model neural network 120 can process an input token string, in particular the previously described input token string 122 representing a question that asks whether the task has been achieved, and a second captured observation of the environment, to thereby obtain the language model output 124 comprising the answer to the question.
• the second captured observation of the environment may be, but need not be, the same as the first captured observation of the environment.
• the input token string 122 representing the question may be generated by the agent 600, e.g. using the language model interface 630, or it may be generated by the language model neural network 120. That is, in some implementations the language model neural network 120 may receive a text input from the agent 600 and generate a text output for the agent 600.
• the agent 102 comprises a human user of a digital assistant device such as a smart speaker, smart display, or other device.
• the information defining the task can be obtained from the digital assistant device, and the digital assistant device can be used to instruct the user based on the answer.
• this may comprise receiving, at the digital assistant device, a request from the user for assistance and determining, in response to the request, a series of tasks for the user to perform, e.g. steps or sub-tasks of an overall task.
• the digital assistant device can be used to output to the user an indication of the task, e.g.
• step or sub-task to be performed. This may be done using natural language, e.g. on a display and/or using a speech synthesis subsystem of the digital assistant device. Visual, e.g. video, and/or audio observations of the user performing the task may be captured, e.g. using the digital assistant device. A system as described above may then be used to determine whether the user has successfully achieved the task or a task step, or sub-task, from the answer. If there are further tasks to be completed the digital assistant device may then, in response, progress to the next task (if any) of the series of tasks, e.g. by outputting an indication of the next task to be performed. In this way the user may be led step-by-step through a series of tasks to perform an overall task.
• FIG. 7 shows an example of a digital assistant device 700, including a system as described above that may be implemented as computer programs on one or more computers in one or more locations.
• the digital assistant device 700 can include a user interface 710 to enable a user to request assistance and to output information.
• this is a natural language user interface and may comprise a keyboard, voice input-output subsystem, and/or a display.
• the user interface 710 can receive a request for assistance with a task, e.g. an overall task as described above, and can output instructions to the user to perform actions 114 to complete each of a series of sub-tasks of the overall task.
• the digital assistant device 700 can include an observation capture subsystem 750, and a language model interface 740 for the language model neural network 120, which may also be implemented locally or remotely.
• the observation capture subsystem 750 may be similar to the observation capture subsystem 610 described above, and can capture visual and/or audio observations of the human user performing a task, and provide corresponding observation data to the language model interface 740.
• the input token string may additionally include text tokens representing words and/or characters that are intended to guide the language model output.
• the natural or computer language question may include prompt language or tags, e.g. to prepend “Q:” or “Question:” or to append “?”, “A:” or “Answer:_”.
• the prompt language or tags may also be converted into text tokens of the input token string.
• successful performance of the task may be defined by a goal observation, such as a goal image or goal sound, or by a goal trajectory defining a target movement.
• a goal image may be used show the environment, e.g. an object in the environment, in a particular state, or it may show a desired trajectory; or a goal video may illustrate a goal behavior or trajectory.
• a goal sound may indicate when the environment, e.g. an object in the environment, has achieved a particular state, for example the click sound of a male Ethernet connector when it is fully inserted into its socket.
• obtaining the information defining the task can comprise obtaining goal observation data defining a goal state or states of the environment for successful performance of the task.
• the goal observation may be referenced in the input token string, e.g. by including a tag that indicates to the language model neural network that the goal observation is part of the natural or computer language question.
• a tag may be included, for example, at the start of the natural or computer language question.
• an “ ⁇ image>” tag may be used to reference a goal image, followed by a question such as “Did the agent successfully achieve this goal state?”.
• a token for the tag may be included in the token vocabulary.
• the input token string including the tag referencing the goal observation may be provided as a prompt to the multimodal language model neural network before the language model neural network processes the observation data from the environment. That is the multimodal language model neural network may process the input token string, and the goal observation data from the environment, and then the observation data from the environment, prompting the multimodal language model neural network 120 in a way that is broadly analogous to prompting a large language model.
• the language model neural network is configured to jointly model the input token string 122 and the observation data to determine the language model output 124.
• the multi-modal language model neural network 120 may comprise a pre-trained visual language model (VLM), of which there are many examples.
• VLM visual language model
• FIG. 8 shows details of one way in which the multi-modal language model neural network 120 can be implemented.
• the compressed representation can comprise a set of observation tokens 812; this may comprise a fixed number of tokens irrespective of the observation data, e.g. a fixed number of tokens per image or video clip.
• the observation tokens 812 can be determined by processing the observation data using the observation encoder 814 to generate encoded observation data comprising a set of observation features, e.g. visual features. For example an image or each image of a video clip may be encoded in this way.
• the set of observation tokens 812 can then be determined from the encoded observation data by processing a set of (learned) latent vectors 818 using one or more observation encoder cross-attention layers configured to cross-attend to the observation features, to generate the set of observation tokens, e.g.
• the set of latent vectors comprises a set of predetermined latent vectors.
• This approach can be used for example to generate a (fixed size) set of visual tokens from an image or video clip.
• the language model neural network 800 can include a token embedding subsystem that applies an embedding function to each token of the input token string to transform it to a respective token embedding, i.e. a vector of numeric values, prior to processing the input token string using the stack of processing layers.
• a token embedding subsystem that applies an embedding function to each token of the input token string to transform it to a respective token embedding, i.e. a vector of numeric values, prior to processing the input token string using the stack of processing layers.
• Such an embedding function may be fixed or learned.
• a positional encoding can be applied, e.g. added or concatenated, to the token embeddings to indicate a position of each respective token in the input token string 122.
• a pre-trained model e.g. a VLM
• a VLM may be used for the language model neural network 120 of FIG. 1, because such pre-trained models can be capable of performing a question answering task, e.g. a visual question answering task, of the type used herein for task success detection. Nonetheless in some implementations such a pretrained model can be “fine-tuned”, i.e. further trained, using additional training data items. These additional training data items may be specific to task success detection. Such an additional training data item can include an observation of a task, a natural or computer language question asking whether the task was achieved, and an answer appropriate to the observation.
• the multimodal language model neural network 120 can be prompted with one or more examples of task success detection, e.g. comprising an image, question text, and example answer text.
• the (additional) training data items can be generated by manually annotating observations of tasks to indicate whether the task was competed. For observations that extend in time, such as video or audio clips, the manual annotation can also indicate, if the task was competed, when success occurred. For example, for a video or audio clip observation this can be done by splitting the clip into non-overlapping subsequences, before and after the point of success. Two (additional) training data items can then be generated, each including a question as to whether the task was successfully completed, and each including an appropriate ground truth answer, e.g. “yes” if the subsequence ends in one or more success frames, and “no” otherwise. With multiple human raters success or failure can be determined by a majority vote; and the point of success can be determined as the median of the first annotated success frame across the raters.
• These (additional) training data items can include observations of environments or tasks that are similar to, or the same as, those for which the system will later be used, although this is not essential.
• a task description “ ⁇ task ⁇ ”
• the question can be generated using a template such as “Did the robot/agent/person successfully ⁇ task ⁇ ”.
• a trained VLM such as Flamingo (arXiv:2204.14198) can be used to convert the narration into a question, e.g. to convert a narration such as “The person is scooping the ice cream” into “Did the person successfully scoop the ice cream?”.
• a training dataset of (additional) training data items was generated by using human operators to provide 101,789 demonstrations of 6 tasks using a 6DoF control device, a Panda robot arm (Franka Emika GmbH).
• Each episode was then annotated by humans with rewards for each task, each episode having 6 reward annotations, one for each task, labelling with a positive reward all frames with a success state, i.e. if the task is solved, and labelling with zero reward otherwise (if a task is accidentally undone within an episode, at that point the reward annotation reverted to zero).
• the reward annotations and corresponding episode frames were then converted into training data items.
• FIG. 9 taken from this training dataset, shows an example of generating two (additional) training data items 910, 912 from comprising an annotated behavior trajectory video 900 for a task, with a point of success 902.
• Each training data item 910, 912 includes a natural language question, a part of the video indicating respectively failure or success of the task, and a corresponding answer to natural language question, respectively “no” or “yes”. Where the answer is “yes” the part of the video in the training data item contains the success frame(s) or follows the success frame(s).
• FIG. 10 is a flow diagram of an example process for training a multimodal language model neural network, such as the language model neural network 800 of FIG. 8.
• the process of FIG. 10 can be performed by a system of one or more computers located in one or more locations, e.g. by the computer system 100 of FIG. 1.
• model e.g. VLM
• VLM multimodal language model neural network
• the model is trained, e.g. fine-tuned, for task success detection this can be done in a way that is appropriate to training the model generally.
• the multimodal language model neural network may comprise a model, such as a transformer-based model, that is configured to process a sequence of tokens comprising tokens from the input token string and observation tokens.
• the observation tokens may comprise image tokens, e.g. from encoding an image or patches of an image, or multimodal tokens.
• a model can process, and be trained on, a mixed sequence of such tokens.
• a model can be natively trained on a combination, e.g. sequence, of text tokens and observation tokens (such as image tokens) provided as an input to the model.
• such training can involve backpropagating gradients of a loss function, e.g. as described below.
• such a model may also include a crossattention mechanism as described below.
• training the multimodal language model neural network 800 starts from a pre-trained and frozen text-only language model. However it is not essential to start from a pre-trained and frozen text-only language model.
• the above described token processing layers 830 may be obtained from the pre-trained and frozen text-only language model, e.g. from an autoregressive model incorporating such layers.
• the language model can be a so-called large language model, e.g. having greater than IO 10 , 10 11 or 10 12 trainable (trained) parameters. There are many such models; merely as one example the “Chinchilla” model may be used (Hoffmann et al., “Training Compute-Optimal Large Language Models”, arXiv:2203.15556).
• Such a text-only language model can be trained on the very large quantities of unlabeled text available e.g. in books and on the Internet.
• the multimodal language model neural network 800 can then be trained whilst keeping the neural network parameter values, e.g. weights, of the token processing layers 830 frozen, i.e. constant, during the training.
• this example training process may comprise obtaining a trained generative natural or computer language neural network (step 1000).
• the generative natural or computer language neural network is configured to process a string of input tokens representing words in a natural or computer language, to generate output tokens for the natural or computer language, and comprises a stack of trained token processing layers.
• the process can then form a multimodal language model neural network (step 1002).
• the multimodal language model neural network is configured to process i) a sequence of text tokens representing words in a natural or computer language, and ii) observation data comprising an observation of the environment, to generate a language model output comprising a string of one or more output tokens, e.g. one at a time.
• the process can further comprise obtaining a set of training data items (step 1004).
• Each training data item can comprise an observation of an environment, and natural or computer language text relating to the observation.
• the process can train the multimodal language model neural network, using the training data items, by adjusting parameters of the cross-attention layers. In some implementations, but not necessarily, this is done whilst keeping parameters of the trained token processing layers frozen, i.e. constant (step 1006).
• the multimodal language model neural network can be trained to predict the text tokens representing the natural or computer language text in the training data item, whilst conditioned on observation data derived from the observation in the training data item.
• Training the multimodal language model neural network can involve backpropagating gradients (with respect to the parameters of the cross-attention layers) of a loss function, through the trained token processing layers 830.
• the loss function can be one that encourages the multimodal language model neural network to generate a language model output 124, in particular an output token string representing the natural or computer language text, that matches the ground truth language corresponding to an observation, in a training data item. This can involve processing, using the multimodal language model neural network, observation data derived from the observation in the training data item and an input token string representing the natural or computer language text for the observation.
• the output token string may be generated one token at a time.
• the loss function may be based on p(y x) where x denotes the observation data (in particular, the above described compressed representation), L is the number of tokens in the input token string, y t denotes the Z-th token of the input token string, y ⁇ t denotes the I — 1 preceding characters of the input token string, and where p (y t
• the loss function may comprise a negative log-likelihood term — y ⁇ i> x ⁇ i), e.g. averaged over a minibatch of training data items.
• training the multimodal language model neural network also involves training, i.e. adjusting parameters of, the above described observation encoder neural network 814.
• a pre-trained and frozen observation encoder neural network may be used, e.g. one trained based on contrastive learning that does not require labelled data.
• a gating parameter is used for the cross-attention layers, this may be gradually increased during training to gradually increase the influence of the cross-attention layers.
• the multimodal language model neural network may be trained partially or wholly in simulation, i.e. using simulated training data representing a simulation of a real-world environment, before the multimodal language model neural network is used to process observations from the real-world environment that was simulated.
• the simulation need only be approximate as one advantage of the described techniques is that they are able to generalize to novel tasks, environments, behaviors, and language.
• FIGS. 12A-C illustrate evaluations of success detection performance when varying camera viewpoint, and when including one or more distractor objects.
• FIG 12A illustrates a video clip from a front camera, as used in the training data in this particular implementation (a baseline);
• FIG. 12B illustrates a video clip from a rear camera (absent from the training data);
• FIG. 12C illustrates a video clip from a test including distractor objects (pegs).
• the baseline success detection performance for an “insert” task of the type illustrated was in the range 90-95%, falling by around 5% with the change to a rear camera viewpoint, and exhibiting little change when distractor objects were present.
• the robustness of the described techniques to variations of this type in the task, agent and environment e.g. visual changes such as camera view angles, lighting conditions, and background changes) facilitates effective training, evaluation, and use of the action selection system 110 as previously described. Implementations of the system can thus provide an action selection system 110 that has an improved capability of acting in a real -world environment
• implementations of the described techniques are also able to detect successful performance of a task by a human, and hence can be used to guide a human performing a task.
• a system can be trained, e.g., using the publicly available Ego4D dataset of egocentric human-in-the-wild videos.
• the videos show people executing common tasks (e.g., washing dishes, cleaning cars, gardening), and the Ego4D Forecasting + Hands & Objects (FHO) dataset has corresponding narrations that describe the actions of the camera wearer in the videos.
• Other annotations include so-called critical state changes, i.e.
• FIG. 13 illustrates generating (additional) training data items for human task performance success detection from the Ego4D dataset, for a task of rolling dough.
• the agent 102 is a mechanical agent
• the environment 104 is a real -world environment
• the observations, in particular the observations processed by the action selection system 110 and the observation from which the observation data is derived for the language model neural network 120 are from one or more sensors sensing the real -world environment.
• the actions 114 are for controlling the mechanical agent acting in the real-world environment to perform the task, and can be used to control the agent 102 in the real -world environment.
• the agent 102 is a simulation of a mechanical agent
• the environment 104 is a simulation of a particular real -world environment
• the observations relate to the particular real-world environment
• the actions relate to actions to be performed by the mechanical agent acting in the particular real -world environment to perform the task.
• the process can then use the mechanical agent in the particular real-world environment to perform the task. That is, training or evaluating of the action selection system may be partly or wholly performed in simulation before using the action selection system in the real world.
• the agent 102 can be a mechanical agent interacting with the real -world environment, such as a robot or an autonomous or semi-autonomous land, air, or sea vehicle operating in or navigating through the environment, and the actions are actions taken by the mechanical agent in the real-world environment to perform the task.
• the agent may be a robot or vehicle interacting with the environment to accomplish a specific task, e.g., to locate an object of interest in the environment, or to manipulate or change the state of a specified object, or to move a specified object to a specified location in the environment, or to navigate to a specified destination in the environment.
• the task may be characterized by an ongoing behavior, e.g. “vacuum the house”.
• the information defining the task e.g. the object, state, location, destination, or behavior, may be specified by a natural or computer language or other instruction, e.g. from a user interface of the system, as previously described.
• the observations processed by the action selection system 110, and/or by the language model neural network 120 can be derived from one or more sensors of the environment 104, for example from a camera or other image sensor or from a microphone.
• the sensor(s) may be mounted on the agent and/or located separately from the agent in the environment.
• the observations may comprise various types of sensor data as well as or instead of that previously described, e.g. object position data, data from a distance or position sensor, data from an actuator, or sensed electronic signals such as motor current or a temperature signal.
• the observations may also include data characterizing the current state of the mechanical agent or robot, e.g., one or more of: joint position, joint velocity, joint force, torque or acceleration, e.g., gravity-compensated torque feedback, and global or relative pose of an item held by the robot or of one or more parts of the agent.
• the observation at any given time step may include (observation) data from a previous time step that may be beneficial in characterizing the environment.
• the actions 114 may comprise control signals to control the robot or other mechanical agent; a control system can be used to generate control signals for controlling the mechanical agent.
• the control signal may comprise, e.g., torques for the joints of the robot or torques to a control surface or other control elements e.g. steering control elements of a vehicle, or higher-level control commands. More generally the control signals can include, for example, position, velocity, or force, torque, or acceleration data for one or more joints or other parts of a robot or other mechanical agent.
• the control signals may also or instead include electronic control data such as motor control data, or signals to control navigation, e.g., steering, movement, braking, and/or acceleration of the agent.
• This specification uses the term “configured” in connection with systems and computer program components.
• a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions.
• one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.
• Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.
• Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non transitory storage medium for execution by, or to control the operation of, data processing apparatus.
• the computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
• the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.
• data processing apparatus refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.
• the apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
• the apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
• a computer program which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code, can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
• a program may, but need not, correspond to a file in a file system.
• a program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code.
• a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.
• the term “database” is used broadly to refer to any collection of data: the data does not need to be structured in any particular way, or structured at all, and it can be stored on storage devices in one or more locations.
• the index database can include multiple collections of data, each of which may be organized and accessed differently.
• engine is used broadly to refer to a softwarebased system, subsystem, or process that is programmed to perform one or more specific functions.
• an engine will be implemented as one or more software modules or components, installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines can be installed and running on the same computer or computers.
• the processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output.
• the processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.
• Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit.
• a central processing unit will receive instructions and data from a read only memory or a random access memory or both.
• the elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data.
• the central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
• a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks.
• a computer need not have such devices.
• a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.
• PDA personal digital assistant
• GPS Global Positioning System
• USB universal serial bus
• Computer readable media suitable for storing computer program instructions and data include all forms of non volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks.
• semiconductor memory devices e.g., EPROM, EEPROM, and flash memory devices
• magnetic disks e.g., internal hard disks or removable disks
• magneto optical disks e.g., CD ROM and DVD-ROM disks.
• embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer.
• a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
• keyboard and a pointing device e.g., a mouse or a trackball
• Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
• Data processing apparatus for implementing machine learning models can also include, for example, special-purpose hardware accelerator units for processing common and compute-intensive parts of machine learning training or production, i.e., inference, workloads.
• Machine learning models can be implemented and deployed using a machine learning framework, e.g., a TensorFlow framework.
• Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back end, middleware, or front end components.
• the components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.
• LAN local area network
• WAN wide area network
• the computing system can include clients and servers.
• a client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
• a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client.
• Data generated at the user device e.g., a result of the user interaction, can be received at the server from the device.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• Data Mining & Analysis (AREA)
• General Health & Medical Sciences (AREA)
• Biomedical Technology (AREA)
• Biophysics (AREA)
• Computational Linguistics (AREA)
• Life Sciences & Earth Sciences (AREA)
• Evolutionary Computation (AREA)
• Artificial Intelligence (AREA)
• Molecular Biology (AREA)
• Computing Systems (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Mathematical Physics (AREA)
• Software Systems (AREA)
• Health & Medical Sciences (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=89619094

Family Applications (1)

Country Status (3)

Families Citing this family (3)

• 2024
  • 2024-01-05 WO PCT/EP2024/050243 patent/WO2024146961A1/en not_active Ceased
  • 2024-01-05 CN CN202480011683.2A patent/CN120712567A/zh active Pending
  • 2024-01-05 EP EP24700225.6A patent/EP4627480A1/de active Pending

Also Published As

Similar Documents

Legal Events