CN111601684B

CN111601684B - Sharing learned information between robots

Info

Publication number: CN111601684B
Application number: CN201880084815.9A
Authority: CN
Inventors: N.拉吉库马尔; P.勒格; N.哈德逊; K.尚卡尔; R.赫斯默
Original assignee: X Development LLC
Current assignee: X Development LLC
Priority date: 2017-12-27
Filing date: 2018-09-13
Publication date: 2023-05-12
Anticipated expiration: 2038-09-13
Also published as: CN116484972A; US20190197396A1; US11475291B2; EP3713719A1; CN111601684A; US20230004802A1; WO2019133081A1

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for sharing learned information between robots. In some implementations, the robot obtains sensor data indicative of a characteristic of the object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedded and data indicative of the classification of the object. The robot transmits the generated embedded and classified data to the server system. The robots receive the embeddings and corresponding classifications generated by the second robot from the server system. The robot stores the received embedments and corresponding classifications in a local cache of the robot. The robot may then use the information in the cache to identify the object.

Description

Sharing learned information between robots

Technical Field

This specification relates generally to sharing learned information between robots.

Background

The robot may be configured to perform a variety of tasks. Robots generally require some form of perception to accomplish a task, such as recognition or identification of an object. Robots may encounter objects and situations where their programming or training does not prepare them for proper handling. For example, while a robot may be able to recognize many different types of objects, it is likely that the robot will likely encounter new objects that it does not recognize.

Disclosure of Invention

In some implementations, the robot may perform a process to learn new information and new capabilities from the situations it encounters. The techniques disclosed herein may enable a robot to incorporate new information such that the robot learns almost instantaneously from user feedback or other sources of information. For example, the robot may have a local cache in which certain types of learned information are stored. When the robot obtains new information, e.g., a classification for a previously unknown object, the robot may store a representation of the new information in a cache so that the information is immediately available to the robot. This technique may allow near instantaneous learning of the robot because the computational requirements of incorporating new information, e.g., generating a representation and saving it in a cache accessed by the robot system, are extremely low. For example, the representation may have been calculated as part of a continuous on-board inference process for the robot. As a result, when the user provides the robot with a classification of previously unknown objects, the robot can store this information and apply it to current and future tasks with minimal delay.

Furthermore, individual robots may share learned information with other robots, allowing each robot in the cluster of robots to quickly acquire combined learning from all robots in the cluster. The learned information may be frequently distributed to robots without retraining or modifying the machine learning model used by the robot. For example, each robot in the cluster may communicate with a server system to upload a representation of recently learned information, such as a classification of objects that the robot observed and previously unrecognizable, but have thereafter learned to properly classify. The server system collects the representations, processes them, and distributes the combined set of representations to robots in the cluster. This process of learning, uploading and distributing representations of information learned by robots may be performed continuously, e.g. periodically every hour, every day or every week, or continuously, i.e. representations are distributed between robots once learned. The process of sharing representations among robots allows for learning throughout a cluster in a quick and efficient manner. In particular, the newly learned information may be distributed and used by the robot in a much shorter time than would be spent updating the machine learning model to reflect the same information.

The learned information collected from the plurality of robots may be used to retrain or update a machine learning model, which may then be distributed to each robot. For example, the server may periodically update the machine learning model used by the robots, with the update interval being much longer than the interval at which the learning representation is shared between robots. For example, if the representation is updated daily between robots, the machine learning model may be updated weekly, bi-weekly, or monthly. The updated machine learning model may incorporate a combined set of robot learning that occurs throughout the fleet in a previous interval (e.g., last week or last month).

In some implementations, the server may send updated machine learning models to the fleet weekly or bi-weekly. The server may also provide updated learned information more frequently in embedded form or in other representations each time a new set of robotic learns is received by the server. Robots in the cluster may store this learned information in a cache and use it with the most recent version of the machine learning model. The overall perceived system behavior of each robot in the cluster may be fine-tuned daily or each time a new set of robot learns is provided to the robots for caching the received new information. Thus, the robot may use the updated machine learning model to incorporate learning from the cluster and may purge its local cache because the cached information has been incorporated into the model.

In general, many aspects of the knowledge and capabilities of a robot may be provided by a trained machine learning model, such as a neural network. While these models are powerful and often efficient to run, training or updating machine learning models is often a time consuming and computationally intensive process, and training processes frequently require a large set of training data to be effective. As an example, neural networks are often trained with iterative processes that use a number of training examples to determine appropriate neural network parameters. In some cases, complex neural network models, such as those configured to identify thousands of different types of items, may require days or even weeks to train, even when distributed computing techniques are used. In many instances, the individual robots have no battery power or computational power at all to update the training of the machine learning model in a reasonable amount of time, let alone in real-time. If the user provides the robot with a classification of a new object, it would be ideal for the robot to hold and use the classification from now on without delay, especially because the user can then give the robot a command about the object.

As discussed further above and below, the present techniques may enable robots to encode learned information, such as object characteristics and corresponding classifications, as discrete representations in a cache of the robot. The robot may be designed to use this cache in its perception process or other process so that saving the representation allows the robot to immediately apply new information to the task. Thus, an individual robot may be able to learn and expand its capabilities in real time. Furthermore, the representation may be shared between robots to allow all robots in a cluster to quickly and efficiently acquire and use new information. At the same time, while the robot uses the shared representation and thus has benefited from collective learning of the cluster, the remote computing system may update the machine learning model for the cluster based on the newly learned information. From time to time, the computing system distributes the updated model to the robots, wherein the updated model incorporates the learning compiled throughout the fleet over a period of time. However, due to the continuous sharing of representations, the robot may already apply the learned information long before it is incorporated into the model update.

In some implementations, the representation of the robotic learned information may be an embedded generated using a machine learning model. For example, the embedding may represent information derived from an output layer of the neural network model or from a hidden layer of the neural network model. When a set of input information, such as sensor data describing an object, is provided to a machine learning model, processing of the machine learning model may encode the information as being used directly as an embedded form, or further processed to generate an embedded form. As a result, the embedding may be a compressed representation of the object or other observation, where the specific values of the embedding may depend on the structure and training state used to generate the embedded machine learning model.

In one general aspect, a method includes: obtaining, by a robot, sensor data indicative of a characteristic of an object; determining, by the robot, a classification for the object; generating, by the robot, a first embedding for the object using a machine learning model stored by the robot; storing, by the robot, the generated first embedment and data indicative of a classification of the object in a local cache of the robot; transmitting, by the robot, the generated first embedded and classified data to the server system through the communication network; receiving, by the robot, the second embedment and the corresponding classification from the server system over the communication network; wherein the received second embedding is generated by a second robot and corresponds to a second object; and storing, by the robot, the received second embedment and the corresponding classification in a local cache of the robot.

These and other embodiments may each optionally include one or more of the following features, alone or in combination. For example, one embodiment includes a combination of all of the following features.

In some implementations, the method includes recognizing, by the robot, one or more objects in the environment of the robot using the first embedding or the second embedding in the local cache.

In some implementations, the method further includes generating, by the robot, sensor data describing a particular object in the environment of the robot.

In some implementations, the method includes generating, by the robot, an embedding for the particular object using the machine learning model and sensor data describing the particular object. The method includes comparing, by the robot, the embedding of the particular object with the first embedding and the second embedding.

In some implementations, the method includes assigning, by the robot, a classification to the particular object based on a comparison of the embedding of the particular object with the first embedding and the second embedding.

In some implementations, the method includes receiving, by the robot, a second machine learning model from the server system over the communication network.

In some implementations, the method includes recognizing, by the robot, the one or more objects using the second machine learning model.

In some implementations, the method includes removing, from a local cache of the robot, an embedding generated using the first machine learning model.

In some implementations, the method includes specifying the generated embeddings to share with other robots.

In some implementations, the method includes sending the generated embeddings with tags or metadata specifying that the embeddings should be shared or sending the generated embeddings using an application programming interface for sharing the embeddings.

In some implementations, the method further includes storing, by the robot, sensor data for generating input to the machine learning model for generating the embedding.

In some implementations, the method includes generating a first embedding based on processing feature data extracted from sensor data of an object using a machine learning model.

In some implementations, the machine learning model is a neural network model and the first embedding is derived from an output at an output layer of the neural network model.

In some implementations, the machine learning model is a neural network model and the first embedding is derived from data indicative of activation at a hidden layer of the neural network model.

In another general aspect, a method includes: receiving, by one or more computers, object classification examples from a plurality of robots, wherein each object classification example includes (i) an embedding generated by the robot using a machine learning model, and (ii) an object classification corresponding to the embedding; evaluating, by the one or more computers, the object classification examples based on similarity of the received embeddings of the object classification examples to other embeddings; selecting, by the one or more computers, a subset of object classification examples based on the evaluation of the embedded quality; and distributing, by the one or more computers, the subset of object classification examples to robots of the plurality of robots.

Implementations can include one or more of the following features. For example, in some implementations, for each object classification example, the embedding is generated by the robot based on sensor data generated by the robot describing objects detected by sensors of the robot, and the respective object classification of the embedding is the detected object indication classification. The embedding of the respective classification examples includes embedding generated by different robots, wherein each robot generates the embedding using a particular machine learning model having a particular training state. Distributing the subset of classification examples includes distributing the classification to a plurality of robots, each robot storing a copy of the same particular machine learning model having a particular training state.

In some implementations, distributing the subset of classification examples includes sending the classification examples in the subset to each of the plurality of robots and designating that the sent classification examples be stored in a local cache of each robot.

In some implementations, distributing the subset of classification examples includes synchronizing a local cache of robots of the plurality of robots such that each of the plurality of robots stores each classification example of the selected subset in the local cache of the robot.

In some implementations, the method includes: the second machine learning model is trained using the selected subset of classification examples and the trained machine learning model is distributed to robots of the plurality of robots.

In some implementations, training the second machine learning model includes generating an updated version for generating the embedded machine learning model. Distributing the trained machine learning model includes distributing an updated version of the machine learning model as a substitute for the machine learning model stored at each of the plurality of robots.

In some implementations, the method includes providing instructions to each of the plurality of robots to remove the distributed classification examples from the local cache of the robot.

In some implementations, the method includes, for each object classification example, receiving sensor data describing an object corresponding to the object classification example. Training the second machine learning model includes training the second machine learning model based on the sensor data received for the object classification example.

In some implementations, the method includes, for each object classification example, receiving sensor data generated by the robot describing an object corresponding to the object classification example. Evaluating the object classification example includes evaluating a quality of the classification example based on the received sensor data.

In some implementations, evaluating the object classification example includes, for a particular embedding of a particular object classification example, determining a level of similarity of the particular embedding relative to one or more other embeddings having the same object classification as the particular object classification example. Selecting the subset of object classification examples includes selecting a particular object classification example to include in the subset based on determining that a level of similarity satisfies a threshold.

In some implementations, evaluating the object classification example includes determining, for a particular embedding of a particular object classification example, a level of similarity of the particular embedding relative to one or more other embeddings having object classifications that are different than the particular object classification example. Selecting the subset of object classification examples includes excluding the particular object classification example from the subset based on determining that the level of similarity satisfies a threshold.

In some implementations, evaluating the object classification example includes determining, for a particular embedding of the particular object classification example, a level of similarity of the particular embedding relative to one or more other embeddings, wherein the level of similarity is indicative of a distance between the particular embedding and the one or more other embeddings in the vector space.

In some implementations, evaluating the object classification example includes determining, for a particular embedding of a particular object classification example, a quality score that indicates a likelihood that the object classification matches an object corresponding to the particular object classification example. Selecting the subset of object classification examples includes selecting a particular object classification example to include in the subset based on the quality score meeting a threshold.

In some implementations, evaluating the object classification examples includes determining a quality score for a particular embedding of a particular object classification example based on: analyzing a quality level of sensor data provided by the plurality of robots; analyzing how the object classification examples adjust the overall accuracy of the machine learning model; and analyzing previous levels of quality of other object classification examples provided by the plurality of robots.

In some implementations, analyzing the level of quality of the sensor data provided by the plurality of robots further includes: determining a match between the sensor data and the embedding by providing the sensor data to a machine learning model; comparing the output embedding from the machine learning model to the embedding; comparing the output classification labels from the machine learning model with the object classifications corresponding to the embeddings; and retraining the machine learning model to detect the embedding and to generate a correct classification label matching the object classification in response to determining that the output classification label from the machine learning model does not match the object classification corresponding to the embedding.

In another general aspect, a method includes: storing, by the robot, a first embedding generated using a first machine learning model, wherein the robot stores the first embedding in a cache of the robot, wherein the first embedding includes one or more first private embeddings that are not shared with other robots; receiving, by the robot, a second machine learning model from the server system over the communication network; generating, by the robot, a second privacy embedding for each of the one or more first privacy embeddings in the cache, wherein each of the second privacy embeddings is generated using a second machine learning model; embedding and adding a second secret by the robot into a cache of the robot; and removing, by the robot, one or more first privacy embeddings from the cache of the robot.

Implementations can include one or more of the following features. For example, in some implementations, the first embedment includes one or more common embedments shared with the server system. The method comprises the following steps: determining that the particular embedding is a public embedding; and after receiving the second machine learning model, removing the particular embedding from the cache of the robot based on determining that the particular embedding is a common embedding.

In some implementations, the method includes storing, by the robot, sensor data for generating one or more first privacy embeddings. Generating a second privacy embedding for each of the one or more first privacy embeddings in the cache includes generating each of the second privacy embeddings from the stored sensor data for generating the one or more first privacy embeddings using the second machine learning model. The method further includes storing the second secret in a cache of the robot.

In some implementations, generating the second privacy embedding from the stored sensor data includes generating the second privacy embedding for the object by inputting feature values derived from the sensor data used to generate the first privacy embedding for the object to the second machine learning model.

In some implementations, the method includes: accessing, by the bot, metadata associated with a particular embedding of the first embedding; and determining that the particular embedding is a private embedding that is not shared with the server system by analyzing metadata associated with the particular embedding against one or more sharing rules stored by the bot.

In some implementations, the first embeddings include one or more first public embeddings in addition to one or more first private embeddings. The method includes providing, by the robot, one or more first public embeddings to the server system and not providing one or more first private embeddings to the server system.

In some implementations, the method includes: determining, for each of the first embeddings, whether to store sensor data for generating the first embeddings, wherein determining whether to store sensor data for generating the first embeddings is based on whether the first embeddings are private or public; and wherein upon the determination, storing, by the robot, sensor data for each of the one or more first private embeddings and not storing sensor data for the one or more first public embeddings.

In some implementations, the first embeddings generated by the first machine learning model include one or more received embeddings received by the robot from the server system over the communication network. The method further includes determining that the particular one of the first embeddings is one of the received embeddings; and after receiving the second machine learning model, removing the particular embedding from the cache based on determining that the particular embedding is one of the received embeddings.

In some implementations, the cache is a local cache stored on a data storage device of the robot.

In some implementations, the first machine learning model is a first neural network model and the second machine learning model is a second neural network model.

In some implementations, the method includes: obtaining, by a robot, sensor data describing an object; generating, by the robot, an inlay for the object based on the sensor data; determining, by the robot, whether to designate the embedding as private or public; and storing, by the robot, the embedding and data specifying whether the embedding is private or public.

In some implementations, determining whether to designate the embedding as private or public includes: determining a position of an object or robot; determining that the location is within a predetermined area that does not share information; and designating the embedding as private embedding based on determining that the location is within the predetermined area.

In some implementations, determining whether to designate the embedding as private or public includes: determining an object type of an object observed by the robot; determining that the object type is one of a predetermined set of object types; and determining that the embedding is to be designated as a private embedding based on determining that the object type is one of a predetermined set of object types.

In some implementations, determining whether to designate the embedding as private or public includes: a sharing rule is accessed that indicates one or more restrictions on sharing of information obtained by the robot, wherein the one or more restrictions limit sharing of information as follows: (i) Information observed by the robot at a certain time or place, or (ii) information describing an object having a certain object type or object property; determining sharing of the object restriction information described by the sharing data by the sharing rule based on the sensor data of the object; and designating the embedding of the object as private embedding based on determining that the sharing rules restrict sharing of information for the object.

In some implementations, the method includes receiving user input indicating one or more criteria including limiting a time, a place, an object type, or an object characteristic of sharing of information; and storing sharing rules that designate embedding of objects that meet one or more criteria as private embedding. Determining whether an embedding is to be designated as private or public includes determining whether the embedding meets one or more criteria.

Other embodiments of these and other aspects of the disclosure include corresponding systems, apparatus, and computer programs, encoded on computer storage devices, configured to perform the actions of these methods. The system of one or more computers may be configured such that, in operation, the system performs these actions by virtue of software, firmware, hardware, or a combination of them being installed on the system. One or more computer programs may be configured with instructions that, when executed by a data processing apparatus, cause the apparatus to perform these actions.

The details of one or more embodiments of the subject matter of the specification are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, the drawings, and the claims.

Drawings

Fig. 1 is a block diagram illustrating an example of a system for sharing learned information between robots.

FIG. 2 is a block diagram illustrating an example of a system for training a robot to recognize objects.

Fig. 3 is a block diagram illustrating an example of a robot recognizing an object.

Fig. 4 is a block diagram illustrating an example of another system for sharing learned information between robots.

Fig. 5 is a block diagram illustrating an example of a system for providing an updated machine learning model to a robot.

Fig. 6 is a flowchart illustrating an example of a process for sharing learned information between robots.

Fig. 7 is a block diagram illustrating an example of a system for selectively sharing learned information between robots.

Fig. 8 is a block diagram illustrating an example of a system for evaluating the quality of information learned by a robot.

Fig. 9 is a block diagram illustrating an example of a system for testing a machine learning model.

Fig. 10 is a flowchart illustrating an example of a process for evaluating and sharing information learned by a robot.

FIG. 11 is a diagram illustrating an example system for enhancing robotics learning.

Fig. 12 is a diagram illustrating an example system for selectively sharing information between clusters of robots.

Fig. 13 is a diagram illustrating an example system for robot regeneration and storage of an embedding.

Fig. 14 is a flow chart illustrating a process for enhancing robotics learning.

Like reference numbers and designations in the various drawings indicate like elements.

Detailed Description

Fig. 1 is a block diagram illustrating an example of a system 100 for sharing learned information between robots. The system 100 includes robots 104A-104D, client devices 108, a network 110, a server system 112, and a database 114. Server system 112 may include one or more computers connected locally or through a network, which may include one or more public or private networks and may include the internet. The example shows a human user 106 interacting with robots 104A-104D. The example of FIG. 1 shows robots 104A-104D in a home 102, but robots 104A-104D may alternatively be located in different homes or other types of buildings or areas. Fig. 1 illustrates various operations in stages (a) to (E) that may be performed in the indicated sequence or in another sequence.

Generally, a robot refers to a programmable machine that may be capable of performing a series of actions at least partially autonomously. For example, a robot may be able to perform tasks related to interacting with or changing the physical perimeter of the robot (e.g., arranging or manipulating objects). In addition to responding to inputs and commands from humans, the robot may initiate actions by itself based on its programming and conditions it detects. The robot may include components for interacting (e.g., physically manipulating) with objects in its environment. For example, a robot may include an arm with actuators to grasp or otherwise move an object. For example, the actuator may be a manipulator with finger-like elements to grasp an object. The robot may also include components that allow movement. Examples include motorized elements, such as wheels, legs, continuous tracks, etc., to allow the robot to travel from one location to another. The robot may include a navigation component that allows it to set a trip and travel along a self-guided path. The robot may include sensory capabilities that allow the robot to perceive its surroundings. The robot may include body components such as a chassis and other connection components. The body components may allow various types of movement of the robotic components relative to each other, such as rotation, swiveling, displacement, and the like. The robot may include various other components such as a battery to power the robot, transmitters, receivers, sensors, a data processor, and a memory to store programmed instructions for the robot.

In the example of fig. 1, the cluster of robots uses distributed learning to learn to identify new objects. The robot clusters do not require that each robot in the cluster be trained by an individual to identify new objects. Instead, robot clusters employ distributed learning techniques in which one robot can learn to identify an object and produce an embedment corresponding to the object. The robot may share the embedding with other robots over a communication network in a manner that allows all robots in the cluster to identify the newly identified object. Providing embedded and corresponding classification information allows the robot receiving the data to learn to classify the object instantaneously. The technique may use a machine learning model to generate an embedding that describes characteristics of the identified object. This may be beneficial because the user only needs to train one robot to identify an object, not every robot. Furthermore, sharing embeddings among robots saves processing resources by allowing robots to learn without having to retrain a machine learning model on each robot.

Any of various types of machine learning models may be used. Examples include neural networks, maximum entropy classifiers, decision trees, random forest classifiers, boosting trees, logistic regression models, and reinforcement learning models. The machine learning model may be trained using supervised or unsupervised training. The machine learning model may have parameters that are learned or trained based on a variety of different examples, which may be from a variety of sources.

In some implementations, sharing may be performed to all robots in the cluster. Alternatively, sharing may be performed to robots within a particular organization. For example, robot 104A may be associated with a residence and robot 104B may be associated with a factory. Further, sharing may be performed to robots in a particular domain. For example, robot 104C may be a hospital robot and robot 104D may be a hotel robot. In some implementations, sharing may be performed between specific groups of robots via access control lists managed by server system 112. In other words, the server system 112 may selectively share learned information only among robots having some commonalities, such as the same hardware model, the same role or responsibility, the same type of location, the same owner, the same group or cluster identifier, and so forth.

In some implementations, each robot in the robot cluster is located in a single location, such as a home 102 or a factory. In other implementations, robots in a robot cluster are located in different locations or settings. For example, robot 104A may be located in residence 102; robot 104B may be located in a residence opposite the street of residence 102; the robot 104C may be located in a grocery store; and robot 104D may be located at an airport. Robots located at different locations may communicate with each other through a communication network 110. Robots in a cluster may represent a large group of robots distributed in many locations, such as all robots provided by a manufacturer, that are owned or used by many different users in different settings.

In some implementations, robots in a robot cluster are programmed to assist humans in the home. For example, a human may instruct a command to a first robot to request retrieval of the human's cellular telephone. In response, the first robot may search for a human cell phone in the home, identify the cell phone by distinguishing the cell phone from other home items, and return the cell phone to the human. While the robot may include an object recognition model to enable the robot to recognize household items, the trained model may not be able to classify some types of objects. When the object recognition model does not enable the robot to recognize the object, a human user may train the first robot with respect to the characteristics of the object to allow the first robot to locate and return to the object. Once the first robot has learned the recognition object, it may share the learned information with other robots. Then, if the user requires the second robot to retrieve the same object or the same type of object, the second robot may use the data received from the first robot to identify the object. In this way, the second robot may still recognize the object based on shared learning from the first robot even though the perception model of the second robot does not recognize the object and even though the second robot has never been trained directly to recognize the object.

The process of distributed learning between robot clusters can be performed quickly by sharing embeddings among robots. An embedding may be a collection of values such as vectors, codes, files, or other data representing a characteristic of an object or a fingerprint of an object. The embedding may be generated by providing data describing the object (e.g., features extracted from sensor data describing the object) to a perception model, such as a machine-learned object recognition model. For example, the embedding may be a multidimensional vector or number representing characteristics of the encoded version of the object, where the encoding is based on processing of object feature data using trained machine learning. Robots that encounter a new object and learn the correct classification share embeddings about the object with the rest of the cluster, so that each robot does not need to be trained individually to identify the object. As a result, when a human trains a single robot to identify an object, the system enables other robots to also identify the object, as a result of receiving the corresponding embedded and classified information.

For example, a human may train a first robot to identify a coffee cup. A person may lift the coffee cup in front of the first robot to have the sensor of the first robot capture the characteristics of the coffee cup. The first robot may acquire images of the coffee cup from various angles to determine characteristics of the coffee cup. The first robot may then use these characteristics to generate an inlay that is stored in a local cache of the first robot along with data indicating that the inlay represents a coffee cup. Once the embedment and classification are stored, the first robot will be able to use this new information immediately to recognize the cup in the future. As a result, the next time a human requests the first robot to retrieve a coffee cup, the first robot may identify and retrieve the coffee cup by comparing the stored embedment for the coffee cup with the embedment generated by the first robot from live sensor data describing the current environment of the first robot.

In general, robotic learning techniques allow for immediate local learning of a single robot, as well as shared learning with other robots. The user may unlock the object type from a master list of items (e.g., a knowledge graph) for use on a single robot. If the robot is still unable to properly identify the object after unlocking the object type, the user may teach the robot to classify the object. This process may involve exposing an instance or "instance" of the object type to the robot. This example of learning is referred to as "teaching" and can be expressed as embedding and corresponding classification. The teaching acts immediately on the robot being taught. In addition, the robot periodically uploads the teachings it receives to the server. The same is done with other robots. The teachings from all robots are aggregated by the server and the existing model is fine-tuned. Fine tuning occurs in two ways. First, a set of teachings, such as embedding and sorting, can be pushed daily to all robots throughout the cluster. All robots in the cluster can now detect updated objects based on the teachings from the individual robots. Second, a model that is fully retrained, such as a machine learning model that is retrained based on aggregate teachings from all robots, may be pushed to the robots every week or every two weeks.

When the robot encounters an unknown object, the user may interact with the robot to unlock the object type from a master list of known object types (e.g., make it available). The user associates this unlocked object type to the unknown object encountered by the robot. The robot can now detect this newly unlocked object of the object type. In general, robots may use perception systems designed to classify many types of objects. However, to reduce the risk of false positives, the individual robots may be configured to initially detect only a subset of the total set of object types. For example, a master list of object types may include tens of thousands or more different object types, while an individual robot may have some subset, e.g., only one hundred or one thousand of these object types, unlocked and available for easy detection. If the robot does not detect the object properly, the user may be presented with an interface to select the appropriate item from the master list so that it will be unlocked and available for detection by the robot. This may allow a user to extend the range of objects that their robot can detect through a remote interface such as an application or web page

However, in some examples, simply unlocking the object type may not be sufficient to allow the robot to reliably detect the object and avoid false positives. To improve performance, and in particular correct errors in robots, a user may identify instances of object types and enable sample images or other sensor data to be acquired. For example, a user may use his cellular telephone to capture images of an object from different sides while tracking the camera position of each image. As another example, after the user indicates the object to the robot, the robot acquires the appropriate image. During teaching of a new object, the robot can ensure that the object is seen from all sides, at various distances and under different lighting conditions. To this end, the robot may need to: driving to different locations around the object to see different sides of the object with different illumination; panning and tilting the head of the robot (e.g., the sensor assembly) to capture objects in different portions of the image, which results in a change in the 2D appearance of the object caused by perspective deformation; the head is panned and tilted to capture the color/brightness changes caused by the overall image exposure.

Image snapshots and corresponding embeddings can be generated continually until the robot determines that the object can again be identified deterministically. When the information is sufficiently different from the existing behavior of the model, the embedding may be saved as new learning information. For example, after a vector distance between a standard embedded vector of an object and an embedded vector of a recently taught example exceeds a threshold, the embedding may be saved as a new learning example. At this point, new embedded vectors are added as examples of teaching, whether for new object types that are not in the master list, for locked object types, or to refine the error for already unlocked object types.

The user may indicate to the robot to stop identifying one or more previously learned object types from the provided list. The user may deselect any previously learned object types as shown in the provided list. For example, the user may issue a voice command, interact with a display of the client device 108 in communication with the robot, or interact with a screen of the robot, to cancel selecting any previously learned object type to be unrecognized. The user may deselect the previously learned object type because the object is no longer present in home 102. Further, the user may wish to deselect one or more previously learned object types because home 102 includes a new version of the object type. For example, the user may deselect the scoops from the provided list because the user brings an updated scoop to home 102. The updated scoop includes different look head and handle shapes that the robot may not be able to detect using information from previously learned scoop objects.

In response to receiving a deselect from the user, the robot may lock the deselected object type. In some implementations, the robot can store a locked list of previously learned object types in memory. The robot will not recognize any objects listed in the locked list. For example, if the robot retrieves an older scoop model, the robot will not be able to identify the older scoop model as a scoop.

In some implementations, the user can unlock a previously locked learned object type. The user may request that the robot provide a list of previously learned object types including locked and unlocked object types. The robot may provide the list to a local display or to a display of the client device 108. The user may choose to unlock a previously learned object type so that the robot may identify the previously learned object type. The user may attempt to unlock previously learned object types when the user brings those object types to home 102. For example, the user may determine that an older shovel model performs better than an updated shovel model, and thus discard the updated shovel model. In this case, the user unlocks the robot's identification of the older shovel model in the provided list. The robot stores the updated list in a memory.

In some implementations, the selection and deselection of previously learned object types applies only to a single robot. The list of previously learned object types for the robot is local to each particular robot and each local list is not sent to other robots. The list remains local so that each user can personalize each individual robot to detect and identify the type of object for which the settings for that robot are most applicable. Personalization allows a user to be specific in instructing a particular robot to identify a particular item of interest.

Furthermore, the first robot may send the embeddings in its local cache over a communication network, such that these embeddings may be shared with other robots to allow the other robots to be able to identify similar objects. For example, the first robot may send the second and third robots an embedding stored in their local caches that allows the first robot to identify the coffee cup. After the second and third robots receive the embedment for identifying the coffee cup, the second and third robots may identify the coffee cup immediately upon observing the coffee cup in the environment of the robot. Similarly, the second robot may communicate an embedding to the first robot and the third robot that allows the robots to identify objects (e.g., caps). As a result, once the first and third robots receive and store in their respective local caches an embedding describing the cap, the first and third robots will be able to identify and retrieve the cap upon request by the user.

In some implementations, each robotic store in the cluster can be used to generate an embedded machine learning model. The machine learning model may receive an input of a characteristic of the object and embed the output representation into the compact coded representation describing the object. The embedding is generated based on the propagation of data through machine learning, so the embedding generated in response to the set of input data is a function of the structure and training state of the machine learning model. As a result, in order for each robot to use the embeddings generated by the other robots, the machine learning model in each robot may be identical, e.g., have the same structure and training state. This allows the robot to compare the robot-generated embeddings with the embeddings provided by other robots and with meaningful results. Otherwise, if two robots use different machine learning models, the two robots may produce very different embeddings for the exact same object, and the robots will not be able to interpret the received embeddings correctly.

The system may include a server separate from the robot that manages training and version control of the machine learning model for the robot. The server may periodically send updated machine learning models to the robot and ensure that the robot has the same model. In some implementations, the server can update the machine learning model when a new embedding is received from the robot. The server may send the updated machine learning model to each robot. The updated machine learning model may be provided at regular intervals, such as every two weeks, monthly, etc., or the updated model may be provided as desired. Having the same machine learning model in common among many robots may provide several advantages. For example, the computational expense of training the model may be done once for each model update at the server, which avoids the need for individual robots to expend power and computational resources for model training. Distributing the same updated model also shares the combinatorial learning of the robot cluster with the robot. All robots receive updated models that provide greater awareness, allowing the performance of each robot to improve in a standardized and predictable manner.

The system 100 illustrates a technique for sharing object recognition capabilities between robots. For example, the example illustrates robot 104B identifying an object, generating an embedding 118 representing the object, and sharing the embedding 118 and additional data with

other robots

104A, 104C, and 104D. This example also shows that the server system 112 trains the machine learning model to add the ability to identify objects represented by the shared embeddings, and the server system 112 distributes the trained machine learning model 130 to the robots 104A-104D.

The process shown in fig. 1 accomplishes two important tasks. First, unlike systems that require individual users to train each robot how to identify objects, the server system 112 aggregates and shares embeddings from each of the robots 104A-104D so that only one robot learns to identify new objects. In response, the server system 112 distributes the aggregated embeddings among robots such that each robot 104A-104D can identify new objects represented by the aggregated embeddings even though the human has never trained the other robots 104A-104D to identify the newly identified objects.

Second, the server system 112 may distribute the newly trained machine learning model 130 to each of the robots 104A-104D without requiring each robot to retrain its own local machine learning model. As a result, the system 100 allows learning while avoiding the need for each robot 104A-104D to perform the calculations required to train a machine learning model.

In some implementations, the server system 112 can store copies of the machine learning model locally on each of the robots 104A-104D. Robots 104A-104D and server system 112 each have the same machine learning model to ensure that the distributed learning of objects is uniform throughout the robot cluster. If the user trains the robot 104A to identify a new object, such as a pen, the robot 104A inputs the characteristics of the pen into local machine learning that produces an embedded as output. The embedded output represents a representation or fingerprint of the pen object. Further, the user may provide a class label to the robot 104A that describes the type of object represented by the embedding. The classification tag may be an audible or text input marking the embedded as a "pen". The class labels are stored with the embedding in a local cache of the robot 104A. Alternatively, the robot 104A may determine the embedded class label from previously identified pen objects stored in a local cache of the robot 104A.

In some implementations, the robot 104A sends the generated embedded and categorized tags to the server system 112 for storage in the database 114. Database 114 may include a table of class labels corresponding to the embeddings generated by the robot. Database 114 may also include sensor data, feature data extracted from sensor data, or other information used to generate the embeddings. The server system 112 may use the newly received feature data to train a machine learning model. The server system 112 trains its machine-learned copy to be able to recognize new objects in the subsequent process. After training with the embedding of the various robots 104A-104D, the server system 112 sends the trained machine learning model 130 to each of the robots 104A-104D. Each of the robots 104A-104D erases its currently locally stored machine learning model and replaces it with the updated, newly trained machine learning model 130 from the server system 112 and clears its respective local cache. Examples of such a system 100 are described further below.

During stage (a), the user 106 instructs the robot 104B to retrieve the cup 116. The user 106 speaks a command to the robot 104B, e.g., "robot, please retrieve my cup". The spoken utterance may include various types of phrases and/or instructions for each robot 104. If the robot 104B does not interpret the illustrated utterance, the robot 104B may request the user 106 to repeat the instruction. For example, robot 104B may process the spoken utterance using speech recognition to determine the context of the instruction.

The robot 104B first uses its perception model to attempt to identify the cup. If the robot 104B finds a cup in its immediate environment or in a map of the contents of its home, the robot 104B may locate the cup 116 and execute the requested command. However, if the robot 104B determines that it cannot find a cup 116, or that its classification model does not have the ability to recognize the object in question, then the robot 104B can say "which object is" my cup "? "in response to user 106. In response, the user 106 may take one or more actions to train the robot 104B to identify the cup 116.

In some implementations, the user 106 can train the robot 104B to identify the cup 116. For example, the user may be directed toward the cup 116 and the robot 104B may move to capture images from different sides of the cup 116. As another example, the user 106 may lift the cup 116 in front of the robot 104B so that the sensors of the robot capture and learn the characteristics of the cup 116. The robot 104B may view the cup 116 from various angles to learn various characteristics of the cup 116. For example, these characteristics may include shape, color, size, contour, and texture of the cup. Further, the robot 104B may utilize RGBD sensors that capture red, blue, green, and depth data. This information, possibly in combination with LIDAR, 3D depth data, or radar data captured by the robot, may allow the robot 104B to determine the three-dimensional shape and volume of the cup 116. The robot 104B may also capture audio data or other data while analyzing the cup 116. The robot 104B may include sensors such as, for example, cameras, LIDAR sensors, proximity sensors, infrared sensors, accelerometers, touch sensors, microphones, depth cameras, and position sensors. Other types of sensors may be used and the previously mentioned sensor types are described as examples.

During stage (B), the robot 104B processes sensor data about the object. For example, after the user has identified the cup 116 for the robot 104B and the robot has captured an image or other sensor data for the cup 116, the robot 104B processes the sensor data to identify characteristics of the object. The robot 104B generates feature data 122 from the obtained characteristics of the object. The feature data 122 may include representations of shape, color, size, contour, depth, and texture data from the sensors. The robot 104B provides the feature data 122 as input to its local machine learning object recognition model.

The local machine learning object recognition model may be a neural network that includes an input layer, an output layer, and one or more hidden layers. In some implementations, the primary purpose of the object recognition model is to recognize objects, not just to generate embeddings. For example, an object recognition model may be configured to provide a score representing a probability distribution over many types of objects that the model is configured to recognize. For example, if a model is trained to recognize 10000 different types of items, the model may have an output layer with 10000 elements, each element corresponding to a different type of item. The values generated at the output layer for each element may indicate the probability that the input feature data provided to the model represents the corresponding object type. Other model configurations may also be used, including dedicated encoding networks that are primarily or solely aimed at generating embeddings.

As the machine learning model processes the input feature data, the model generation may be used to derive embedded 118 information for the object described by the input feature data. For example, the embeddings 118 may be a set of values provided at the output layer of the neural network. As another example, the embedment 118 may be a set of values representing the activation at a hidden layer of the neural network, such as a set of activations of nodes at a final hidden layer. As another example, a function or transformation may be applied to information derived from the output layer and/or hidden layer of the model to generate the embeddings 118. In addition to neural networks, other types of machine learning model outputs may be used to generate the embeddings.

The insert 118 represents a coded representation of the characteristics of the cup 116. In general, the embeddings of different objects may each have a common format and size, e.g., each embedment may be a 128-unit vector, e.g., where each unit includes 64-bit data values. In another example, the embedding may be a 256-cell embedding representing an object. 256-unit embedding may indicate a higher fidelity embedding of the representation object than 128-unit embedding, but at the cost of using more memory. The embedding may be generated using local machine learning for each object observed by the robot 104B. However, the robot 104B does not recognize objects that the model has been able to recognize for the object to store the embedded in its cache. Instead, the cache is used to store the embeddings for the newly learned object, or the embeddings representing the newly learned object received from other robots when the object recognition model is unable to properly recognize the object. Thus, caching is typically used to store embeddings for objects that the object recognition model is not trained to detect, or to correct errors in the object recognition model that incorrectly classify the object. The embedded and classification information in the cache may complement the awareness capabilities of the trained machine learning model and even override incorrect classification made by the model.

In some implementations, the user 106 can provide the category labels 120 for the embeddings 118 generated by the bot 104B. For example, the robot 104B may request that the user 106 provide a classification tag 120 for objects that the robot cannot recognize. The user 106 may provide the classification tags 120 in any of a variety of ways, such as voice input to the robots, or information input to the client device 108, such as a telephone or computer in communication with each robot 104. The client device 108 may communicate with the robots 104A-104D via any communication protocol, such as Bluetooth or Wi-Fi. For example, the user 106 may type the category label 120 "cup" into the client device 108. Alternatively, the user may speak an utterance detected by the client device 108, as shown in fig. 1, say: "this is" cup ". In other implementations, the user 106 may communicate directly with the robot 104B to provide the classification tag 120 using text input or speaking to the robot 104B.

In some implementations, the user 106 may provide audible teachings to the robots 104A-104D. The audible teachings may be incorporated into previously known class labels that are understood by, for example, the robot 104A in order to understand the audible teachings. For example, the robot 104A may include an insert corresponding to a "beer mug" and another insert corresponding to a "beer stein" object. If the user 106 instructs the robot 104A to retrieve "beer-stent", the robot 104A will know to associate "beer mugs" with "beer-stent" and retrieve "beer-stent" objects.

In response, the robot 104B stores the generated embeddings 118, corresponding class labels 120, and feature data 122 in a local cache of the robot 104B. In some implementations, multiple embeddings can be stored for a single category label 121 to illustrate different examples of items of the same type. The robot 104B may capture sensor data showing characteristics of the object from various angles. Each angle of the object captured by the robot produces different sensor data that can be provided to local machine learning to produce different embeddings. As another example, multiple views of an object may be used to generate a single embedding representing three-dimensional aspects of the object. In some implementations, the robot may view multiple objects of the same type and use different instances of the object type to generate different embeddings. For example, the home 102 may have multiple pens of different shapes and sizes, but each of these is still a valid example of a "pen" classification. As a result, the robot 104B may store multiple embeddings for the "pen" class, with each embedment representing a different physical pen observed by the robot 104B.

Further, the robot 104B may store additional metadata corresponding to the category labels 120 and the embeddings 118. The robot 104B may store version codes for the current machine learning model stored on the robot 104B. The version code number describes the state of the current machine learning used to generate the embedment 118. For example, the version may be version number "1.0" or data corresponding to the time at which the machine learning model was installed. The robot 104B may also store location information corresponding to where the object corresponding to the embedding is located. The location information may include the type of location (e.g., home, store, library, etc., or living room, kitchen, restaurant, etc., in the home), the GPS coordinates of the location, and the date/time of day. For example, the user 106 may provide the type of location to the robot 104B using the client device 108 or by directly interacting with the robot 104B. For example, the type of location may be "kitchen"

In some implementations, the additional metadata may also include a description of how the robot 104B determines the classification of the identified object. For example, the classification may be determined using user input, such as user 106 inputting classification tags 120 into the robot. Alternatively, the classification may be determined by the robot 104B itself. For example, robot 104B may scan one or more objects as it moves through each room of home 102. As the object is scanned, the robot 104B provides characteristics of the object as input to a local machine learning model of the robot 104B. The output of the local machine learning model produces an embedding, such as embedding 118, that describes a representation of the object. Robot 104B may compare the embedded 118 output with the stored embedded in its local cache. If the robot 104B determines that the generated embedment matches similarly to at least one of the embedments in the local store, the robot 104B may store the generated embedment in the local cache along with the classification tag from the local cache. This process will be described further below.

In some implementations, the additional metadata can include an identification corresponding to the robot 104B that identified the object. For example, the identification of robot 104B may include an identification number, e.g., "0001," to identify robot 104B. The additional metadata may include a group ID and an owner ID number of the robot 104B. The group ID number specifies that the robot 104B belongs to a group that may include other robots. For example, group ID number 2 may specify that robot 104B is in a group with

robots

104A and 104C. The owner ID number designates the robot 104B to a particular owner.

In some implementations, the additional metadata may also include a robot type. The robot type may indicate a physical model of the robot, a role of the robot, or a description of the purpose of the robot. Even if the robots are physically the same or similar, their programming and functionality may be different. For example, the user 106 may designate the robot 104B as a "kitchen robot" and designate the robot 104A as a "golf robot". Other robot types are possible, such as "hotel robots", "hospital robots", "warehouse robots" and "garage robots". These robots can be trained differently to better perform their respective roles. Because they perform different types of tasks, the information and capabilities learned by one type of robot may not always be applicable or helpful to another type of robot. By marking the embeddings with the corresponding robot types, the server system 112 can selectively share the embeddings such that the robots receive embeddings that are applicable to their robot types (e.g., those generated by other robots in the same robot type) and do not receive the inapplicable embeddings.

During stage (C), robot 104B sends the embedded 118 and the corresponding class labels 120 to server system 112 over network 110. The robot 104B may also send corresponding characteristic data 122 or raw sensor data for generating the embeddings 118, as well as any metadata about the embeddings 118 or the robot 104B. The robot 104B may tag the embedment 118 with an indication of the destination of the server system 112. For example, the flag may include an indication that the embedding is "new", e.g., a classification that indicates new learning that the model is unable to recognize, and that the information should be shared with other robots. The server system 112 determines that new embeddings and classifications may be distributed to the

other robots

104A, 104C, and 104D. As another example, the tag of robot 104B may include an indication that the embedding is "known," e.g., the embedding is another example of a classification that is already known, which may be used to refine the model. The server system 112 may use the feature data 122 provided with the embedding to refine and train the machine learning model stored by the server system 112.

In some implementations, the server system 112 can store the received embeddings 118, the received respective class labels 120, and the feature data 122 in the database 114. Database 114 may include one or more tables including classification tags, characteristic data or sensor data describing objects, and corresponding embeddings generated by robots in system 100. Database 114 may additionally store each robot-generated metadata corresponding to each embedding. The server system 112 may retrieve the category labels, feature data, and corresponding additional metadata from the database 114 when training the local machine learning model.

During stage (D), the server system 112 distributes the received embedded, corresponding class labels and feature data to other robots. After the server system 112 receives the embeddings 118, the corresponding class labels 120, and the feature data 122 from the robot 104B, the server system 112 may determine that the embeddings 118 and the corresponding class labels 120 represent new learners that may be distributed to other robots. For example, the server system 112 may determine that the embedded 118 and corresponding class labels 120 should be distributed to other robots based on the indication labels provided by the robot 104B. For example, if the indicator reads as "new," the next action of the server system 112 is to distribute the embedded 118 and corresponding class labels 120 to the

robots

104A, 104C, and 104D. Further, the server system 112 may send additional metadata to the

robots

104A, 104C, and 104D along with the embeddings 118 and corresponding class labels 120.

In some implementations, the server system 112 can determine that the embedment 118 should be sent to other robots based solely on the received class label 120. For example, server system 112 may compare received class labels 120 with class labels stored in database 114. If the server system 112 determines that the received class label 120 does not match any of the class labels stored in the database 114, or that no match occurs depending on the context, the server system 112 stores the newly received embedded 118 and the received class label 120 in the database 114.

In some implementations, for example, the contextual matching occurs, for example, when the server system 112 receives the classification tag 120 of "cup" and the server system 112 finds the classification tag of "coffee cup" in the database 114. Although the class labels have different string characters, the context of "cup" and "coffee cup" have similar characteristics. If a context match does occur, the server system 112 may send a notification to the device 108 to inform the user 106 that the two embeddings correspond to different or similar objects. Alternatively, server system 112 may store and distribute the new embedding each time a new embedding is received from the robot, whether or not the new embedding is different from the embedding stored in database 114.

During stage (E), after collecting the embeddings from the robots 104A-104D, the server system 112 updates the machine learning model and sends the updated machine learning model 130 to each of the robots 104A-104D. In some implementations, the server system 112 can train a copy of its machine learning model using any newly received feature data from the robots 104A-104D. At the same time, while training is ongoing at the server system 112, the robots 104A-104D are still able to use learning incorporated by model training. The server system 112 has shared the embeddings and classifications that describe the examples being used in the model training process. Thus, robots 104A-104D can utilize embedding to recognize or classify new objects even before receiving updated models.

In one example, the robot 104D may learn to identify the object 124 in the home 102. After capturing sensor data describing the object 124 and learning the classification 132, the robot 104D may convert the characteristics into feature data 128 and provide the feature data 128 as input to its local machine learning model. From the output of the local machine learning model, the robot 104D generates an embedment 126. The bot 104D also determines a class label 132 for "camera" corresponding to the embedment 126, e.g., based on input from the user 106. Robot 104D uploads the embedment 126, the feature data 128 for generating the embedment 126, and the classification tag 132 to the server system 112. The server system 112 distributes the embedded 126 and classification tags 132 to the other robots 104A-104C, and the robots 104A-104C store this information in their local caches. The server system 112 also uses the embeddings 126, classification labels 132 and feature data 128 to further train the object recognition model.

The server system 112 may utilize the feature data and classification labels from each of the robots 104A-104D to train a machine learning model. For example, the server system 112 may train the neural network based on the feature data 122 and the class labels 120 and the feature data 128 and the class labels 132 from the robot 104B. By training the machine learning model with examples and learning from each of the robots 104A-104D, the server system 112 may improve the machine learning model to correctly recognize objects that each of the robots 104A-104D has learned.

Fig. 2 is a block diagram illustrating an example of a system for training a robot to detect objects. In some implementations, the user 106 may train a robot, such as the robot 104A, to identify and detect objects, such as cups in the example of fig. 2. Training robots 104A-104D to identify and detect objects allows user 106 to request robots 104A-104D to retrieve the objects at a later point in time.

In some implementations, each robot 104A-104D may include the components shown in FIG. 2. For example, as mentioned previously, robot 104A includes a robot sensor 202. The robot sensor 202 provides a method for the robot to interpret its environment. For example, the robotic sensors 202 may include cameras, LIDAR sensors, proximity sensors, infrared sensors, accelerometers, touch sensors, microphones, and position sensors, for example. The robot 104A may utilize each of these sensors to avoid obstacles while moving throughout the home 102, to understand the encountered objects that the robot 104A is gripping, and to receive requests from the user 106.

In some implementations, the robotic sensor 202 generates sensor data 204. The sensor data 204 includes image data 206 and depth information 208, and may include other sensor data. Further, the sensor data 204 may include contour, shape, and texture data. The image data 206 may include one or more videos and/or one or more still images of the object. Image data 206 may include color information (e.g., red, green, and blue or RGB information) and/or monochrome image data.

In some implementations, the depth information 208 is determined using a camera module. This information may be generated with a stereo camera, wherein the difference between the images between the two image sensors is used to determine the distance of the object from the camera. Additionally or alternatively, the depth information 208 may be generated using a LIDAR system, a radar system, or other depth sensor of the robot 104A.

In some implementations, the robot 104A may generate the feature data 212 from the sensor data 204. The feature data 212 may be a representation of the sensor data 204 that the machine learning model 214 is configured to receive as input.

The robot 104A includes a local machine learning model 214. The local machine learning model 214 may be stored in a memory of the robot 104A, optionally separate from the local cache 218. The server system 112 may provide the robot 104A with a machine learning model 214 over a network and may ensure that each robot 104A-104D uses the same machine learning model 214, e.g., the most current version.

The robot 104A determines the embedment 118 representing the encoded representation of the cup described by the image data 206, the depth information 208, and any other sensor data generated by the robot 104A. The embedment 118 is stored in a local cache 218 of the bot 104A along with the class label 120 provided by the user 106. The user 106 may provide the category label 120 through interaction with the client device 108. Alternatively, the user 106 may provide the category labels 120 by direct interaction with the robot 104A, by speaking into a microphone on the robot 104A, or by entering text via a terminal on the robot 104A.

In some implementations, the local cache 218 stores a set 220 of data for each object. For example, the set 220A of data about recently trained "cup" objects includes the embeddings 118, the classification tags 120 corresponding to the embeddings 118, and the sensor data 204 corresponding to the embeddings. By storing the set of data 220 in the local cache 218, the robot 104A can quickly access the set of data for use in recognizing objects it encounters. The robot 104A may store any embedded and classified tags received from the server system 112 in its local cache.

In some implementations, when the robot receives the embedded and classified tags from the server system 112, only the embedded and classified tags are received without the corresponding sensor data or feature data. Embedding is a data element for comparison in an object recognition process discussed further with reference to fig. 3. The embedded sensor data or feature data need not be generated as long as the embedded machine learning model used to generate the reception is the same machine learning model used by the receiving embedded robot 104A.

Fig. 3 is a block diagram illustrating an example of a system 300 for robotically detecting objects. In some implementations, each robot 104A-104D performs the following steps to detect and identify objects.

In some implementations, the robot 104D may find an object, such as a camera, in the home 102. The robot 104D captures sensor data 302 describing the camera with the sensor 202. The robot 104D may scan the object 124 at various angles and distances to obtain a full view of the object 124. The sensor data 302 may include image data 304 and depth information 306, as well as other information describing the object 124.

The robot 104D provides the sensor data 302, or the feature data generated from the sensor data 302, to the machine learning model 214 stored locally at the robot 104D. The model 214 used by the robot 104D may be a copy of the same machine learning model 214 stored and used by the other robots 104A-104C. The machine learning model 214 is configured to generate information indicative of the likelihood that the input data represents different types of objects. For example, the model 214 may output a set of values that form a probability distribution over a set of object types, e.g., where one value indicates a likelihood that the input data describes a "spoon," another value indicates a likelihood that the input data describes a "book," and so on. From the output of the model 214, the robot 104D may select the model output to indicate the object type most likely describing the observed object 124. For example, robot 104D may identify the model output that indicates the highest probability and select the classification corresponding to that output. In this example, the model 214 indicates, based on its training, that the observed object 124 is most likely to be a "box" represented by the class label 312, and the confidence score 310 for that class is 50%. The confidence score 310 indicates a confidence level that the classification label 312 accurately describes the object 124 and may be determined based on the output of the machine learning model 214 or based on another evaluation of the model's output.

In addition to using the machine learning model 214 to predict a classification for the object 124, the robot 104D also uses the processing of the machine learning model 214 to generate an embedment 314 for the object 124. For example, the embedding may be derived from activation at one or more hidden layers of the model 214 and/or from data at an output layer of the model 214.

The robot 104D compares the generated embeddings 314 for the object 124 with the embeddings stored in the local cache 218 of the robot. The robot 104D may compare the embeddings 314 separately from each of the embeddings in the local cache 218. For example, the robot 104D compares the embeddings 314 with the embeddings in the data sets 220A-220N and generates data, e.g., a confidence score, indicating how closely the buffered embeddings match the embeddings 314. To perform this comparison, robot 104D may perform a bit-wise comparison, vector distance determination, or other comparison to generate a measure of similarity between the embeddings.

In some implementations, each comparison may produce a confidence score. For example, a comparison between the embeddings 314 and the embeddings in the dataset 220A yields a 10% confidence score 320. The comparison between the embeddings 314 and the embeddings in the dataset 220B yields a 30% confidence score 322. The comparison between the embeddings 314 and the embeddings in the dataset 220C yields a 90% confidence score 324. The comparison between the embeddings 314 and the embeddings in the dataset 220D yields a 50% confidence score 326. The comparison between the embeddings 314 and the embeddings in the dataset 220N yields a confidence score 328 of 50%.

Robot 104D then identifies which of the embeddings has the highest confidence score or measure of similarity to embedment 314. In the example shown in fig. 3, this is an embedding in the dataset 220C with a corresponding class label 330 of "camera".

The robot 104D then selects whether to use the classification predicted by the machine learning model 214 or the classification determined using the cache 218. For example, the robot 104D compares the 90% confidence score 324 with the 50% confidence score 310 generated by the machine learning model 214. Robot 104D selects the classification corresponding to the higher of the confidence scores. In this example, robot 104D selects the category labels in dataset 220C to correspond to embedment 314 because 90% confidence score 324 is higher than 50% confidence score 310. This allows the local cache 318 to provide new or higher quality classification results than the machine learning model 214 provides.

In some implementations, the robot 104D may apply a threshold to ensure that the classification has at least a minimum confidence score before using the classification. For example, before the robot 104D accepts the classification results, the classification may be required to have a confidence score of at least 70%.

After selecting the classification of "camera" as the most likely classification of the object 124 based on finding a close match using the local cache 218, the robot 104D may provide the classification to other models and modules of the robot 104D to perform tasks. For example, the robot 104D may retrieve the camera when the user requests the object.

The robot 104D may repeat the process discussed in fig. 3 for each object observed by the robot 104D to classify the object encountered by the robot 104D. This recognition or classification process may be performed very frequently, e.g., almost continuously as the robot observes its perimeter, and for multiple objects that may be in view at any given time. As a result, it is important that the process be computationally efficient and power efficient. The architecture discussed herein provides these benefits because the processing of the machine learning model can be very fast and efficient, and the lookup and comparison process using the data sets in the local cache can be similarly fast and efficient.

In general, the embedded 314 and sensor data 302 about the observed object 124 is discarded after the object 124 is recognized. However, in some implementations, this information may be saved and provided to the server system 112 as additional example data of the object type, and the server system 112 may use this information in further training the machine learning model 214.

Fig. 4 is a block diagram illustrating an example of another system 400 for sharing learned information between robots. Fig. 4 illustrates an example of sharing awareness information among

robots

104A, 104B, and 104C. Each

robot

104A, 104B, and 104C stores and uses a copy of the same machine learning model 402. As a result, each robot 104A-104C produces a similar embedding when viewing similar objects. Each of the robots 104A-104C has a local cache 404A-404C in which the embeddings and corresponding classifications are stored.

The local cache 406A of the robot 104A initially includes a dataset 406A that is generated by the robot 104A based on sensor data captured by the robot 104A. The data set 406A may include at least embedded and corresponding class labels. The robot 104A may also send sensor data and metadata corresponding to the embedding in the data set 406A. Robot 104A sends data set 406A to server system 112 via network 110. The robot 104A may specify that the dataset 406A should be shared with other robots. Similarly, the robot 104A may indicate that the dataset 406A is new or that it represents a classification example that addresses an deficiency or error of the machine learning model 402.

The local cache 404B of the robot 104B initially includes a dataset 406B that is based on sensor data captured by the robot 104B. The data set 406B includes embeddings and classifications, but with respect to objects other than the one represented by the data set 406A. Robot 104B sends data set 406B to server system 112 over network 110.

Server system 112 distributes

data sets

406A and 406B so that each robot receives this information for its local cache. For example, the server system 112 sends the data set 406B generated by the robot 104B to the robot 104A. The server system 112 sends the data set 406A generated by the robot 104A to the robot 104B. The server system 112 sends the

data sets

406A and 406B to the robot 104C.

Due to the distribution by the server system 112, the robot 104C will be able to identify the objects represented by each of the embedded

datasets

406A and 406B without having to be trained by the user 106 to detect these objects. Further, the robot 104B will be able to identify the object represented by the embedment in the dataset 406A without being trained by the user 106 to detect the object. Similarly, the robot 104A will be able to identify the object represented by the embedment in the dataset 406B without having to be trained by the user 106 to detect the object. Sharing information in the data set (e.g., embedding) between robots over network 110 reduces the time required to train robots to recognize new objects.

The server system 112 may also store the

data sets

406A and 406B in the database 114. The server system 112 collects data sets from the robots over a period of time and further trains copies of the machine learning model 402 to be able to recognize the objects indicated in the

data sets

406A and 406B.

In some implementations, the server system 112 may perform periodic synchronization between the robots 104A-104C to ensure that each robot includes an embedded up-to-date set. For example, periodic synchronization may occur at a predefined rate, such as hourly, daily, weekly, etc., and synchronization may be initiated by the robot or by the server system 112. Periodic synchronization may include a request from the server system 112 to each of the robots 104A-104D to upload any new embeddings since the last synchronization. Each robot 104A-104D may also provide a list of identifiers for the data sets in its cache so that the server system 112 can specifically determine which data sets should be sent and which already exist. The server system 112 determines which embeddings are provided to the robots 104A-104D such that each robot 104A-104D includes the most current embeddings from each of the other robots 104A-104D in its local cache.

Fig. 5 is a block diagram illustrating an example of a system 500 for providing an updated machine learning model to a robot. Periodically, e.g., monthly, the server system 112 may generate and distribute an updated machine learning model that incorporates combinatorial learning from the various data sets provided by the robots 104A-104D. FIG. 5 illustrates an exemplary distribution of updated machine learning models to

robots

104A, 104B, and 104C. As previously shown in fig. 4, each

robot

104A, 104B, and 104C includes the same machine learning model 402. In addition, each

robot

104A, 104B, and 104C has its own separate

local cache

404A, 404B, and 404C.

The server system 112 includes a model trainer 502 that iteratively trains a machine learning model 504. This model 504 may be the same model 402 used by the robots 104A-104D prior to training, or it may represent a different model. The result of the training process is an updated machine learning model 506 that incorporates what the various robots 104A-104D have learned over a period of time. Model trainer 502 may update machine learning model 504 based on each received data set of the classification designated by the robot as newly learned. In some implementations, model trainer 502 may train machine learning model 504 with data representing additional examples of classifications that model 402 has been able to accurately recognize, such as with reinforcement learning. By training the machine learning model 504 based on the "new" and "known" data sets, the server system 112 may create an updated machine learning model 506 to accurately recognize objects from real world observations for both new object types and object types that the model has predicted.

In some implementations, the server system 112 sends the updated machine learning model 506 to each

robot

104A, 104B, and 104C over the network 110. In response to the robots 104A-104D receiving the updated machine learning model 506, the robots 104A-104D will remove the machine learning model 402 and replace it with the updated machine learning model 506. From this point on, robots 104A-104D use updated machine learning model 506 and benefit from the improved accuracy they provide.

When the robots 104A-104D receive the updated machine learning model 506, the robots 104A-104D clear their local caches 404A-404D. For example,

data sets

406A and 406B are deleted. The classification represented by the data set in the cache is now directly predicted by the updated model 506, so that the stored data set is no longer needed. Additionally, since the data sets in the cache are generated using the old machine learning model 402, the embedding from these data sets will not be compatible with the updated machine learning model 506. If some of the data sets have not been shared by the robot, but have not been incorporated into the training of the updated machine learning model 506, the server system 112 may use the robot-provided sensor data to generate an embedding with the updated machine learning model 506. The server system 112 may then send the updated data set with the embedded for the updated model 506 to the robots 104A-104D for storage in the local cache and use with the updated model 506.

Fig. 6 is a flow chart illustrating an example of a process 600 for sharing learned information between robots. A first robot, such as robot 104A, may perform process 600.

In process 600, a first robot obtains sensor data indicative of a characteristic of an object (602). The robot may include sensors such as cameras, LIDAR sensors, proximity sensors, infrared sensors, accelerometers, touch sensors, microphones, and position sensors, for example. For example, a robot may capture and learn characteristics of an object from several different angles. The robot may move itself or its sensors to different positions to obtain these different views. The obtained characteristics may include information about the shape, color, contour, texture, and depth of the object. The robot may convert the obtained sensor data of the object into feature data for the machine learning model to process the feature data.

The robot determines a classification for the object (604). For example, a user may provide a classification for an object. The user may provide classification of the object using voice input or text input to the client device 108. Alternatively, the user may provide classification for the object by interacting with the robot itself. The classification may be a text label that is marked to the object.

The robot generates an embedding for the object using a machine learning model stored by the first robot (606). The robot extracts feature data 122 from the sensor data for the object. The feature data may include a digital version of the sensor data as understood by the machine learning model, such as a score or value representing aspects of the sensor data. For example, robot 104B provides feature data 122 as input to its local machine learning model. The machine learning model processes the feature data to produce an embedding.

In some implementations, the machine learning model is a neural network model, and the embedding may be derived from an output at an output layer of the neural network model. In other implementations, the embedding is derived from data indicative of activation at a hidden layer of the neural network model. The specific values embedded may depend on the structure and training state of the machine learning model.

The robot stores the generated first embedded and data indicative of the classification of the object in a local cache of the first robot (608). For example, the robot stores the generated first embedded and classification tags of the object in a local cache of the robot. Further, the robot may store the respective sensor data of the identified object in a local cache along with the generated first embedded and classified tag.

The robot transmits the generated first embedded and classified data to a server system (610) through a communication network. The robot may also send metadata to the server system. The metadata may specify, for example, whether the first embedding is to be shared with other robots. The metadata may also include information such as version code or an identifier for generating the first embedded machine learning model. As another example, the metadata may indicate a robot identifier of the robot, a type or model of the robot, or an identifier of a role of the robot, and/or data indicating a group of robots to which the robot belongs. The server system may use this information to select which robots should receive the embedding of the share, e.g., it may be that the share is limited to occur between robots in the same group or between robots of the same type, model, or role. In some implementations, the first embedding may be sent using an application programming interface (application programming interface, API) for sharing such that the first embedding and other data sent using the API are designated to be shared as being sent through the API.

The robot receives the second embedding and the corresponding classification from the server system over the communication network (612). The second embedding may be that generated by the second robot. For example, the second robot generated embeddings and corresponding classifications may be for objects that the first robot has not learned to recognize.

The robot stores the received embedments and corresponding classifications in a local cache of the robot (614). The robot may then use the embedding and classification in the cache for both objects observed by the robot and for received data representing objects not observed by the robot to recognize the objects when performing tasks.

In some implementations, the robot may then use the first embedding or the second embedding in the local cache to recognize one or more objects in the robot's environment. When the robot recognizes one or more objects in the environment using the generated first embedding or the generated second embedding, the robot generates sensor data describing a particular object observed by the robot in the environment of the robot. The robot generates an inlay for the particular object using the second machine learning model and sensor data describing the particular object. The robot then compares the newly generated embeddings for the specific object with the first embeddings and the second embeddings. The robot assigns a classification to a particular object based on a comparison of the embedding of the particular object with the generated first embedding and the generated second embedding. As described above, if the embedding of a particular object is not within a threshold level of similarity to any of the embeddings in the cache (e.g., the vector distance between the embeddings is less than a maximum threshold), then the classification indicated by the machine learning model is used. On the other hand, if the embeddings of a particular object are within a threshold level of similarity, or at least sufficiently similar such that the confidence score indicates a better match than the machine learning model indicates, the robot may use the classification corresponding to the most similar embeddings in the cache as the classification of the particular object.

In some implementations, the robot may receive the second machine learning model from the server system over the communication network. The received second machine learning model may be one generated by the server system. The second machine learning model may be an updated machine learning model that has been updated based on a combined set of robot learning that occurred throughout the fleet in a previous time interval (e.g., last week or last month). The robot may use the second machine learning model to recognize one or more additional objects and previously identified objects. Further, the robot may remove from its local cache the embeddings generated using the first machine learning model.

Fig. 7 is a block diagram illustrating an example of a system 700 for selectively sharing learned information between robots. As described above, it is advantageous that robots share their learned information so that all robots in a cluster can learn more quickly and improve their capabilities. However, some of the information learned by the robot may be of low quality, such as incorrect or conflicting information. Propagating low quality information throughout the cluster of robots can degrade performance or introduce errors in the perception of the robots. Similarly, training a machine learning model with low quality information may reduce model accuracy. To ensure that the sharing of robot learning improves performance, the computing system may evaluate the information learned by the robot and limit the sharing of information to items beneficial to the fleet. The computing system may also set quality thresholds and quality checks for training of the machine learning model such that updating of the model improves performance and is based on the efficient example. These evaluation and filtering steps may also provide measures of security by preventing malicious or intentionally incorrect information from propagating between robots and degrading robot performance.

In FIG. 7, server system 112 evaluates the data sets uploaded by robots 104A-104D. The server system 112 distributes only data sets that pass certain quality tests. Similar to FIG. 4, each robot 104A-104C includes a local cache 404A-404C, respectively. Each robot 104A-104C also stores and uses the same machine learning model 402. Unlike fig. 4, the server system 112 does not automatically share the uploaded data set with other robots, but rather verifies the quality (e.g., accuracy) of the data set before distributing it to other robots.

In this example, robot 104A sends newly acquired

data sets

406A and 406B to server system 112 over network 110. The robot 104A may designate each

dataset

406A and 406B as information that may be shared with a server and with other robots. The server system 112 will evaluate these

data sets

406A and 406B to determine if they should be shared, e.g., sent to the

other robots

104B and 104C. In some implementations, sharing may be performed on a group of robots using an access control list, on robots within a domain, or on each robot in a cluster. To avoid tampering, the robots 104A-104D may encrypt the data set 406 or securely sign the data set 406 so that the server system 112 may securely receive the transmitted data set 406.

Before the server system 112 distributes the

data sets

406A and 406B to the

other robots

104B and 104C, the server system 112 may evaluate the quality of the

data sets

406A and 406B based at least in part on the embedding in the data sets. The server system 112 may also store the

data sets

406A and 406B in the database 114.

The server system 112 may include a quality assurance system 702 that provides a quality assessment of each data set received from the robots 104A-104D. The quality assurance system 702 includes a quality analysis module 704 that analyzes each of the embedments received from the robots 104A-104D. The quality analysis module 704 generates a quality score indicative of the quality of the embedding in the data set 406. How the quality analysis module 704 analyzes the embedding in the data set 406 and generates a quality score that indicates the quality of the embedding is described further below. For example, the quality analysis module 704 generates a quality score of "82" for the data set 406A. In addition, the quality analysis module 704 produces a quality score of "46" for the data set 406B. In response to determining quality scores corresponding to the embeddings in the dataset, the quality analysis module 704 compares each of the determined quality scores to a quality threshold.

The quality assurance system 702 may store and apply quality thresholds to the data sets being evaluated. The quality threshold may be predefined or may be dynamically set based on the data being evaluated. As shown in fig. 7, the quality analysis module 704 sets the quality threshold to "70", meaning that the individual data set may be required to have a quality score of at least this minimum value before it can be distributed to other robots.

The quality analysis module 704 compares the quality score for each data set 406 to a quality threshold. If the quality analysis module 704 determines that the quality score is greater than or equal to the quality threshold, the quality analysis module 704 allows the server system 112 to send the data set to the other robots 104A-104D. If the quality analysis module 704 determines that the quality score is less than the quality threshold, the data set is not transmitted. In some implementations, an indication of whether the data set 406 is shared by the quality analysis module 704 is stored in the database 114.

For example, the quality analysis module 704 compares the quality score of "82" of the dataset 406A to the quality threshold "70". The quality analysis module 704 determines that the quality score of "82" of the data set 406A is greater than the quality threshold of "70". As a result, the quality analysis module 704 indicates the shared data set 406A to the server system 112.

The quality analysis module 704 also compares the quality score of the dataset 406B to a quality threshold "70". The quality analysis module 704 determines that the quality score for "46" of the data set 406B is less than the quality threshold of "70". As a result, the quality analysis module 704 indicates to the server system 112 that the data set 406B is not shared.

Based on the evaluation of the data sets, the server system 112 selectively transmits the data sets to the

robots

104B and 104C. For example, the quality analysis module 704 determines to share the data set 406A and not share the data set 406B based on comparing their respective quality scores to a quality threshold. As shown in fig. 7, server system 112 sends data set 406A to

robots

104B and 104C. Each

robot

104B and 104C then stores the data set 406A in a respective

local cache

404B, 404C. In some implementations, the quality analysis module 704 can provide an indication to the robot 104A that transmitted it that the transmitted embedment has a particular quality. For example, the indication may indicate that the transmitted embedding has low quality or has high quality. By indicating a quality level or quality score, robots 104A-104D may weight data set 406A appropriately to give more or less confidence, for example, in the classification they indicate.

The server system 112 may instruct the robots 104A-104D to store the data set 406A in their local caches. For example, the designation may include a data flag, tag, instruction, version number of the local cache, and an indicator in the header of the message sent to robots 104A-104D. The version number may be provided for the individual dataset, for the entire cached state, and/or for indicating the state of the embedded machine learning model used to generate the dataset. By indicating the appropriate version of the cached content, if the robot cancels the update or loses connectivity halfway, the server system may check the version of the most recently completed cached update to determine what update the robot needs. In other examples, server system 112 indicates that data set 406A should be cached by sending data set 406A via some source, port, or particular API. The version number of the local cache allows the robots 104A-104D to track what embeddings should be included in the local cache. The server system 112 may verify the embedding stored in the local cache of each robot 104A-104D based on the version number of the local cache.

The server system 112 may synchronize the data sets in the local caches between each of the robots 104A-104D. The server system 112 may synchronize local caches between each of the robots 104A-104D periodically daily, weekly, or monthly. The server system 112 synchronizes the local caches by retrieving data sets from each of the robots 104A-104D. The server system 112 distributes the data sets retrieved from each of the robots 104A-104D such that each of the robots 104A-104D includes the data sets from each of the other robots 104A-104D. As a result, each of robots 104A-104D includes the same data set after each synchronization.

Fig. 8 is a block diagram illustrating an example in which the quality analysis module 704 evaluates the quality of information from a robot. The quality analysis module 704 evaluates the quality of the embedded information received from the robots 104A-104D. The quality analysis module 704 evaluates the data sets by evaluating and filtering the data sets using various criteria. For example, various criteria may include accuracy of classification, consistency with reference data, and redundancy with respect to other data sets, for example.

One technique that the quality analysis module 704 may use to evaluate a data set is to map an embedding into a high-dimensional space, such as a vector space, and evaluate the relative position of the embedding in that space. In general, embedding encodes the characteristics of objects, so the embedding of the same type of object should be similar. This will generally result in the embedding of objects of the same type substantially clustered together in a high dimensional space. The spacing between clusters may indicate a distinction between the types of objects represented by the embeddings in the clusters. These attributes may be used to evaluate the quality of the data set uploaded by robots 104A-104D to server system 112. For example, the embedding of an object may be evaluated to determine if the embedding is properly close to other embeddings of the same object type, and at least a reasonable distance from the embeddings of different object types.

The quality analysis module 704 maps the received embeddings in the dataset 406 to the high-dimensional space 802. The high-dimensional space 802 may be an N-dimensional space, where N represents the number of dimensions (e.g., the number of values) in each embedding. For example, each embedding in the dataset may be a 128-element array of 64-bit values, and thus may be compared to other embeddings in 128-dimensional space. Other N-dimensional spaces are possible depending on the number of dimensions provided in the embedding.

The illustration provided in fig. 8 shows a legend 801 to illustrate the elements drawn. The illustration of the high-dimensional space 802 shows the embedding of the mappings, where each embedding is represented by an "X". Reference points "·" are also shown to represent the location of the embedment verified to represent a certain object type. To obtain reference data to evaluate the quality of the newly learned classification, training data or other high quality examples may be provided to the machine learning model and used to generate the embeddings. For example, images that are known to show cups and that are accurately classified as being cups by a machine learning model may be used to generate embeddings for "cup" object classification. This embedding, which is known to be a correct representation of the cup, can be used as a reference point in the high-dimensional space 802, to which other embeddings can be compared.

When the server system 112 receives a data set from a robot, the quality analysis module 704 performs various steps to evaluate the quality of the data set. For example, the quality analysis module 704 may compare the received embedment 825 with other embedments, whether from data sets uploaded by other robots or reference embedments with known correct classifications.

The quality analysis module 704 may filter the received insert 825 for redundancy in step 808, filter the received insert 825 for maliciousness in step 810, filter the received insert 825 for sensor data quality in step 812, and filter the received insert 825 for machine learning model versions in step 814. Finally, the quality analysis module 704 applies a threshold comparison to the received embedded 825 in step 816.

For example, the received embeddings 825 and embeddings generated using machine learning models. The high-dimensional space 802 includes regions corresponding to various objects. For example, the high-dimensional space 802 includes the following regions: an embedded area 818 including an object type "watch", an embedded area 822 including an object type "mug", an embedded area 826 including an object type "shoe", an embedded area 830 including an object type "pen", and an embedded area 836 including an object type "paper".

The quality analysis module 704 maps the received embeddings 825 to locations in the high-dimensional space 802. The closer the received insert 825 is mapped by the quality analysis module 704 to one of the regions in the high-dimensional space 802, the higher the likelihood that the received insert 825 represents an object described by that region in the high-dimensional space 802. For example, the mass analysis module 704 maps the received insert 825 in a location closest to the embedded region 822 that includes the object type "mug". Because of the proximity of the received insert 825 to the center of the region 822, the received insert 825 is more likely to represent a "mug".

In some implementations, the received embeddings 825 are compared to reference embeddings derived from the machine learning model 402 stored on the server system 112. For example, a reference point "·" in region 822 representing a "mug" object illustrates a verified exact embedding of the object "mug". This reference point represents an embedding derived from the machine learning model 402 stored on the server system 112. The quality analysis module may determine one or more reference embeddings for each object type, an ideal embedment for each object defined in the high-dimensional space 802 generated by the machine learning model 402 on the server system 112. Specifically, the standard check 804 determines the ideal embedding by analyzing various embeddings stored on the database 114 that represent the same object (e.g., "mug"). For example, the criteria check 804 may use an average of the various embedding outputs to determine the ideal embedding. In another example, the criteria check 804 may select one of a variety of embeddings to represent a desired embedment. In another example, the criteria check 804 may use heuristics, bayesian, or any other type of statistical model to determine an ideal embedding. The quality analysis module 704 stores the ideal embedding in the database 114 for further comparison.

In some implementations, the quality analysis module 704 determines which ideal embedding the received embedding 825 most closely corresponds to in the high-dimensional space 802. For example, the quality analysis module 704 may employ a nearest neighbor algorithm to determine with which ideal embedding the received embedding 825 is in the closest vicinity in the high-dimensional space 802. The mass analysis module 704 compares the received insert 825 with the ideal insert of the "watch" 818, with the ideal insert of the "mug" 822, with the ideal insert of the "shoe" 826, and with the ideal insert of the "pen" 830. As a result, the mass analysis module 704 determines that the received insert 825 is in the closest vicinity of the ideal insert of the "mug" 822.

In some implementations, the quality analysis module 704 compares the classification tag corresponding to the received embedding with the tag corresponding to the ideal embedding that is closest to the received embedding. For example, an ideal insert of a class label of "mug" may be most closely adjacent to the received insert 825 of a class label that includes "mug". Alternatively, the quality analysis module 704 may compare the received embedding with one or more other embeddings corresponding to two different classification tags. In response, the quality analysis module 704 may omit one or both of the embeddings in view of two embeddings in close proximity to each other, however including different classification tags. The quality analysis module 704 may omit one or both of these embeddings by not sharing with the other robots 104A-104D, not storing in the database 114, or not being used to train the machine learning model 506.

The mass analysis module 704 then measures the distance 824 between the received embedded 825 and the ideal embedded point 821 and compares it to a threshold. For example, the quality analysis module 704 uses vector measurements to determine a vector distance 824 between the received embedded 825 and the ideal embedded point 821. If the quality analysis module 704 determines that the vector distance 824 is greater than the threshold, the quality analysis module 704 discards the received embedment 825 and waits for the next embedment received from the robot 104 or database 114. Otherwise, the mass analysis module 704 proceeds to the next process.

In some implementations, the quality analysis module 704 filters 808 the received embeddings 825 for redundancy. More specifically, it is important that the database 114 store different embeddings to ensure that robots 104A-104D have a more diverse set of data to improve recognition of similar and different objects.

In some implementations, the quality analysis module 704 filters the received embedded 825 in step 810 to avoid unsafe or malicious input. For example, the quality analysis module 704 may determine whether the embedding has been tampered with or assigned an incorrect classification. The user 16 may have entered a wrong classification label for the object, for example, may teach a robot to provide a wrong classification for the object. In some instances, the data set may be counterfeited, wherein the correct embedding may be replaced with a different one. The quality analysis module 704 may detect these errors or misclassifications in the data set by comparing the received embeddings with the same class of reference embeddings. For example, if the received embedment is classified as a cup, but is a vector distance from the embedment of a known correct cup, the dataset may be determined to be in error. Similarly, the server system 112 may use the sensor data uploaded with the data set to generate a corresponding embedding. If the embedding does not match the uploaded embedding, the server system 112 may determine that the embedding may be counterfeit.

In some implementations, the quality analysis module 704 filters the received embedded 825 for sensor data quality (step 812). The quality analysis module 704 checks the sensor data quality by analyzing whether the image is too dark, too bright, out of focus or whether the image is not included in the feature data of the dataset. Further, the quality analysis module 704 checks for obstructions to the detected object in the image and any inconsistencies between the shape of the depth information and the image data. The quality analysis module 704 also checks for missing sensor information, a time stamp, GPS location information in the additional metadata, and the presence of the additional metadata.

In some implementations, the quality analysis module 704 may evaluate whether the image is too dark or too bright, for example, by determining an average brightness. In some implementations, the quality analysis module 704 may perform an evaluation of whether the image data is out of focus or blurred. For example, an application may perform edge detection or perform a discrete fourier transform (discrete Fourier transform, DFT) to evaluate image quality. If these processes exhibit poor edge contrast or a low number of high frequencies, the image data and the entire dataset may be considered unsatisfactory and not used to train the model.

In some implementations, the quality analysis module 704 can utilize an external server for external verification of the image data. For example, the external server may perform an image recognition process or a shape recognition process to classify objects in the image data. The quality analysis module 704 may send the image data and the corresponding classification tags from the data set 406 to an external server to see if the results of the image recognition process or shape recognition process are consistent with the corresponding classification tags. If the image recognition process or the result of the shape recognition process matches the corresponding class label, the quality analysis module 704 may pass the data set 406 to the process in 814. If the image recognition process or the result of the shape recognition process does not match the corresponding class label, the server system 112 may send a notification to the client device 108 of the user 106 indicating that an incorrect class label exists for the embedding. In this way, server system 112 may request edits to the category labels from user 106. Alternatively, the server system 112 may provide updates to the image recognition process or shape recognition process if their results are incorrect and the classification labels provided in the dataset 406 are correct.

In some implementations, the quality analysis module 704 filters 814 the received insert 825 for the machine learning model version number. For example, the quality analysis module 704 analyzes additional metadata in the dataset 406. As previously mentioned, the additional metadata in the dataset 406 includes a version code number of the current machine learning model stored on the robot 104 that sent the dataset 406. The additional metadata may also include location information corresponding to whether the robot 104 identified the object represented by the embedding. Further, the additional metadata may also include a description of how the robot 104 determined the classification of the identified object, such as by user input or by comparison with other classification tags and corresponding embeddings stored in a local cache of the robot 104.

In some implementations, the quality analysis module 704 compares the retrieved version code number of the current machine learning model stored on the robot 104 with the version number of the machine learning model stored on the machine learning model 402 on the server system 112. If the quality analysis module 704 determines a version number match between two machine learning models (e.g., a machine learning model on the robot 104 and a machine learning model on the server system 112) (e.g., indicating that the same machine learning model exists on both the robot 104 and the server system 112), the quality analysis module 704 proceeds to the next process. Otherwise, the quality analysis module 704 indicates to the server system 112 that the robot 104, e.g., the robot 104A that sent the dataset 406A, does not include the latest machine learning model 402. Thus, the criteria check 804 discards the data set 406A. In addition, the server system 112 sends the machine learning model 402 to the robot 104A. In response, the robot 104A will remove its machine learning model 402 from memory and erase the data set in its local cache 404A. In some implementations, the server system 112 sends instructions to each robot 104 to erase the data sets in its respective local cache. The instructions may be included with the machine learning model 402 that is sent. In other implementations, the server system 112 sends the instructions as an individual message. In response to the robot 104 receiving the instruction, the robot 104 erases the content of its local cache.

In some implementations, the quality analysis module 704 filters 816 the received insert 825 by comparing the location of the received insert 825 in the high-dimensional space 802 to a threshold value. For example, each region in the high-dimensional space 802 defines an object, such as region 822 defining a "mug", region 818 defining a "watch", region 836 defining a "paper", region 830 defining a "pen", and region 826 defining a "shoe". Each region includes a threshold region for filtering purposes. For example, a threshold region 827 for a "mug" object, a threshold region 829 for a "shoe" object, a threshold region 831 for a "pen" object, a threshold region 833 for a "paper" object, and a threshold region 835 for a "watch" object. Each threshold region defines an N-dimensional region around the ideal embedding of an object type, which allows acceptable embedding of the same object type.

In some implementations, the quality analysis module 704 measures a vector distance between the location of the received insert 825 and an ideal insert location in the high-dimensional space 802 that is closest to the location of the received insert 825. This process allows the quality analysis module 704 to determine which ideal insert is the most closely received insert 825 in the high-dimensional space 802. Once the ideal embedding that is closest to the location of the received embedding 825 is determined, the quality analysis module 704 determines whether the location of the received embedding 825 falls within a threshold region corresponding to the ideal embedding. For example, the mass analysis module 704 determines that the received insert 825 is most immediately adjacent to the region 822 defining the "mug". As a result, the mass analysis module 704 determines whether the received embedded location falls within the threshold region 827 of "mug". As shown in fig. 8, the received insert 825 falls within a threshold region 827 and passes through step 816. Alternatively, if the embedding falls outside of the threshold region corresponding to the most immediately adjacent ideal embedding, the quality analysis module 704 discards the dataset 406 that includes the embedding.

In other implementations, the mass analysis module 704 may designate an insert that falls within a threshold region (e.g., threshold region 827) as an insert to be sent to the robots 104A-104D. This is because these embeddings indicate a higher likelihood of representing objects within the threshold region 827. As a result, the quality analysis module 704 selects the embeddings found outside the threshold region 827 to store in the database 114. In some implementations, the quality analysis module 704 designates an embedding found outside of a threshold region (e.g., threshold region 827) as not being used to train the machine learning model 506. In some implementations, the quality analysis module 704 will discard any embeddings that exist beyond a threshold distance from the ideal embedment. For example, the location of the embedment 837 exists outside of a threshold distance from each region in the high dimensional space 802. The ideal inlay of the "shoe" area is the distance 828 between the inlay 837. In this way, the mass analysis module 704 will discard the data set corresponding to the embedding 837.

In some implementations, the quality analysis module 704 generates a quality score that indicates the quality of the embedding of the received embedding 825 through each of steps 806 through 816 in the quality analysis module 704. The quality score indicates that the quality analysis module 704 is more confident that the received embedment represents an object (e.g., "mug" or "pen").

In some implementations, the quality analysis module 704 generates a quality score based on a process performed by the quality analysis module 704 to evaluate the embedded quality. For example, as described in 806, the quality analysis module 704 generates a proximity score using the proximity of the received location of the insert 825 to the region of the high-dimensional space 802 defining the object type. The closer the location of the received insert 825 is to the ideal insert location for the region, the higher the proximity score. Similarly, the farther the location of the received insert 825 is from the ideal insert's location in the region, the lower the proximity score. The quality analysis module 704 provides the proximity score as a quality score to the quality assurance system 702 for further comparison with a quality threshold.

In other examples, the quality analysis module 704 generates a quality score based on other criteria. Other criteria include the horizontal quality of the sensor data or features provided by the robot 104 for the respective data set 406. Other criteria may also include embedded consistency of the adjacent received embedded 825 in the high-dimensional space 802. For example, how close the embedded values are to each other; the characteristics of the characteristic data corresponding to each of the adjacently received embeddings 825 closely correspond to each other and provide an identification of the embedded robot.

In some implementations, the quality analysis module 704 can generate a quality score for the received insert 825 based on how the received insert 825 changed the accuracy of the machine learning model 506. If the received embedment 825 increases the accuracy of the machine learning model 506, the quality analysis module 704 may generate a higher value quality score. Alternatively, if the received embedding 825 reduces the accuracy of the machine learning model 506, the quality analysis module 704 may generate a lower value quality score. In some implementations, the accuracy of the machine learning model 506 can be adjusted for the particular object type received by the insert 825.

In some implementations, the quality analysis module 704 can adjust the quality score of the received embedded 825 if the corresponding classification tag includes personalized information. For example, if the received class label of the insert 825 includes personalized aspects such as "I'm favorite spoon", "John's toy", or "Blaine's glasses". The quality analysis module 704 may search for key phrases in the category labels, such as "my" or any name that indicates a personalized aspect of the object represented by the received insert 825. If a personalized aspect is identified, the quality analysis module 704 may adjust the quality score by increasing the value of the quality score.

In some implementations, the robot 104 can have a history corresponding to the type of data set 406 provided. For example, robot 104C may have a history that provides higher quality embeddings, while robot 104A may have a history that provides lower quality embeddings. The lower quality of the embedment provided by the robot 104A over time may indicate that the robot 104A includes bad components, such as a worn out camera. Alternatively, the lower quality of the embedment provided by the robot 104A may indicate that an update to its locally stored machine learning model 402 is required. As a result, the quality analysis module 704 adjusts the quality score for the received insert 825 based on a determination of the history of the robot 104 providing the high or low quality data set 406.

Fig. 9 is a block diagram illustrating an example of a system 900 for testing a machine learning model 506. Machine learning model 506 represents an updated version of machine learning model 402 used in fig. 7. Once a set of high quality data sets are identified, the machine learning model 506 is trained to incorporate classification information represented in those data sets. The machine learning model 506 is tested after training or as part of training. Testing has the general purpose of verifying the accuracy of the machine learning model 506. For example, the test may evaluate whether the training effectively achieves the goal of teaching the machine learning model to classify a new type of object indicated in the data set uploaded by the robot. If the machine learning model 506 does not provide an accurate classification (e.g., provides a different classification than the one learned by the robot) in response to the input data about the data set, training may continue until the machine learning model 506 may accurately classify the object.

As another example, the test may determine whether the machine learning model 506 maintains an appropriate level of accuracy for classifying objects for which it was previously sortable. Training the model should improve accuracy rather than decrease accuracy, so the server system 112 may implement the following rules: training should not degrade performance with respect to some or all of the categories of objects. If the training reduces the ability of the machine learning model to accurately classify a particular type of object, the server system 112 may continue training and use that particular type of example. This testing and continued training may provide a comprehensive model that is not unduly affected by the new training example. In this way, the performance of the model may steadily improve over time, as the user does not want the object awareness performance to decrease over time or fluctuate depending on the object type between one update to the next.

The system tests the quality of the machine learning model 506 in a training module 902 and an evaluation/test module 904. The system tests the quality of the machine learning model 506 by providing data from the dataset 406 to the machine learning model 506 stored on the server system 112 and evaluating the output of the machine learning model 506 in response.

In some implementations, the system provides data from the data set 406 provided by the robots 104A-104D each time a new object is learned by the robot. A robot, such as robot 104C, may learn objects, such as "pictures," whose local machine learning model 402 has not learned to identify. The output of the machine learning model 402, or a layer of the machine learning model 402, may produce a new embedding that the robot 104C has not yet understood. This is because the robot 104C is learning a "picture" object for the first time. The robot 104C requests a classification tag for this new object from the user 106. The user 106 provides a class label "picture" for the object "picture" and the robot 104C corresponds the newly received class label in the dataset 406 with the newly generated embedded and feature data captured by the robot 104C. The robot 104C provides the data set 406 to the server system 112 for provision to other robots and updates the machine learning model 506 stored on the server system 112.

In some implementations, the system provides data from the data set 406 retrieved from the database 114 to test the output layer or hidden layer of the machine learning model 506 stored on the server system 112. After the server system 112 trains the machine learning model 506 to detect the new data set 406 provided by the robots 104A-104D, the server system 112 validates the data set 406 derived from the machine learning model 506. The server system 112 performs this verification using the data set 406 from the database 114 to ensure that the correct embedding and classification output can be derived from the machine learning model 506. In some implementations, once the server system 112 determines that the correct output can be properly derived from the machine learning model 506, the server system 112 provides the machine learning model 506 to the robots 104A-104D.

In some implementations, the training module 902 attempts to train the machine learning model 506 with the newly received data set 406. For example, robot 104A provides dataset 406C for a "bicycle" object and robot 104B provides dataset 406D for a "glasses" object. The data set 406C includes feature data 908, an embedment 910, and a classification tag 912 representing a "bicycle" object. The data set 406D includes feature data 914, an insert 916, and a class label 918 representing "glasses" objects.

In some implementations, the server system 112 provides the feature data from each dataset 406 to the machine learning model 506. For example, the server system 112 provides feature data 914 of the "glasses" object to the machine learning model 506. As shown in training module 902, a layer of machine learning model 506 may generate an output that includes an embedding 920 and a corresponding classification tag 922. The server system 112 compares the output class label 922, representing "pencil", with the corresponding class label 918 provided in the dataset 406D. The result of the comparison does not match because the class label 922 representing "pencil" is not the same as the class label 918 representing "glasses". In response, the training module 902 updates the machine learning model 506.

In some implementations, the training module 902 updates the machine learning model 506 by providing the received feature data 914 of "glasses" and the classification tag 918 representing "glasses" as inputs to the machine learning model 506. In response, the machine learning model 506 generates an embedding 920 corresponding to the class label 918 representing "glasses". The server system 112 stores the feature data 914, the generated embeddings 920, and the class labels 918 representing "glasses" in the database 114. The training module 902 then provides the next subsequent data set, namely data set 406C, to the machine learning model 506 to evaluate its output at the particular layer.

In some implementations, the evaluation module 904 evaluates the output at a particular layer of the machine learning model 506. After the training module 902 trains the machine learning model 506, the evaluation module 904 evaluates the new machine learning model 506 using the data stored in the database 114. The data stored in database 114 includes reference data that is trusted and verified by previous machine learning model versions. The reference data includes classification tags corresponding to the embedded and characteristic data. For example, the reference data may include a classification tag of the "cup" object, embedding of the "cup" object, and feature data captured by the robot 104 describing the "cup" object. The evaluation model 904 is required to evaluate the machine learning model 506 because the machine learning model 506 must be backward compatible. For example, the machine learning model 506 may identify "cup" objects, "plate" objects, and "fork" objects. The training module 902 may then train the machine learning model 506 to identify "camera" objects. After the training module 902 trains the machine learning model 506 to recognize "camera" objects, the new machine learning model 506 needs to be evaluated to ensure that it is still able to recognize "cup" objects, "pan" objects, "fork" objects, and now "camera" objects.

In some implementations, the reference data may include a high quality data set. The high quality data set may include a high quality embedding that almost represents an ideal embedding. High quality embedding may be indicated by an indicator in database 114. The high quality embedding may include high quality image data, high quality RGBD data, high quality embedding, or a combination of data.

In some implementations, the evaluation module 904 retrieves the feature data 924 of the data 906 from the database 114. The data 906 includes similar data found for a particular object in the data set 406. For example, data 906 of a particular object, such as "dish," includes feature data describing the dish, a classification tag 922 of the "dish," and an insert 923 representing the dish. In another example, the data 906 of the "cup" object includes feature data 924 of the cup, a classification tag 926 of the "cup" and an insert 927 representing the cup. The assessment module 904 provides the feature data 924 of the cup as input to the machine learning model 506. The trained machine learning model 506 generates the embeddings 928 and the corresponding class labels 930.

In some implementations, to evaluate the machine learning model 506, the evaluation model 904 performs a test under the comparison module 932 to compare the embeddings 928 generated by the machine learning model 506 to one or more embeddings of similar object types stored in the database 114. If the results match, the evaluation model 904 may determine that the machine learning model 506 is able to properly recognize the particular object and thereby test the next object from the database 114.

In some implementations, the comparison module 932 includes a test of the evaluation module 904 to evaluate the quality of the machine learning model 506 using the heuristic model 919. Further, the comparison module 932 includes a test of the evaluation module 904 that utilizes the threshold comparison 921. For example, the evaluation model 904 may compare the embedding 928 to the embedding stored in the database 114 using one or more heuristic models. Database 114 may store a plurality of embedments for "cup" objects. The evaluation model 904 can use one or more heuristic models to compare the embeddings 928 generated by the machine learning model 506 to the plurality of embeddings in the database 114. As a result, the evaluation module 904 can determine whether the machine learning model 506 produced the correct embedding 928.

In some implementations, the evaluation model 904 can use the threshold comparison 921 to determine whether the machine learning model 506 produced the correct embedding 928. For example, the assessment model 904 may use vector comparisons to compare bits or bytes between the embedment 928 and each embedment in the database 114 representing a "cup" object. The evaluation model 904 may set a threshold, for example, no more than 10 lines may differ between two embeddings, so that the embeddings 928 pass the evaluation test. The evaluation model 904 may set other threshold comparison values. The evaluation model 904 utilizes a threshold comparison test 921 to determine if the machine learning model 506 produced the correct embedding 928.

In some implementations, the evaluation model 904 provides an indication 934 to the server system 112 after evaluating the machine learning model 506. Indication 934 may indicate that machine learning model 506 is properly trained. As a result, the server system 112 may send the machine learning model 506 to the robots 104A-104D for storage. Alternatively, the indication 934 may indicate that the machine learning model 506 does not recognize one or more objects from the database 114. For example, the machine learning model 506 recognizes "cup" and "vessel" objects, but does not recognize "glasses" objects. As a result, the server system 112 provides the machine learning model 506 to the training module 902 to retrain the machine learning model 506 to detect "glasses" objects. Further, the machine learning model 506 may not recognize a plurality of objects, and the training module 902 may be used to retrain the machine learning model 506 to detect each of the plurality of objects. Once the evaluation module 904 creates an indication 934 indicating that the machine learning model 506 is properly trained, the server system 112 sends the machine learning model 506 to the robots 104A-104D.

In some implementations, the server system 112 sends the machine learning model 506 to each of the robots 104A-104D in place of the machine learning model 504 stored on each of the robots 104A-104D. Robots 104A-104D remove their previous machine learning model 504 in response to receiving machine learning model 506 from server system 112. In addition, robots 104A-104D purge data sets 406 from their respective caches.

Fig. 10 is a flow chart illustrating an example of a process 1000 for evaluating and sharing information learned by a robotic. Process 1000 may be performed by one or more computers. For simplicity, the process is described as being performed by server system 112, but any suitable computing system or combination of computing systems may be used.

In process 1000, server system 112 receives object classification examples from a plurality of robots (1002). Each object classification example may include (i) an embedding generated by a robot using a machine learning model, and (ii) an object classification corresponding to the embedding. Each robot 104A designates each data set 406 as "new", indicating to the server system 112 that each data set 406 is to be distributed to other robots. In some implementations, each data set 406 can include an embedding, a respective class label, a respective feature or sensor data, and additional metadata corresponding to the embedding.

In each object classification example, the embedding that a robot generates with its machine learning model is based on sensor data generated by the robot that describes objects detected by the sensors of the robot. For example, a robot captures sensor data of a "cup" object with a sensor of the robot. The robot provides the sensor data as input to a local machine learning model to generate an embedment from the sensor data. The embedded corresponding object classification indicates a classification for the detected object. For example, a user may provide a sort tab of "cup" for a "cup" object by interacting with the client device 108 or with the robot itself.

In some implementations, the embedding of the respective classification examples includes embedding generated by different robots, where each robot generates the embedding using a particular machine learning model having a particular training state. For example, each robot 104A-104D includes the same machine learning model 402 that includes the same structure and training states. This allows robots 104A-104D to compare the robot-generated embeddings with the embeddings provided by other robots. If two robots use different machine learning models, the two robots may produce different embeddings for the same object, causing misclassification of the object.

In some implementations, the server system 112 receives, for each of the received object classification examples, sensor data generated by the robot describing an object corresponding to the object classification example. For example, the server system 112 receives sensor data including data indicative of the shape, color, size, contour, and texture of an object. The server system 112 evaluates the quality of the classification examples based on the sensor data received from each of the robots 104A-104D.

The server system 112 evaluates the object classification examples based on the similarity of the received embeddings of the object classification examples to other embeddings (1004). The quality assurance system 702 of the server system 112 provides embedded quality assessment received from each of the robots 104A-104D. A quality analysis module 704 included in the quality assurance system 702 generates a quality score indicative of the quality of the embedding in the data set 406 received from each robot 104. The quality analysis module 704 evaluates the embedded information by evaluating and filtering the embedded information using various criteria, including comparing distances between the embeddings when mapped to a high-dimensional space.

The server system 112 evaluates the object classification examples by determining a quality score for a particular embedding of a particular object classification example. The server system 112 determines a quality score by analyzing the horizontal quality of the sensor data provided by the plurality of robots. The server system 112 analyzes the level quality by providing the sensor data to a machine learning model stored on the server system 112 to determine a match between the sensor data and the embedment. The machine learning model provides an output embedding that the server system 112 compares to the particular embedding received. Further, the server system 112 compares the output classification labels from the machine learning model with the object classification corresponding to the particular embedding from the particular object classification example. In response to the server system 112 determining that the output classification label from the machine learning model does not match the object classification corresponding to the particular embedding, the server system 112 retrains the machine learning model to detect the particular embedding and generates a correct classification label that matches the object classification.

Further, the server system 112 determines a quality score by analyzing how the object classification examples adjust the overall accuracy of the machine learning model. For example, if the object classification example improves the overall accuracy of the machine learning model, the server system increases the quality score corresponding to that particular embedding.

The server system 112 also determines a quality score by analyzing previous quality levels of other object classification examples provided by the plurality of robots. For example, server system 112, in analyzing the particular inlay received from robot 104B, analyzes previous sensor data, classification examples, and corresponding quality scores from robots 104A-104D. The server system 112 may use this prior data to determine a quality score for a particular embedding from, for example, the robot 104B. In addition, the server system 112 may use this prior data to determine quality scores for particular embeddings from any of the robots 104A-104D of similar object types. The greater the similarity between the current data and any past data from any of the robots 104A-104D that produced a high quality score, the greater the quality score for that particular embedding will be.

In some implementations, the quality score represents a likelihood that the object class matches the object corresponding to the particular object class example. For example, the quality score indicates a likelihood that the object classification of "cup" matches the actual object corresponding to the received inlay (e.g., the received inlay indicates a "cup" object). The higher the likelihood that an object class assigns an object corresponding to an embedding received from a robot (e.g., robot 104A), the greater the quality score.

In some implementations, the quality assurance system 702 of the server system 112 determines, for a particular embedding of a particular object classification example, a level of similarity of the particular embedding relative to one or more other embeddings having the same object classification as the particular object classification example. For example, the quality assurance system 702 determines a level of similarity of the received embedment of the "cup" object to one or more other embedments of the "cup" object. The quality assurance system 702 may determine the level of similarity by comparing the distances between two embeddings mapped in the high-dimensional space 802. The closer the distance (e.g., vector distance) between two embeddings, the higher the level of similarity between the two embeddings.

The server system 112 selects a subset of object classification examples based on the assessment of the embedded quality (1006). For example, the mass analysis module 704 may designate an embedding that falls within a threshold region as an embedding to be sent to the robots 104A-104D. A higher likelihood of embedding the representation object is indicated within the threshold region. On the other hand, the mass analysis module 704 stores the embedments that exist outside the threshold region 827 in the database 114. In other implementations, the quality analysis module 704 discards any embeddings that exist outside a threshold distance from the ideal embedment. By discarding the embedding, the corresponding data set is also discarded.

The server system 112 selects a particular object classification example to include in the subset based on the quality score meeting a threshold. In some implementations, the server system 112 selects a particular object classification example, e.g., representing the embedding of a "cup" object, based on a threshold value with a quality score greater than 60. If server system 112 determines that the quality score for the particular object classification example is not greater than the threshold, server system 112 does not include the particular object classification example in the subset. Further, server system 112 may store the particular object classification example in database 114. Alternatively, server system 112 may discard the particular object classification instance.

In some implementations, the server system 112 evaluates the object classification example by determining, for a particular embedding of the particular object classification example, a level of similarity of the particular embedding relative to one or more other embeddings having object classifications that are different than the particular object classification example. For example, the server system 112 may compare the distance between the received embedment representing the "cup" object and the embedded of the map representing the "pencil" object in the high-dimensional space. Further, the server system 112 may compare the distance between the received embedment representing the "cup" object and the embedded of the map representing the "hat" object in the high-dimensional space. The distance between the received embedding and the embedding of the other mapping may be measured in a high-dimensional space as a vector distance. As a result, the server system 112 determines a level of similarity between the received embeddings and one or more other embeddings based on how close they exist to each other in the high-dimensional space.

The server system 112 distributes the subset of object classification examples to robots of the plurality of robots (1008). The server system 112 sends a subset of the object classification examples to each of the plurality of robots and specifies that the sent classification examples are stored in a local cache for each robot. For example, the server system 112 tags a subset of the object classification examples with a tag, such as "store," to indicate to the robots that they are stored in their respective caches. Although the cache is described as being local to the robot, the robot may also utilize network-based storage or "cloud" storage in some instances. Thus, depending on the implementation, the robot may store some or all of the cached data locally or remotely. In addition, using remote storage for the cache may be separate from shared embedding for distribution, as each robot may have a private cache or network storage account dedicated to that robot.

The server system 112 distributes a subset of the object classification examples to robots 104A-104D that each store a copy of the same machine learning model 402. Each machine learning model 402 includes the same training states and similar structures. The server system 112 distributes a subset of the object classification examples in response to the quality analysis module 704 analyzing the received embeddings.

In some implementations, the server system 112 synchronizes the caching of robots of the plurality of robots such that each of the plurality of robots stores the classification examples in the selected subset in the local cache of that robot. For example, the server system 112 may synchronize local caches between each of the robots 104A-104D periodically daily, weekly, or monthly. After synchronization, each of the plurality of robots includes the same classification instance in its respective local cache.

In some implementations, the quality analysis module 704 generates a quality score that indicates the quality of the embedded analysis. The quality score indicates how confident the quality analysis module 704 is for the received embedded representation of the particular object.

In some implementations, the quality threshold 704 compares the quality score for each data set 406 to the quality threshold. If the quality analysis module 704 determines that the quality score is greater than the quality threshold, the quality analysis module 704 allows the server system 112 to send the data set to each robot 104. Robots 104A-104D store the new data sets in respective local caches 404.

In some implementations, an indication of whether the data set 406 is shared by the quality analysis module 704 is stored with the data set 406 in the database 114. The server system 112 will use the indications stored in the database 114 to determine whether to use a particular embedding to generate the updated machine learning model 506. For example, if database 114 stores an indication that the embedment is not shared, server system 112 will not use the embedment to generate updated machine learning model 506.

In some implementations, the server system 112 trains the second machine learning model with the selected subset of classification examples. For example, the server system utilizes the selected subset of classification examples to generate an updated machine learning model 506. The server system 112 receives, for each object classification example, sensor data describing an object corresponding to the object classification example. The server system 112 may receive sensor data from each of the robots 104A-104D. For example, the sensor data may include the shape, color, size, contour, and texture of the object. Further, the sensor data may include red, blue, green and depth data provided by RGBD sensors.

During training, server system 112 generates an updated version of machine learning model 504. For example, the server system 112 updates the training state and structure of the machine learning model 504 to produce an updated machine learning model 506. Once training has progressed to the point where the server system 112 determines that the correct output can be properly derived from the updated machine learning model 506, the server system 112 provides the machine learning model 506 to the robots 104A-104D.

The server system 112 distributes the trained second machine learning model 506 to robots of the plurality of robots. The trained second machine learning model 506 replaces the machine learning model stored at each of the plurality of robots. Further, the server system 112 provides instructions to each of the plurality of robots to remove the distributed classification examples from the robot's local cache. The trained second machine learning model 506 includes the ability to detect objects represented by the removed distributed classification examples. As a result, classification examples that do not require distribution are stored by local caches in each robot.

Fig. 11 is a diagram illustrating an example system 1100 for enhancing robotics learning. As described above, a robot can quickly acquire knowledge about its environment by associating information with an embedding representing observations made by the robot. For example, a robot may capture image data showing objects that the robot is unaware of. The robot may use a machine learning model, such as a neural network object classification model, to process sensor data describing the object and generate an embedding, such as a vector or a set of values, that represents the object in encoded form. If the object classification model is unable to determine the correct classification of the object, a human user or another robot or system may be able to provide the correct classification. The robot may then store the embedding of the object in association with the classification of the object in order to learn quickly about the object. When the object is observed again, the robot may process the sensor data and compare the resulting embedding with the stored embedding. When the generated embeddings match or are sufficiently similar to the stored embeddings, the robot may determine that the classification of the stored embeddings is appropriate.

The ability to generate, store, and use embeddings for object classification and other tasks may allow robots to quickly and efficiently learn to work in new environments and situations. For example, generating and storing an embedding requires a very small amount of time and computation and allows a robot to expand its knowledge base almost instantaneously. In contrast, retraining machine learning models often requires many training examples and computationally demanding iterative training processes.

Another advantage of learning using embedded robots is the ability of the robots to easily exchange embeddings with other robots using the same machine learning model. This allows for rapid dissemination of learned information among the clusters of robots. However, it may not be desirable for a robot to share all of its generated embeddings with a remote server system or with other robots in the cluster, especially if the embeddings represent private or confidential information. For example, a robot may share embeddings with a server system in relation to general objects (e.g., chairs, kitchen tools), but may not share generated embeddings in relation to proprietary objects (e.g., proprietary machines), personal objects (e.g., user favorite mugs), objects containing personally identifiable information (e.g., credit cards), or other sensitive objects (e.g., precious paintings or other objects). To control the use of robotically learned information, some embeddings may be designated as shared "public embeddings" and other embeddings may be designated as non-shared "private embeddings". While robots do not share private embeddings with other robots or central server systems (for distribution), robots may retain private embeddings in local caches so that they retain the benefit of learned information represented by these embeddings. While the embedded and associated classifications represent learned objects, the private embedding may enable the robot to recognize the corresponding objects as they encounter them.

The robot may determine the sharing status of the embedding (e.g., public, private, or restricted sharing) by analyzing information related to the embedding (e.g., classification, metadata, tags, labels, sensor data, or the embedding itself) using a set of sharing rules. In some cases, the sharing rules may be predetermined and may be based on any of a variety of factors, including the classification of the embedding, the location of the object associated with the embedding, or other factors. In some implementations, the user of the robot can customize the sharing rules. In some examples, the user may indicate that the robot-specific object is private such that its embedment is not shared with other robots in the server system or cluster. Similarly, robots may be configured to share information related to the embedding of objects observed while the robot is in certain locations (e.g., public), but not in relation to objects observed in other locations (e.g., in private homes). In some cases, robots may be configured to share a particular embedding with only a limited subset of other robots. For example, robots may share some embeddings only with other robots in similar locations (e.g., between robots in a factory) or with other robots of similar roles (e.g., between robots that are home assistants).

In general, an embedding generated with a machine learning model has a unique meaning that is tied to the training state of the machine learning model. Embedding may represent the result of processing the input data with a machine learning model, such as information generated at the output layer of the neural network or at the hidden layer of the neural network in the case of a neural network model. For a given observation (e.g., an object detected in the image data), different machine learning model parameters will produce different embeddings for the same set of input data. As a result, changing from the first machine learning model to the second machine learning model would mean that the previously stored embeddings generated with the first machine learning model would no longer be useful for comparison with the embeddings generated by the second machine learning model. In other words, changing the machine learning model will likely cause learning loss represented by the embedding of the cache.

For this reason, updated machine learning models that are not trained with data about objects may not be able to correctly classify the objects, even with the embeddings generated by previous models. However, the robot may maintain the set of knowledge represented by the embeddings by re-generating a new embedment upon receipt of the updated machine learning model. The robot may generate a new embedding for the object using the new machine learning model and use the new embedding to continue to recognize the object while using the new network model.

In some implementations, the robot can store raw or processed sensor data associated with the generated private embedding. For example, the robot may store features, statistics, or scores extracted from sensor data related to privacy embeddings. When the robot receives the updated machine learning model, the robot may utilize the updated machine learning model and stored raw or processed sensor data associated with each private embedding to regenerate each private embedding in its local cache. The robot may then store the regenerated privacy embedment in its local cache along with the associated classification and raw or processed sensor data. The robot can then use the regenerated privacy embeddings to recognize and classify relevant privacy objects using its updated machine learning model.

In some implementations, after updating the machine learning model or regenerating the private embedding, the robot may remove the private embedding generated by the previous version of the machine learning model from its local cache. In some implementations, the robot may retain information associated with the private embedding, including metadata, raw or processed sensor data, object classification, and shared status, and associate the information with the regenerated private embedding.

These techniques may provide one or more of the following advantages. By designating some of the embeddings as private embeddings that are not shared with other robots in the server system or cluster, individual robots can each protect the privacy of the users of the robots while still allowing the robot cluster to benefit from shared learning of information that is not private or confidential. The robot may receive and use the embeddings shared by other robots while still using its own private embeddings to recognize and classify its surrounding private objects. In addition, the particular robot may share the generated non-private embeddings (i.e., restricted or public embeddings) with the server system for distribution such that other robots in the cluster may receive and use these public or restricted embeddings without compromising the privacy of the user of the particular robot. By allowing a user to customize the sharing rules used to determine the embedded sharing status, the user can adjust the application to the embedded privacy according to their preferences. For example, a user who prefers a higher level of privacy may set sharing rules that allow sharing of less embedded with other robots in the cluster. By storing raw or processed sensor data related to the privacy embedding, the robot may regenerate the privacy embedding after the robot receives the updated machine learning model. By storing the regenerated privacy embeddings in their local caches, the robot can utilize its updated machine learning model to recognize and classify objects related to the privacy embeddings, which may not have been trained based on these particular objects.

Still referring to fig. 11, in system 1100, robot 1110 receives data from one or more sensors of robot 1110. The computer system of robot 1110 uses the sensor data to identify and classify one or more objects in the perimeter of the robot. In some implementations, the computer system of the robot 1110 can include generating an embedded machine learning model related to the object identified by the robot 1110. In some cases, the user may teach the robot classification of the object. For example, the robot may be located in a user's home and may identify an object that is a mug. The machine learning model of the robot 1110 may generate an embedding related to a mug and the user may teach that the object identified by the robot is a mug belonging to a particular resident of the home (i.e., a "joe mug"). The robot 1110 may then store the embeddings associated with the object in a local cache of the robot 1110, as well as sort "joe mug". By storing the embeddings and classifications in the robot's local cache, the robot can recognize the object as a "joe mug" in the future without retraining the machine learning model.

In some implementations, the robots may share the embeddings stored in their caches with a remote server system, which may then distribute these embeddings to other robots in the cluster. In some cases, a robot may wish to share only some of the embeddings stored in its cache with other robots. For example, a robot may not share its embeddings with other robots that are determined to be related to proprietary or personal objects, objects containing personally identifiable information, or other sensitive objects. To indicate which embeddings it will share and which embeddings it will not share, the robot may designate the embeddings as "public" (to be shared), "private" (not shared), or "restricted" (to be shared with a subset of robots). For example, a robot located in a home may store an inlay associated with an object classified as a "teacup". Because "teacup" is a generic object, the robot can determine that the embedment associated with "teacup" can be shared with the robot cluster and thus designate "teacup" embedment as "public". Alternatively, the robot may also store the embeddings associated with objects classified as "joe mugs". Because "joe's mug" is a personal object, the robot may determine that the embedment associated with "joe's mug" may not be shared with the robot cluster and thus designate the "joe's mug" embedment as "private".

In some implementations, robots may designate an embedding as "constrained" in which the robot shares the embedding with a subset of other robots (e.g., only with robots of a particular location or role). For example, a robot located at home may determine that even though the "joe mug" is a personal object, the embeddings associated with it should be shared with other robots at home so that they can also identify it. In this case, the robot may designate a "joe mug" as a restricted insert that may be shared with only other robots in the home.

The system 1100 includes a robot 1110, which robot 1110 may have the same features as any of the robots 104A-104D discussed above. For example, robot 1110 may be a device with computing capabilities that are capable of moving or interacting with its surroundings. The robot 1110 may include a computing system that includes computer hardware, such as one or more processors, chipsets, general purpose computing systems, memory systems, and data storage systems. In some cases, robot 1110 may include specialized computing hardware including, but not limited to, microcontrollers, field programmable gate arrays (field programmable gate array, FPGAs), or application specific integrated circuits (application specific integrated circuit, ASICs). The computer hardware of the robot 1110 may be configured to execute software that controls the motions and processes of the robot 1110.

The computer system of robot 1110 may also include a local cache 1150. Local cache 1150 may be a memory system or a portion of a memory system of robot 1110 that stores data used by a computer system of robot 1110 for various computing functions.

The robot 1110 may also include sensors that collect data from the perimeter of the robot. For example, the robot 1110 may be equipped with one or more cameras that capture images of the robot's perimeter. The sensors may also include LIDAR, radar, proximity sensors, light detectors, or other sensors that enable the robot 1110 to detect and identify objects in its vicinity. In some implementations, the sensor data 1115 collected by the sensors may include images of the perimeter of the robot 1110 captured by one or more cameras of the robot 1110.

The perimeter of the robot 1110 may include one or

more objects

1120, 1121, 1130 that may be recognized by the robot 1110. Robots 1110 located in different environments may identify

different objects

1120, 1121, 1130 at their perimeter. For example, the robot 1110 in the home may identify

objects

1120, 1121, 1130 including furniture (e.g., tables, chairs), decorations (e.g., vases, paintings) and personal items (e.g., user favorite mugs, bed of a particular child, pet favorite toys). The robot 1110 in the factory may identify

objects

1120, 1121, 1130 that include equipment (e.g., machines, tools), protective devices (e.g., helmets, safety glasses), or output products (e.g., individually manufactured widgets, a box of widgets, etc.). The robot 1110 may identify any of a variety of objects around its perimeter. In the example system 1100, the

objects

1120, 1121, 1130 of the local perimeter of the robot include a mug (1121), a diamond ring and doll (1120), and a teacup and baseball cap (1130).

The robot 1110 may identify

objects

1120, 1121, 1130 in its surroundings by analyzing sensor data 1115 collected by sensors of the robot 1110. For example, a camera of the robot 1110 may capture sensor data images 1115 depicting one or

more objects

1120, 1121, 1130 of the robot's perimeter. The computer system of the robot 1110 may analyze the sensor data image 1115 to determine that there are

objects

1120, 1121, 1130 in the local perimeter of the robot. The computer system of the robot 1110 may use any of a variety of techniques to identify the

objects

1120, 1121, 1130. For example, the computer system may process the sensor data image 1115 containing an image of the object and then generate a bounding box within the image that follows the contour of the identified object. In the example system 1100, the sensor data 1115 includes an image depicting the object 1122, the object 1122 being a doll.

In some implementations, the computer system of the robot 1110 can process the sensor data 1115 to extract features related to the identified object 1122. For example, the computer system may process the sensor data 1115 to determine a physical characteristic of the object 1122, such as a location, size, dimension, shape, material composition, orientation, or other physical characteristic. In some cases, the processed sensor data 1115 may include extracted physical features, extracted statistical features or scores, or other metrics related to the object 1122 determined from the collected sensor data 1115. The computer system of the robot 1110 may also analyze or process the sensor data 1115 in other ways, including but not limited to filtering, transforming, truncating, annotating, or enhancing the sensor data 1115. In some implementations, the processed sensor data 1115 may include extracted features required as input by the machine learning model 1140.

In some implementations, the robot 1110 can collect or generate additional data related to the identified object 1122. For example, the computer system of the robot 1110 may collect or generate metadata related to the object 1122 including the location of the object 1122, the time and date the object 1122 was identified, or other information related to the object 1122. The metadata may also include other information related to the object 1122, e.g., the metadata may indicate a user associated with the object 1122. The metadata may describe the size, shape, material, composition, or other properties of the object.

In some implementations, the computer system of the robot 1110 can input raw or processed sensor data 1115 into the machine learning model 1140. The machine learning model 1140 may be part of a computer system of the robot 1110 and may be implemented in hardware, software, or any combination of hardware and software. The machine learning model 1140 accepts raw or processed sensor data 1115, as well as any other data provided to it by the computer system of the robot 1110, as input, and outputs an embedding 1142 corresponding to the object 1122 identified from the sensor data 1115. For example, the machine learning model 1140 may accept as input processed sensor data 1115 including the extracted features of the object 1122. The machine learning model 1140 may also accept metadata associated with the sensor data 1115 or the object 1122 as input, such as location information indicating where the sensor data 1115 was collected. Based on the input sensor data 1115 and metadata, the machine learning model 1140 may generate an embedding 1142 related to the identified object 1122.

In some implementations, the machine learning model 1140, like the other machine learning models discussed herein, may include a neural network or other type of classifier. For example, the machine learning model 1140 may be a parametric model, a non-parametric model, or a semi-parametric model. Which may be a statistical model or an empirical model. The machine learning model 1140 may include various probability classifiers, including bayesian classifiers. The machine learning model 1140 may be implemented in any combination of hardware, firmware, and/or software.

The embedding 1142 may be, for example, a multidimensional vector representation of the output of the machine learning model 1140. For example, one embedded representation may represent a 128-element vector, where each element is a 64-bit integer. For clarity of illustration, the embedding 1142 is represented as a 12 bit integer. In the example of fig. 11, computer of robot 1110 provides sensor data 1115 related to doll 1122 to machine learning model 1140. Based on input sensor data 1115, machine learning model 1140 outputs an embedding 1142 of "129057892039" associated with doll 1122.

In general, the embeddings 1142 generated by the machine learning model 1140 are unique functions of the raw or processed sensor data 1115 related to the object 1122 and the particular training state of the machine learning model 1140. For example, two machine learning models 1140 using different parameters or model structures will generally generate two different embeddings 1142 for the same input sensor data 1115 related to the object 1122. As a result, the embedding 1142 is generally only useful or meaningful when used with the machine learning model 1140 that uses the same parameters and training states as that used to generate the embedding 1142.

In some implementations, the machine learning model 1140 may also generate a classification related to the object 1122. For example, based on the embedding 1142, the machine learning model 1140 may generate a classification for the object 1122 from a set of known classifications. However, for some objects 1122, the user may wish the robot 1110 to assign a different classification to the object 1122 than the classification generated by the machine learning model 1140. For example, the user may wish to correct an incorrect classification generated by the machine learning model 1140 (e.g., the machine learning model 1140 classifies the object 1122 as a "pencil" and it is actually a "pen"). As another example, a user may wish to classify object 1122 more specifically or differently than the classification generated by machine learning model 1140 (e.g., machine learning model 1140 classifies object 1122 as "mug" and the user wants robot 1110 to classify object 1122 as "joe mug").

In the event that the user wishes the robot 1110 to classify the object 1122 differently than the generated classification, the user may teach the robot 1110 to associate a new classification 1147 with the object 1122 by providing user input 1145 to the robot 1110. The user may provide user input 1145 to the robot 1110 by any of a variety of means, including, for example, by voice commands, by a peripheral device (e.g., keyboard, mouse, touch screen) in communication with the robot 1110, or by a wireless or wired network connection. In the example system 1100, the user desires the robot 1110 to classify the object 1122 not simply as a "doll" but as a "doll Suxi". Thus, the user provides user input 1145 to the robot 1110 indicating that the object 1122 is "doll Suxi" and the robot's computer system associates the classification 1147 "doll Suxi" with the embedding 1142 associated with the object 1122.

The computer system of the robot 1110 may then store the embedded 1142 and the user input classification 1147 related to the object 1122 in a local cache 1150 of the robot 1110. By storing the embeddings 1142 and the classifications 1147 in the robot's local cache 1150, the computing system of the robot 1110 can recognize the object 1122 as a user-provided classification 1147 in the future without retraining the machine learning model 1140. In some implementations, the computer system of the robot 1110 may also store some or all of the raw or processed sensor data 1115 and metadata related to the embedding 1142 in the local cache 1150, as well as a version identifier for generating a machine learning model of the embedding 1142.

In the example of fig. 11, the computer system of robot 1110 stores, in entry "3" of local cache 1150, embedded 1142"129057892039", classification 1147 "doll sulci", sensor data 1115, and a version of the machine learning model used by machine learning model 1140 to generate embedded 1142 (version "1.0"). After storing the embeddings 1142 and the classifications 1147 in the local cache 1150, the next time the robot 1110 encounters the object 1122, it will be able to identify the object 1122 as "doll Suxi".

In some implementations, when the robot 1110 is part of a cluster of robots, the robot 1110 may share the embeddings 1142 from its local cache 1150 with a remote server system that distributes the embeddings 1142 to other robots in the cluster. In some cases, the robot 1110 may share only some of the embeddings 1142 stored in its cache 1150 with other robots. For example, robots 1110 may share an insert 1142 representing a general object, such as a chair, or a book, but robots 1110 may not share an insert 1142 representing a personal object, such as a driver's license or a favorite toy of the user, or a valuable object, such as a piece of jewelry or an expensive drawing.

The robot 1110 may not share the embeddings 1142 associated with the object 1122 for any of a variety of reasons. For example, if the associated object 1122 is considered limited or confidential, the robot 1110 may not share the embedding 1142. For example, robots 1110 in a factory may not share an embedding 1142 associated with a proprietary machine or instrument. Robots 1110 in the hospital may not share the embeddings associated with objects 1122 in the patient's room. The robots 1110 may not share the embedded 1142 that contains personally identifiable information (e.g., a driver's license or an ID card). In some implementations, the robot 1110 may not share the embedding 1142 if the embedding 1142 is associated with the personal object 1122. For example, the robots 1110 in the home may not share the embedment 1142 associated with the homeowner's house key or with the resident's favorite mug.

In some cases, robots 1110 may not share embeddings 1142, as the user instructs robots 1110 to not share information about a certain object or class of objects. For example, the user may instruct the robot 1110 to not share any embeddings 1142 related to precious jewelry in the user's home or the user may instruct the robot 1110 to not share any embeddings related to objects located in the user's bedroom. In general, the user may instruct the robot 1110 not to share a particular embedding 1142 associated with any object 1122 that it wishes to keep private.

In some cases, robots 1110 may share embeddings 1142 with only a limited subset of other robots. For example, robots 1110 may share a particular embedding 1142 with other robots in the same organization (e.g., all robots of a given company or a given restaurant), with other robots of a particular type or domain (e.g., all hospital robots or all kitchen service robots), or with other robots of a given location (e.g., all robots in the home). In other cases, the robot 1110 may share the embedding 1142 with all other robots in the cluster.

To indicate which embeddings 1142 the robot 1110 will share and which embeddings 1142 it will not share, the robot 1110 may associate each embedment 1142 with a shared state 1162. In some implementations, the robot 1110 can determine the sharing status 1162 of each of the embedments 1142 as "public" (those shared), "private" (those not shared), or "restricted" (those shared with a subset of the robots). To determine the sharing status 1162 of the embedding 1142, the computer system of the robot 1110 may analyze information related to the embedding 1142 against a set of sharing rules 1160.

Sharing the restricted embeddings 1142 to a subset or group of robots may be accomplished via an access control list, wherein the computer system of the robot 1110 maintains a list of robots with which one or more restricted embeddings 1142 may be shared. In some implementations, the computer system of the robot 1110 can maintain different access control lists for different restricted embeddings 1142.

The computer system of the robot 1110 may use any of a variety of information related to the embedding 1142 to determine the sharing status 1162 of the embedding 1142. For example, the computer system may use the embedded 1142 itself, the classification 1147 associated with the embedded 1142, or metadata associated with the embedded 1142. In some cases, the computer system may use metadata that indicates where the object related to the embedding 1142 is located or when the embedding 1142 was generated. The computer system of the robot 1110 may also use other data or metadata associated with the embedding 1142 to determine the sharing status 1162 of the embedding 1142. For example, the computer system may use the sensor data 1115, the classification generated by the machine learning model 1140, the location of the robot 1110 (e.g., from GPS), other data from sensors of the robot 1110, information provided by a user, or other information to determine the sharing status of the embedment 1142.

The sharing rules 1160 provide criteria that enable the robot 1110 to determine the sharing status of a particular embedding 1142. The sharing rules 1160 may be stored, for example, in a memory system of the robot 1110 and accessed by a processor of a computer system of the robot 1110. In some implementations, the sharing rules 1160 may be predetermined (e.g., a "default" rule). In some implementations, the sharing rules 1160 may be learned by the robot 1110 over time.

Different robots 1110 may utilize different sharing rules 1160. For example, robots 1110 in a factory may use a set of sharing rules 1160, while robots 1110 in a hospital may use a different set of sharing rules 1160. In some implementations, the robot 1110 may utilize different sharing rules 1160 for objects 1122 associated with different users.

The sharing rules 1160 may include rules that consider any of a variety of factors that affect whether the robot 1110 embeds 1142 the sharing. For example, sharing rules 1160 may include rules that consider the location of object 1142 (e.g., in a bedroom, in a kitchen, outdoors), the type of object 1142 (e.g., gardening tool, piece of furniture, personal item, machine), the classification of object 1142 (e.g., personal item such as "joe's mug"), the location of robot 1110 (e.g., home, factory, restaurant, hospital), or other factors. For example, the sharing rules may be: any of the embeddings 1142 associated with the object 1122 located in the user's home are limited embeddings that are shared only with other robots in the home, or any of the embeddings 1142 associated with the object 1122 containing personally identifiable information are private embeddings.

In some examples, the robot 1110 can request that the user determine the sharing status 1162 of the embedment 1142 associated with the object 1122, wherein the robot 1110 specifies the sharing status 1162 of the embedment 1142 in accordance with the user's response. Here, the user may teach the robot 1110 the embedded shared state 1162 by providing input to the robot 1110, where providing input may be accomplished, for example, by providing voice commands to the robot or inputting data into the robot through a peripheral device (e.g., keyboard, mouse, or touch screen).

In some implementations, the user of the bot 1110 can customize, modify, or add to the sharing rules 1160. For example, the robot 1110 may have default sharing rules 1160 that indicate that any embeddings 1142 related to objects 1122 located in the user's office have private sharing status 1162. In some cases, the user may modify the sharing rules 1160 to indicate that only the embeddings associated with objects 1122 on the user's desk will have a private sharing state 1162, while the embeddings 1142 associated with any objects 1122 in the user's office but not on his desk will have a public sharing state 1162.

In some implementations, the user of the robot 1110 can designate the group of robots with which to share the restricted embedding 1142. For example, a user of robot 1110 may indicate that the embedding 1142 may be shared with only other robots in the same location, or that the embedding 1142 may be shared with only other robots in the same organization.

Based on the sharing rules 1160 and/or input from the user, the computer system of the robot 1110 determines the sharing status 1162 of each of the embeddings 1142 as public or private. The computer system of robot 1110 associates shared state 1162 with embedded 1142 and stores shared state 1162 in local cache 1150. In some implementations, the computer system of the robot 1110 determines the sharing status 1162 of the insert 1142 when the machine learning model 1140 generates the insert 1142.

In some implementations, the computer system of the robot 1110 may maintain one or more caches that store the embedments (e.g., one cache that stores the public embedment and a second cache that stores the restricted and private embedments). In some examples, the computer system of robot 1110 may allocate particular portions of the cache to store public, private, or restricted embedded and related information.

In the example system 1100, the perimeter of the robot 1110 includes a private object 1120 ("diamond ring" and "doll Suxi"), a restricted object 1121 ("joe's mug"), and a public object 1130 ("cup" and "baseball cap"). For each

object

1120, 1121, 1130, the robot 1110 has generated an embedding (based on sensor data) from the machine learning model 1140 and determined a classification (based on user input). For each identified

object

1120, 1121, 1130, the computer system determines a sharing status by analyzing information about the

object

1120, 1121, 1130 against the set of sharing rules 1160 and stores the sharing status in the local cache 1150 in association with the generated embeddings. Here, for example, the user designates "joe mug" as a restricted insert, shared only with other objects in the home.

In particular, in the example of FIG. 11, metadata associated with embedding 1142 indicates that doll 1122 is identified in the user's home. Sharing rules 1160 include a rule that states that any object 1122 located in the user's home is private unless specified by the user. Based on sharing rules 1160, computer system determines that an embedding 1142 associated with doll 1122 is private. Thus, the computer system of robot 1110 stores the "private" shared state 1162 associated with the embedding 1142 in local cache 1150.

In some implementations, the computer system of the robot 1110 can determine whether to store additional information associated with each of the embeddings 1142 based on the shared status 1162 of the embeddings 1142. For example, the computer system may store raw or processed sensor data 1115 associated with the embedding 1142 only if the embedding 1142 is private or limited.

Fig. 12 is a diagram illustrating an example system 1200 for selectively sharing embeddings among clusters of robots. In system 1200, one or more robots in a cluster of robots share a public embedding from their local caches with a remote server system, but do not share a private embedding. In some cases, robots may share limited embeddings with a subset of robots in a cluster. Here, the robots use the same machine learning model, so that the embedding generated by one robot can be used by another robot. The remote server system then distributes the shared embeddings to one or more robots in the cluster of robots. In this way, robots in a cluster can leverage learning of other robots by leveraging public and/or embeddings generated by another robot, such that each robot can recognize an object associated with the embeddings without first encountering the object itself and without retraining a machine learning model. Fig. 12 includes stages (a) and (B) of a flow of representing data.

The system 1200 includes a robot 1110 of the system 1100, the robot 1110 having an embedded collection generated by the robot 1110 in its local cache 1150. In the example system 1200, the local cache 1150 initially contains private, restricted, and private embeddings generated by the robot 1110, specifically, restricted embeddings designated as "1" in the local cache ("joe mug"), private embeddings designated as "2" and "3" in the local cache 1150 ("diamond ring" and "doll Suxi"), and public embeddings designated as "4" and "5" in the local cache 1150 ("teacup", "club cap").

The system 1200 also includes one or more other robots 1210. The one or more robots 1210 may be the same type of robot as the robot 1110 or a different type of robot than the robot 1110.

Robots

1110 and 1210, whether of the same or different types, can use copies of the same machine learning model to classify objects.

The system 1200 also includes a remote server system 1270. The remote server system 1270 may be, for example, one or more computer systems, servers, distributed computing platforms, or other processing systems. In some implementations, the remote server system 1270 may be a cloud computing system.

The robot 1110 and one or more other robots 1210 may exchange electronic information with a remote server system 1270 through a network 1280. The network 1280 may be any communication infrastructure that supports bi-directional exchange of data between the remote server system 1270 and the

robots

1110, 1210. Network 1280 may include a local area network (local area network, LAN), wide area network (wide area network, WAN), the internet, or other network topology. The network 1280 may be any one or combination of wireless or wired networks and may include ethernet, cellular telephone, and Wi-Fi technology. Communication over the network 1280 may be accomplished through any one or combination of various protocols, including 802.11, bluetooth, GSM, 3G, 4G, 5G, LTE, or other custom or standard communication protocols. In some implementations, the network 1280 may include an optical data link.

In stage (a),

robots

1110, 1210 send information to server system 1270 regarding shared (public and restricted) embeddings stored in their local caches.

Robots

1110, 1210 can send various information related to shared embedding to server system 1270. For example,

robots

1110, 1210 may send the embeddings themselves, classifications, versions of machine learning models used to generate the embeddings, and any other information or metadata related to sharing the embeddings. However,

robots

1110, 1210 do not send information to server system 1270 regarding private embeddings stored in their local caches.

In the example system 1200, the robot 1110 sends information associated with the restricted insert "1" and the common inserts "4" and "5" to the server system 1270, which are related to "joe's mug", "teacup", and "baseball cap", respectively. Robot 1110 does not send information associated with privacy inserts "2" and "3" to server system 1270, which are associated with privacy objects "diamond ring" and "doll Suxi", respectively.

Similarly, during stage (a), robot 1210 sends information associated with public and/or restricted embeddings stored in its local cache to server system 1270. The bot 1210 may not send information associated with any private embeddings stored in its local cache to the server system 1270.

In stage (B),

robots

1110, 1210 receive the embeddings provided to server system 1270 by the other robots from server system 1270.

Robots

1110, 1210 may receive information related to some or all of the shared embeddings provided by other robots to server system 1270.

Robots

1110, 1210 may store information associated with these received embeddings in their local caches, and they may use the received embeddings for object identification and classification in the same manner as they use their generated private and public embeddings.

In some implementations, the server system 1270 may share a common embedding with all

robots

1110, 1210 in the cluster, but may share limited embedding with only a designated subset of

robots

1110, 1210 in the cluster. For example, server system 1270 may send to robots 1110 those common embeddings that are shared by other robots 1210, as well as those limited embeddings that are shared by robots that operate in similar settings with robots 1110 (e.g., sharing limited embeddings between robots that operate in a room) or those limited embeddings that are shared by robots that operate in similar roles with robots 1110 (e.g., sharing limited embeddings only between work robots in a factory). A subset of

robots

1110, 1210 with which a particular embedment is shared may be indicated by the user who generated the embedment, by the server system 1270, by a system administrator, or by another means.

In the example system 1200, the robot 1110 receives information about a common embedding associated with one received object 1235 classified as "elm". In particular, in the example of fig. 12, robot 1110 receives the embedding, categorizes, and is configured to generate an embedded machine learning model version associated with elm 1235. The computer system of robot 1110 adds information associated with the received "elm" embedment to the local cache 1150 of robot 1110. In some implementations, the computer system of the robot 1110 may determine that the shared state of the received embeddings is "public" because it was generated by another robot 1210 as a public embedment. In some implementations, the computer system of the robot 1110 may determine that the received shared status of the embedding is "received," indicating that the embedding was received by the robot 1110 and was generated by a different robot 1210. The computer system of the robot 1110 may store additional metadata with the received embedment, such as the date and time the embedment was received, other robots 1210 that generated the embedment, the location of the embedment, or any other information related to the received embedment.

Similarly, during stage (B), in the example system 1200, other robots 1210 receive information related to a common embedding sent by the robot 1110 to the server system 1270. In the example of fig. 12, information associated with a common insert regarding "teacup" and "cap" inserts generated by robot 1110 is distributed to robot 1210 by server system 1270. The robot 1210 then stores information associated with the common embedment of the "teacup" and "baseball cap" in its local cache. In the example system 1200, none of the robots 1210 are in a group of robots authorized to receive the restricted embedment associated with "joe mug" (i.e., none of the robots 1210 are in the same home as robot 1110). As a result, the server system 1270 does not share a restricted embedded "joe mug" with the robot 1210.

After adding the received embedment to its local cache, the robot will be able to use the embedment to identify and classify objects associated with the received embedment. For example, in system 1200, while robot 1110 has not previously encountered elm and its machine learning model has not been trained to recognize elm, the next time robot 1110 encounters elm, it will be able to identify it as "elm" based on the received embedding. In this way, robot 1110 can utilize the learning of other robots 1210 in the cluster to improve its object recognition and classification performance.

Fig. 13 is a diagram illustrating an example system 1300 for robotic regeneration and storage of an embedding. In some implementations, robots having private and/or restricted embeddings stored in their local caches may receive updates to their machine learning models, for example, from a remote server system. Because the embedding is unique to a given machine learning model, the private and/or restricted embedding stored in the local cache of the robot generated using the previous machine learning model is not compatible with the updated machine learning model based on public embedding training. As a result, to enable the robot to continue to utilize the updated machine learning model to identify and classify objects associated with the privacy embedment, the computer system of the robot may generate a new updated privacy embedment. The computer system of the robot generates an updated privacy embedding by inputting raw and/or processed sensor data associated with the privacy embedding into the updated machine learning model. The robot's computer system then stores the regenerated privacy embedment in its local cache along with the associated raw and/or processed sensor data. In some implementations, the computer system removes privacy and/or restricted embeddings associated with previous machine learning models from the cache. Fig. 13 includes stages (a) to (C), which represent the flow of data.

System 1300 includes the same robot 1110 of systems 1100 and 1200, and a remote server system 1270 from system 1200. The computing system of robot 1110 includes a machine learning model 1340, and a local cache 1150. In fig. 13, as indicated, the local caching of the robot 1110 before and after the regeneration of the privacy embedding is shown.

In stage (a), the robot 1110 maintains the embedding in its local cache. Initially, the robot 1110 maintains a local cache 1150 ("pre-rebinning local cache") that stores private and public embeddings generated by the robot 1110, as well as received public embeddings distributed to the robot 1110 by the server system 1270. In example system 1300, robot's local cache 1150 stores one restricted insert generated by robot 1110 (insert "1" classified as "joe's mug"), two private inserts generated by robot 1110 (inserts "2" and "3" classified as "diamond ring" and "doll su" respectively), two public inserts generated by robot 1110 (inserts "4" and "5" classified as "teacup" and "building block" respectively), and one received public insert (insert "6" classified as "elm"). These embeddings are all generated using version 1.0 of the machine learning model, as indicated in the table of local cache 1150.

In stage (B), the robot 1110 receives the updated machine learning model 1343 from the server system 1270. In some cases, the updated machine learning model 1343 may have been updated and trained by the server system 1270 with common embedded information shared by the robot 1110 and other robots in the fleet with the server system 1270. As a result, a robot 1110 implementing the updated machine learning model 1343 may be able to properly classify objects related to a shared common embedding without reference to the embedding in its local cache 1150. However, because the updated machine learning model 1343 is trained solely based on public embeddings, in order to continue to properly classify objects associated with private and/or restricted embeddings, the robot 1110 must update and regenerate those embeddings with the updated machine learning model 1343.

In some implementations, the server system 1270 can send the updated machine learning model 1343 to the robot 1110 via a network, such as the network 1280 of fig. 12. In some examples, the updated machine learning model 1343 may be a complete replacement for a model implemented by the machine learning model 1340 stored in the computing system of the robot 1110. In some examples, the updated machine learning model 1343 may replace only some of the previous models implemented by the machine learning model 1340. For example, the server system 1270 may send some or all of the node weights, inter-layer node connectivity information, or intra-layer node connectivity information to update the machine learning model. In some examples, server system 1270 may update machine learning model 1340 of the robot by sending only those parameters (e.g., node weights, node interconnectivity) of machine learning model 1343 that were changed relative to previous models implemented by machine learning model 1340. In example system 1300, machine learning model 1340 of robot 1110 previously implemented version 1.0 of the machine learning model and server system 1270 sent updated version 2.0 of machine learning model 1343 to robot 1110.

After receiving the updated machine learning model 1343 from the server system 1270, the computer system of the robot 1110 updates (e.g., replaces) the machine learning model 1340 with the updated machine learning model 1343. In some implementations, the computer system of the robot 1110 can update the machine learning model 1340 by storing one or more parameters associated with the updated machine learning model 1343 in a memory system of the robot 1110. In some examples, the robot 1110 may overwrite or remove a previous version of the machine learning model from the memory of the computer system of the robot.

In stage (C), the computer system of robot 1110 regenerates the private and restricted embeddings stored in its local cache 1150. To regenerate the privacy embedment 1142, the computer system of the robot 1110 inputs the raw and/or processed sensor data 1115 associated with the raw embedment 1142 into the updated machine learning model 1340. In some examples, the sensor data 1115 includes the same raw or processed data 1115 used by the computer system of the robot 1110 to generate the raw embedding 1142 with a previous version of the machine learning model. The data used by the updated machine learning model 1140 may include raw sensor data 1115 collected by sensors on the robot 1142, may include processed sensor data 1115 of features of the object 1122 extracted by the computer system of the robot 1110, and other data, including metadata related to the embedding 1142. In some cases, the sensor data 1115 may be stored in a local cache 1150 of the robot.

Based on inputting the data 1115 into the updated machine learning model 1340, the machine learning model 1340 generates a new privacy and/or restricted embedment 1342. The computer system of robot 1110 associates this new embedding 1342 with the classification 1147 and sharing state 1162 associated with the previous embedding 1142. The new embedment 1342 may be different from the previous embedment 1142 stored in the local cache 1150 prior to the update. For example, the new embedding 1342 may contain a different value than the previous embedding 1142, or may have a different dimension.

The computer system of the robot 1110 then stores the regenerated embedded data 1355, including the new embedded 1342 and the classification 1147, shared status 1162, and sensor data 1115 associated with the embedded 1342, in a local cache 1150. In some implementations, the computer system of the robot 1110 may also save a version of the machine learning model 1343 that is used to generate the new privacy and/or restricted embedment.

In some implementations, after receiving updated machine learning model 1343 or after regenerating the previous privacy and/or restricted embeddings, the computer system of robot 1110 may remove the embeddings associated with the previous version of the machine learning model from the local cache. For example, the computer system of robot 1110 may remove information from the local cache about the privacy, public, restricted, and received embeddings generated by the previous version of the machine learning model.

For example, in system 1300, after updating machine learning model 1340 with updated version 2.0 machine learning model 1343, the computer system of robot 1110 generates three new embeddings in place of the three previous restricted and private embeddings (embeddings "1", "2" and "3") in its local cache 1150 ("local cache before regeneration") generated with version 1.0 of the machine learning model. The computer system of robot 1110 then associates the previously embedded classifications, sensor data, and shared status with the new embedding and stores the regenerated embedded data 1355 in its local cache 1150 ("regenerated local cache").

The example of fig. 13 shows the regeneration of the secret embedding "3" (1142), which is associated with object 1122 "doll sulci". The computer system of the robot 1110 generates a new embedding for the object 1122 by inputting sensor data 1115 related to the original embedding 1142 into the updated machine learning model 1340. Here, machine learning model 1340 implements an updated version 2.0 of machine learning model 1343 provided to robot 1110 by server system 1270.

The updated machine learning model 1340 generates a new privacy embedding 1342 of "554039487109" related to the object 1122. Here, the new privacy embedment 1342 of "554039487109" generated with machine learning model version 2.0 (1343) is different from the privacy embedment 1142 of "129057892039" generated with the previous machine learning model version 1.0, although these embedments are related to the same object 1122 and are derived with the same data 1115.

In the example of fig. 13, the computer system of robot 1110 associates new embedding 1324 with classification 1147 ("doll sulci"), shared state 1162 ("secret") and sensor data 1115 of previous secret embedding 1142, and then stores new secret embedding 1324 along with classification 1147, shared state 1162, sensor data 1115 and a version ("2.0") for generating the embedded machine learning model in local cache 1150 ("regenerated local cache"). The computer system of robot 1110 also removes from local cache 1150 any embedded information that was generated using previous machine learning model version 1.0. Here, the computer system of robot 1110 removes information from local cache 1150 about previous embeddings of private object 1120 and restricted object 1121, as well as information about public embeddings (items "4" and "5" relating to "teacup" and "baseball cap", respectively) and received embeddings (item "6" relating to "elm").

In some implementations, the server system 1270 can utilize both the common and restricted embeddings provided by the robot 1110 to generate an updated machine learning model 1343 (e.g., the updated model is trained utilizing both the common and restricted embeddings). In this case, the robot 1110 may be able to identify public objects and restricted objects using the updated machine learning model 1343 and the computer system of the robot 1110 may only regenerate and store private embeddings.

As new machine learning models are provided to the robot 1110 by the server system 1270, the system 1300 may repeat stages (a) through (C). For example, server system 1270 may send updated machine learning model version 3.0 to robot 1110. In this case, the robot 1110 will update its machine learning model 1140 to implement the updated machine learning model version 3.0 and generate a new private and/or restricted embedding in place of the embedded data 1355 generated with machine learning model version 2.0. The robot 1110 may also generate new embeddings for any additional private or restricted embeddings generated using machine learning model version 2.0.

Fig. 14 is a flow chart illustrating a process 1400 for the robot's regeneration and storage of privacy embeddings. Briefly, the process 1400 includes storing, by a robot, first embeddings generated using a first machine learning model in a local cache of the robot, wherein the first embeddings include one or more first private embeddings (1402); receiving (1404) a second machine learning model by the robot from the server system over a communication network; generating, by the robot, a second privacy insert for each of the one or more first privacy inserts using the second machine learning model (1406); adding, by the robot, a second privacy embedding to the cache of the robot (1408); and removing, by the robot, one or more first privacy embedments from the cache of the robot (1410). This process may be repeated each time the machine learning model of the robot is updated.

In more detail, the process 1400 includes storing, by the robot, a first embedding generated using a first machine learning model (1402). In some implementations, the first machine learning model may be a model for classifying objects identified by the robot. The machine learning model may include information describing node placement, layer placement, node interconnectivity, layer connectivity, interconnections, or node weights (e.g., synaptic weights), or other parameters describing the architecture and functionality of the machine learning model. The machine learning model may be implemented in any combination of hardware and software. In some implementations, one or more robots in a cluster of robots may implement the same machine learning model. In some implementations, the first machine learning model may have been provided to the robot by the server system. In some implementations, the robot may store a copy of the first machine learning model locally in hardware of the robot (e.g., in a cache of the robot). In some implementations, the robot can access a copy of the first machine learning model stored on a remote server system (e.g., a cloud computing system).

The robot may store a first embedding generated using the first machine learning model in a cache of the robot. The embedding may be, for example, an output of the first machine learning model representing the object. In some implementations, the embedding may be a multi-dimensional numerical value, one or more numerical values, a vector, a tensor, a matrix, an array, or another numerical representation.

In some implementations, the robot may generate an embedding for the object by inputting sensor data related to the object into the machine learning model. In some implementations, the sensor data may be collected by a sensor of the robot and may include image or video data, depth data, LIDAR, radar, IR, or proximity sensor data. In some implementations, the robot can process the sensor data before inputting the sensor data into the machine learning model to generate the embeddings. For example, the robot may process the sensor data to extract statistical or numerical feature values related to the object. The robot may also filter or transform the sensor data. The robot may input some or all of the raw, processed, filtered, or transformed sensor data to a machine learning model to generate an embedding. In some implementations, the robot may store some or all of the data used to generate the one or more first embedded sensors, including raw, processed, filtered, or transformed data.

The first embeddings can include one or more first private embeddings, wherein the first private embeddings are not shared with other robots or server systems. For example, a first privacy embedding may represent a private object (e.g., a personal object, a proprietary or confidential object, a valuable object, etc.) that a user of a robot does not wish to share with other robots of a server system.

The robot may determine that the first embedding is a first private embedding by any of a variety of techniques. For example, the robot may determine that the first embedding is a first private embedding by accessing a shared state associated with the first embedding.

In some implementations, the robot can access metadata associated with the embedding and based on analyzing the metadata against one or more sharing rules, the robot can determine that the embedding is a private embedding. The metadata may be, for example, a classification of the embedding, a location, a date or time stamp, another tag or label associated with the embedding, or an object of the embedded representation. The metadata may also include other information about the object embedded or generated or received by the robot. For example, metadata may include information describing the size, shape, material, composition, or other properties of an object. In some implementations, the robot can store metadata associated with the one or more first embeddings in a cache of the robot.

In some implementations, the robot may store one or more sharing rules in a local memory or cache of the robot. The robot may use one or more sharing rules to determine whether the identified embedment associated with the object is private. In some examples, the sharing rules may be predetermined (e.g., default rules). In some examples, the robot may receive the sharing rules from the server system. In some examples, the sharing rules may be customized or set by a user of the robot. In some examples, the robot may learn sharing rules over time (e.g., through machine learning or other automated learning techniques). The sharing rules may be based on any of a variety of factors, including metadata associated with the embedding, classification of the embedding, location of objects associated with the embedding, users of robots associated with the embedding, or other factors.

In some implementations, the first embeddings stored in the cache of the robot may include one or more common embeddings shared with the server system. The common embeddings may be, for example, embeddings representing common objects that a user of a robot wishes to share with other robots in a cluster of robots.

Similar to private embedding, the robot may determine that a particular embedding is a public embedding by any of a variety of techniques. For example, the bot may analyze metadata associated with a particular embedding against one or more sharing rules to determine that the embedding is a public embedding.

In some implementations, where the first embeddings include one or more first public embeddings and one or more first private embeddings, the robot may provide the one or more first public embeddings to the server system and not provide the one or more first private embeddings to the server system.

In some implementations, the robot may determine whether to store sensor data related to one or more first embeddings based on whether the particular embeddings are private or public. For example, the robot may determine to store sensor data for each of the one or more first private embeddings, but not for the one or more first public embeddings.

The robot may receive a second machine learning model from the server system over the communication network (1404). The second machine learning model may provide the same functionality as the first machine learning model. In some examples, the second machine learning model may be an updated or retrained version of the first machine learning model. The second machine learning model may have been updated or trained with data provided by one or more robots in the cluster. In some implementations, the second machine learning model may be a one-to-one replacement of the first machine learning model.

Using the second machine learning model, the robot may generate a second privacy embedding (1406) for each of the one or more first privacy embeddings in the cache of the robot. In some implementations, the robot can generate a second privacy inlay from stored sensor data used to generate one or more first privacy inlays. For example, the robot may generate a second privacy embedding by inputting raw or processed sensor data to a second machine learning model for generating the first privacy embedding. In some cases, the robot may generate a second privacy inlay for the object by inputting, to the second machine learning model, feature values derived from sensor data used to generate the first privacy inlay for the object.

After generating the second privacy embedding, the robot may add the second privacy embedding to the cache of the robot (1408). In some examples, the robot may also remove one or more first privacy embeddings from the cache of the robot (1410).

In some implementations, the first embedment in the local cache may include one or more common embedments. After receiving the second machine learning model, the robot may remove one or more common embeddings from the cache of the robot based on determining that the embeddings are common embeddings.

In some implementations, the first embedding in the local cache may include one or more received embeddings received by the robot from the server system over the communication network. For example, the received embeddings may be generated by different robots in the cluster of robots and shared by the different robots with the server system. The robot may determine that the particular embedding of the first embedding is a received embedding and, after receiving the second machine learning model, it may remove the particular embedding from the cache based on determining that the particular embedding is one of the received embeddings.

The robot may use various techniques to determine whether to designate information about the observed object as public or private. For example, when a robot obtains sensor data describing an object, the robot may generate an embedding for the object based on the sensor data. The robot determines whether to designate an embedding as private or public, and then stores the embedding and data designating whether the embedding is private or public.

To determine whether information about the observed object should be public (e.g., shared with a server system and other robots) or private, the robots may use sharing rules that indicate one or more restrictions on sharing of information obtained by the robots. These limitations may limit the sharing of information observed by the robot at a certain time or place, or may limit information describing objects having a certain object type or object characteristics. Similarly, these restrictions may limit the sharing of information to which certain tags are applied, e.g., personal designations such as a user designating an item as a personal "favorite" item of someone. In some instances, the user may provide specific instructions indicating that information about a certain object should be shared or not. In general, the restrictions and sharing rules may be set by the user, or may be set by the bot or a server system that facilitates sharing.

For example, objects observed in some locations (e.g., outdoors and in public locations) may be set to be public, while objects observed in private locations (e.g., private homes or locations designated as private by a user) may be kept private. When the robot observes a certain object, the robot can determine the position of the object or the robot. The robot may then determine whether the location is within a predetermined area for which no information is shared, such as a room of a residence for which the user may be tagged as private, or a public place where observations may be shared. Based on determining that the location is within the predetermined area, the robot designates the embedding as public or private accordingly.

As another example, the robot may determine an object type for an object observed by the robot, determine that the object type is one of a predetermined set of object types that have been designated as private or public states, and the robot may determine whether to designate the embedding as private embedding based on the object type determination.

In general, the robot may access the sharing rules and then determine that the sharing rules restrict sharing of information of the object described by the sharing data based on sensor data of the object or a result of processing the sensor data. Based on determining that the sharing rules limit sharing of information of the object, the robot designates embedding of the object as private embedding. The user input may indicate one or more criteria including a time, a place, an object type, or an object characteristic for which sharing of information is limited. In response to this input, the robot may store sharing rules that designate embedding of objects that meet the one or more criteria as private embedding. Determining whether to designate the embedding as private or public may include determining whether the embedding meets one or more criteria.

The embodiments of the invention and all of the functional operations described in this specification may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the invention may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions, encoded on a computer-readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium may be a non-transitory computer readable storage medium, a machine readable storage device, a machine readable storage substrate, a memory device, a composition of matter embodying a machine readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. In addition to hardware, the apparatus may include code that creates an execution environment for the computer program of interest, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. The propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not have to have such a device. In addition, the computer may be embedded in another device, such as a tablet computer, mobile phone, personal digital assistant (personal digital assistant, PDA), mobile audio player, global positioning system (Global Positioning System, GPS) receiver, to name a few. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disk; CD ROM and DVD-ROM discs. The processor and the memory may be supplemented by, or incorporated in, special purpose logic circuitry.

To support interaction with a user, embodiments of the invention can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the user can provide input to the computer. Other kinds of devices may also be used to support interactions with a user; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, speech, or tactile input.

Embodiments of the invention may be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the invention), or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (local area network, "LANs") and wide area networks (wide area network, "WANs"), such as the internet.

The computing system may include clients and servers. The client and server may generally be remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Although a few implementations are described in detail above, other modifications are possible. For example, while the client application is described as accessing the proxy(s), in other implementations, the proxy(s) may be used by other applications implemented by the processor(s), such as applications executing on the server(s). Moreover, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Further, other actions may be provided to, or may be eliminated, from, the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Additionally, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Additionally, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As one example, the processes depicted in the accompanying drawings are not necessarily required to achieve the desired result, but are shown in the particular order or sequential order. In some implementations, multitasking and parallel processing may be advantageous.

Claims

1. A computer-implemented method, comprising:

Obtaining, by a robot, sensor data indicative of a characteristic of an object;

determining, by the robot, a classification for the object;

generating, by the robot, a first embedding for the object using a machine learning model stored by the robot, the machine learning model encoding input information to generate the first embedding or to generate an output that is processed to generate the first embedding, wherein the input information is or is generated from the sensor data;

storing, by the robot, the generated first embedment and data indicative of a classification of the object in a local cache of the robot;

transmitting, by the robot over a communication network, the generated first embedded and data indicative of the classification to a server system;

receiving, by the robot from the server system over the communication network, a second embedding and a corresponding classification, wherein the received second embedding is generated by a second robot and corresponds to a second object, wherein the first embedding is an encoded representation of a characteristic of the object and the second embedding is an encoded representation of the characteristic of the second object; and is also provided with

The received second embedment and the corresponding classification are stored by the robot in a local cache of the robot.

2. The computer-implemented method of claim 1, further comprising:

one or more objects in the robot's environment are recognized by the robot using the first or second embeddings in the local cache.

3. The computer-implemented method of claim 2, wherein using, by the robot, the first embedding or the second embedding in the local cache to recognize one or more objects in the environment of the robot comprises:

generating, by the robot, sensor data describing a particular object in an environment of the robot; and is also provided with

Generating, by the robot, an inlay for the particular object using the machine learning model and sensor data describing the particular object;

comparing, by the robot, the embedding of the particular object with the first embedding and the second embedding; and is also provided with

A classification is assigned to the particular object by the robot based on a comparison of the embedding of the particular object with the first embedding and the second embedding.

4. The computer-implemented method of any preceding claim, wherein the machine learning model is a first machine learning model, the method further comprising:

Receiving, by the robot, a second machine learning model from the server system over the communication network;

using the second machine learning model by the robot to recognize one or more objects; and is also provided with

And removing the embedded generated by the first machine learning model from the local cache of the robot.

5. A computer-implemented method as in any of claims 1-3, wherein sending the generated embeddings comprises specifying the generated embeddings to share with other robots.

6. The computer-implemented method of claim 4, wherein specifying the generated embeddings to share with other robots comprises: the generated embeddings are sent with tags or metadata specifying that the embeddings should be shared or sent using an application programming interface for sharing the embeddings.

7. A computer-implemented method as any one of claims 1-3 recites, further comprising storing, by the robot, sensor data for generating input to the embedded machine learning model.

8. The computer-implemented method of any of claims 1-3, wherein generating the first embedding for the object using a machine learning model stored by the robot comprises:

The first embedding is generated based on processing feature data extracted from sensor data of the object using the machine learning model.

9. The computer-implemented method of claim 8, wherein the machine learning model is a neural network model, and wherein the first embedding is derived from an output at an output layer of the neural network model.

10. The computer-implemented method of claim 8, wherein the machine learning model is a neural network model, and wherein the first embedding is derived from data indicative of activation at a hidden layer of the neural network model.

11. A robot, comprising:

one or more processors; and

one or more machine-readable media storing instructions that, when executed by the one or more processors, cause the robot to perform operations comprising:

obtaining, by the robot, sensor data indicative of a characteristic of an object;

determining, by the robot, a classification for the object;

12. The robot of claim 11, further comprising:

13. The robot of claim 12, wherein using the first or second embeddings in the local cache by the robot to recognize one or more objects in the environment of the robot comprises:

14. The robot of any of claims 11-13, wherein the machine learning model is a first machine learning model, further comprising:

15. The robot of any of claims 11-13, wherein transmitting the generated embeddings includes designating the generated embeddings for sharing with other robots.

16. One or more non-transitory machine-readable media storing instructions that, when executed by one or more processing devices, cause a robot to perform operations comprising:

determining, by the robot, a classification for the object;

17. The one or more non-transitory machine-readable media of claim 16, further comprising:

18. The one or more non-transitory machine readable media of claim 17, wherein using the first or second embeddings in the local cache by the robot to recognize one or more objects in the environment of the robot comprises:

19. The one or more non-transitory machine readable media of any of claims 16-18, wherein the machine learning model is a first machine learning model, further comprising:

20. The one or more non-transitory machine readable media of any of claims 16-18, wherein sending the generated embeddings comprises specifying the generated embeddings to share with other robots.