CN112936304B - Self-evolution type service robot system and learning method thereof - Google Patents
Self-evolution type service robot system and learning method thereof Download PDFInfo
- Publication number
- CN112936304B CN112936304B CN202110144204.2A CN202110144204A CN112936304B CN 112936304 B CN112936304 B CN 112936304B CN 202110144204 A CN202110144204 A CN 202110144204A CN 112936304 B CN112936304 B CN 112936304B
- Authority
- CN
- China
- Prior art keywords
- service robot
- data
- user
- new
- learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/008—Manipulators for service tasks
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
Abstract
The invention discloses a self-evolution type service robot system and a learning method thereof, belonging to the technical field of intelligent robots. When a service robot equipped with an initial local model trained according to a training data set predefined by a manufacturer of the service robot needs to learn some new categories appearing in a user home environment after entering the home environment of a specific user, the service robot can request the user to give manual labels for the new category data, and gradually improve the initial local model with defects by continuous learning by using the new category data with the manual labels. Referring to the learning habit that when a human learns new knowledge, the human usually learns related knowledge of all new courses in the same time period, the optimal learning method for the robot to continuously learn is to learn a small part of data corresponding to all new classes in a certain time period. The invention has the characteristics of being close to the practical application scene of the service robot and balancing the timeliness and the effect of learning.
Description
Technical Field
The invention relates to a self-evolution type service robot system and a learning method thereof, belonging to the technical field of intelligent robots.
Background
The elderly population in the world today is steadily increasing, and it is expected that by 2050, one in six people will be over the age of 65 throughout the world. Meanwhile, the aging degree of the population of China is becoming more and more obvious since the 90 s of the last century. Therefore, the aging will become a problem to be solved urgently in the near future facing the world society. Most elderly people prefer home care for their elderly due to factors such as better privacy protection and autonomy. Due to physical and cognitive function degradation caused by age increase, an old user urgently needs help of a service robot to improve life quality of the old user, and one of key premises for the service robot to realize user assistance is to have reliable and robust perception intelligence so as to correspond to an unstructured and nondeterministic user home environment. However, currently existing service robots do not generally have this high level of perceptual intelligence.
Generally, a manufacturer of a service robot trains a deep neural network model according to a predefined training data set, and uses the deep neural network model as an initial local model of the service robot to provide a basic perception intelligence to the robot. When the service robot enters the home environment of a specific user, it may encounter some new classes that do not appear in the predefined training data set of the manufacturer, and the initial local model equipped with the service robot cannot give a high-confidence prediction result for these new classes, and further adjustment needs to be performed on the local model with defects in the robot itself.
The continuous learning is a centralized learning method which is different from the traditional method of centralizing all training data required by learning, and is more suitable for application scenarios with limited storage space and needing to continuously acquire training data required by learning, such as learning performed on a service robot. The knowledge distillation type continuous learning becomes one of the most concerned continuous learning types due to the characteristics of easy realization, excellent performance and the like. When the deep neural network is trained according to knowledge distillation continuous learning, the training data used in the training process arrives in a continuous form, so that the performance of the continuous learning is intuitively influenced by the arrival mode of the training data, the arrival batch number and other factors, and the incidence relation between the performance influencing the continuous learning and the influencing factors thereof needs to be deeply analyzed.
The invention provides that the relevant knowledge of the new category required by the robot when adjusting the local model of the robot to adapt to the home environment of a specific user can be given by the user, the robot sends the new category data collected in the home environment of the user to the user through the user terminal, and requests the user to give the label information of the data. After obtaining the new category data with manual labeling, updating the local model of the user according to a continuous learning method. Meanwhile, by referring to the learning habit that when a human learns a new course, the human usually learns all the new courses within a certain time period instead of only learning a certain new course, the optimal method for the robot to collect new category data to be learned (i.e. the optimal learning method for the robot to continuously learn) is designed, and the perception intelligence of the service robot is gradually improved under the assistance of a user.
Disclosure of Invention
The invention provides a self-evolution type service robot system and a learning method thereof in order to overcome the defects in the prior art, which can provide the robot with the related knowledge of a new category in a user home environment by the aid of the assistance of a user, and realize that the service robot can self-adaptively adjust a starting local model with defects after entering the home environment of a specific user.
In order to achieve the above purpose, the solution of the invention is:
one of the objects of the present invention is to provide a self-evolving service robot system, comprising:
a self-evolving service robotic system comprising:
the service robot body is used for providing intelligent service for the user according to the user instruction; a local database and a local model are deployed in the service robot body;
a cloud comprising a cloud database and a global neural network model; the cloud can be communicated with all the service robot bodies;
and the user terminal is used for interaction between the user and the service robot.
Another objective of the present invention is to provide a self-evolving service robot learning method based on the above system, which includes the following steps:
1) in a manufacturer stage, a pre-defined training data set is set in a cloud database, a global neural network model is trained by using the pre-training data set, and the trained global neural network model is deployed into all service robot bodies and serves as an initial local model of each service robot;
2) when the service robot enters the home environment of a specific user, the service robot continuously moves in the home environment of the user, collects new category data, stores the new category data into a local database, and marks the collected new category data according to the instruction of the user;
3) the service robot continuously updates the local model according to a continuous learning method by using the obtained new category data with manual labeling;
4) the service robot provides intelligent services to the user by using the latest updated local model.
Further, the continuous learning method specifically comprises: collecting a small part of data corresponding to all the new categories in the same period of time, labeling, taking a labeling result as a label, and training and updating the local model; in the next training, the trained data is not adopted any more, and a new data set collected in a new time period is used as a data set for the next training;
the data volume of the small part of data is obtained by dividing the preset training times on the basis of the total number of the preset training samples, wherein the preset training times are 5-20 times.
Furthermore, the new category data collected in the same period of time are all learned through a single step, and training batches are not set any more.
Furthermore, a gradient descent algorithm is adopted for updating when the model parameters are updated.
Further, manufacturers need to maintain and update the pre-training data set and the global neural network model in the cloud database.
Further, the method for determining the new category data in step 2) includes: and judging the data acquired by the service robot by using the local model, and if the confidence coefficient of the output prediction result of the local model is lower than a threshold value, storing the data as new type data in a local database.
Furthermore, after the new category data is updated in the local database, the new category data is presented to the user through the user terminal, and the user is reminded to carry out manual labeling.
Compared with the prior art, the invention has the advantages that:
1) the invention provides a self-evolution service robot architecture, which can provide the robot with related knowledge of a new category in a user home environment by means of the assistance of a user, and realizes that the service robot self-adaptively adjusts a starting local model with defects after entering the home environment of a specific user.
2) The invention provides an optimal learning method for knowledge distillation continuous learning, which is characterized in that the training of a deep neural network is similar to multi-step approximation of real posterior probability by Bayes' theorem and specific assumptions in the forms of prior probability and posterior probability, so that in the knowledge distillation continuous learning, the learning step number is set to be a smaller value or a random data arrival mode, and better performance can be shown compared with the learning step number which is set to be a larger value or a data arrival mode in a set sequence. Based on the method, the optimal learning method of the service robot is as follows: the service robot collects a small part of data corresponding to all the new categories in the same time period, and after the manual labeling of the small part of the new category data is obtained, the local model of the service robot is updated by using the new data with the manual labeling. By using the learning habit of the human in learning the new course, the learning effect of continuous learning is ensured, and the requirement of the service robot on the learning timeliness under the actual application scene is met
3) The continuous learning method provided by the invention can learn the related knowledge of a new category without the help of historical data, and meanwhile, the forgetting of the learned related knowledge of the category is reduced, and the characteristic that the storage space of the robot is limited is met.
Drawings
FIG. 1: performance comparisons between different learning step count settings;
FIG. 2 is a schematic diagram: comparing the performance of different data arrival modes;
FIG. 3: performance comparisons between different learning step count settings.
Detailed Description
The invention is described in further detail below with reference to the accompanying drawings:
as shown in fig. 1, the self-evolution service robot system provided by the present invention mainly includes:
the service robot body is used for providing intelligent service for the user according to the user instruction; a local database and an initial local model trained according to a training data set predefined by a manufacturer are deployed in the service robot body;
a cloud comprising a cloud database and a global neural network model; the cloud can be communicated with all the service robot bodies;
and the user terminal is used for interaction between the user and the service robot.
The idea of the invention is as follows: a manufacturer of the service robot trains a deep neural network model (namely a cloud model maintained by the manufacturer of the robot) according to a predefined training data set, and uses the deep neural network model as an initial local model of all service robots under the deep neural network model to endow the robot with basic perception intelligence; after the service robot enters the home environment of a specific user, the initial local model provided by the service robot is concentrated on training data which are not predefined by a manufacturer, and the newly-appeared category in the home environment of the user cannot provide a prediction result with high confidence, so that the local model with defects needs to be adjusted; the service robot can continuously move in the home environment of the user, meanwhile, the collected new category data is presented to the user through the user terminal, and the user is requested to give manual labeling information of the new category data to the robot; the robot utilizes the obtained new category data with manual labeling and continuously learns the related knowledge of the new category and updates the local model of the robot according to a continuous learning method under the condition of only using the currently collected new category data and not using the learned historical data, and meanwhile, forgetting of the learned related knowledge of the category is reduced as much as possible, so that the robot can be better adapted to the home environment of a user.
According to the above thought, the following steps can be obtained:
1) in a manufacturer stage, a pre-defined training data set is set in a cloud database, a global neural network model is trained by using the pre-training data set, and the trained global neural network model is deployed into all service robot bodies and serves as an initial local model of each service robot;
in this embodiment, the manufacturer needs to maintain and update the pre-training data set and the global neural network model in the cloud database.
2) When the service robot enters the home environment of a specific user, the service robot continuously moves in the home environment of the user, collects new category data, stores the new category data in a local database, and marks the collected new category data according to the instruction of the user.
3) And the service robot continuously updates the local model according to a continuous learning method by using the obtained new category data with manual labeling.
4) The service robot provides intelligent services to the user by using the latest updated local model.
Considering the practical application scene of the service robot, the optimization of the initial local model of the service robot by a continuous learning method is designed, and compared with the learning habit that when a human learns a new course, the method for collecting data required by continuous learning by the service robot (i.e. the learning method when the robot learns continuously) is set to collect and label a small part of data corresponding to all new classes in the same period of time by generally learning all the new courses in a certain period of time instead of learning only one new course so as to achieve a better learning effect on all the new courses, and the labeled result is used as a label to train and update the local model; in the next training, the trained data is not adopted any more, and a new data set collected in a new time period is used as a data set for the next training; the data volume of the small part of data is obtained by dividing the preset training times on the basis of the preset total number of training samples, wherein the preset training times are 5-20 times, and preferably 10 times; aiming at the new category data collected in the same period of time, the single step learning is completed, and no training batch is set; and updating the model parameters by adopting a gradient descent algorithm.
The significance of this learning method is that, in view of the timeliness of the service quality of the robot, the robot is better suited to implement in a progressive manner rather than in a step-by-step manner when learning new classes. On the one hand, a large amount of corresponding labeled data is generally needed for completing the learning of the new category, and under the self-evolution service robot architecture, the labeling information of the new category data is provided by the user. In consideration of the manual labeling burden of the user, the amount of manual data labeling tasks that the robot can bear in a certain time period has a certain upper limit, that is, the amount of new type data with manual labeling that the robot can obtain in a certain time period is limited. On the other hand, compared with the method that the local model is updated after a large number of new category data with manual labels are collected, the method that the local model is updated after a part of new category data with manual labels are acquired is more suitable for the service robot in the practical application scene, because the latter can enable the robot to obtain an adjusted local model earlier.
In this embodiment, the method for determining whether the data is the new category data includes: and judging the data acquired by the service robot by using the local model, and if the confidence coefficient of the output prediction result of the local model is lower than a threshold value, storing the data as new type data in a local database. And after the new category data is updated in the local database, presenting the new category data to the user through the user terminal to remind the user of manual labeling.
Examples
In this embodiment, the best learning method proposed by the present invention is compared with two other learning methods: one is centralized learning, in which the robot updates its local model only after a large amount of data of all new classes is collected, this learning method is called comparison method 1, and its pseudo code is shown in table 2;
the other is that the robot collects a small part of data corresponding to a part of new categories in a certain period of time, and uses the data to update the local model of the robot, and the difference between the learning method and the best learning method provided by the invention is that the former collects only a part of data of the new categories in a certain period of time, and the latter collects all data of the new categories. The former learning method is referred to as comparison method 2, and its pseudo code is consistent with the present invention, as shown in table 1.
A small batch gradient descent algorithm is used as a basic method for updating the parameters of the local model of the robot. k represents the size of a single Batch in the mini-Batch gradient descent algorithm. N and N ep Respectively representing the total number of all new category data used for training and the total number of training rounds. The learning of the new category data collected by the robot in a certain time period can be regarded as single-step learning, and the new category data learned by the robot in the certain time period can also be regarded as a sub data set under the whole new category data sets.
Represents the number of learning steps, and T is the total number of learning steps and the total number of sub data sets under the learning method. iter represents the number of training rounds in a certain step of learning. S represents the number of training data used in the single-step learning, i.e., the total number of data in a single subdata set. Eta is the learning rate. w represents the parameters of the local model of the robot, and g (w) represents the gradient values of the loss function with respect to the parameters of the local model of the robot.
Table 1 pseudo code of learning method and comparison method 2 of the present invention
Table 2 pseudo code of comparative method 1
From the perspective of learning timeliness, the comparison method 1 requires the robot to update its local model after collecting a large amount of new category data, while the comparison method 2 and the proposed best learning method only require the robot to update its local model after collecting a small amount of data, so the comparison method 1 is relatively poor in timeliness.
From the aspect of learning effect, the comparison method 1 can utilize all the collected new category data in the learning process, while the comparison method 2 and the proposed optimal learning method can only utilize the new category data collected in the current time period, which means that when a small part of data is learned, the knowledge corresponding to the previously learned new category data is forgotten to some extent, so that the comparison method 1 is better than the comparison method 2 and the proposed optimal learning method in the aspect of learning effect. Considering that the actual application scene of the service robot needs to require better timeliness and learning effect at the same time, the proposed optimal learning method is more balanced in timeliness and learning effect than the other two comparison methods, and therefore is more suitable for realizing the self-evolution service robot architecture.
A comparison of the learning effect between the proposed best learning method and the two comparison methods can be illustrated by fig. 2. The best learning method and the comparison method 2 in the example each include 60 steps of learning. The learning effect of the optimal learning method is better than that of the comparison method 2 while keeping the degree of forgetting for the learned category the same. After learning of all the new categories is completed, the classification accuracy of the model for the new categories is obviously improved, and the improvement of perception intelligence is realized.
The learning step number is also explored, the influence of the learning step number on the learning effect of knowledge distillation type continuous learning can be shown by fig. 3, in the example, the training data amount used by the continuous learning under all the settings is the same, and the X symbol represents the training result obtained according to the centralized learning. Under the condition that the accuracy of the network model on the initial category is the same after all training data are learned (namely under the condition that the forgetting degree of the model on the learned knowledge is consistent, the end points of the curves in the corresponding left sub-graph are consistent), the continuous learning performance under different settings can be compared by directly observing the accuracy of the network model on the new category after the continuous learning is completed (comparing the end points of the curves in the right sub-graph). As can be seen from the figure, in the knowledge distillation type continuous learning, setting the number of learning steps to a smaller value (i.e., 10-step learning) shows better performance than setting the number of learning steps to a larger value (i.e., 100-step learning). Therefore, in order to achieve a better learning effect, it is preferable that the number of learning steps is 5 to 20.
The above description is only a preferred embodiment of the present invention, and it should be noted that several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (5)
1. A service robot self-evolution method facing the elderly users comprises the following steps:
the service robot body is used for providing intelligent service for the user according to the user instruction; a local database and a local model are deployed in the service robot body;
a cloud comprising a cloud database and a global neural network model; the cloud can be communicated with all the service robot bodies;
a user terminal for interaction between a user and the service robot;
the self-evolution method is characterized by comprising the following steps:
1) in a manufacturer stage, a pre-defined training data set is set in a cloud database, a global neural network model is trained by using the pre-training data set, and the trained global neural network model is deployed into all service robot bodies and serves as an initial local model of each service robot;
2) when the service robot enters the home environment of a specific user, the service robot continuously moves in the home environment of the user, collects new category data, stores the new category data into a local database, and marks the collected new category data according to the instruction of the user; the method for judging the new category data in the step 2) comprises the following steps: judging data acquired by the service robot by using a local model, and if the confidence coefficient of an output prediction result of the local model is lower than a threshold value, storing the data as new category data into a local database;
3) the service robot continuously updates the local model according to a continuous learning method by using the obtained new category data with manual labeling; the continuous learning method specifically comprises the following steps: collecting a small part of data corresponding to all new categories in the same period of time, marking the small part of data, taking a marking result as a label, and training and updating the local model; in the next training, the trained data is not adopted any more, and a new data set collected in a new time period is used as a data set for the next training;
the data volume of the small part of data is obtained by dividing the preset training times on the basis of the total number of preset training samples, wherein the preset training times are 5-20 times;
4) the service robot provides intelligent services to the user by using the latest updated local model.
2. The self-evolution method of a service robot for elderly users as claimed in claim 1, wherein the new category data collected in the same period of time is learned by one step, and no training batch is set.
3. The elderly-user-oriented service robot self-evolution method according to claim 2, wherein a gradient descent algorithm is adopted for updating the model parameters.
4. The self-evolution method of a service robot for elderly users as claimed in claim 1, wherein a manufacturer needs to maintain and update the pre-training data set and the global neural network model in the cloud database.
5. The elderly user oriented services robot self-evolution method of claim 1,
and after the new category data is updated in the local database, presenting the new category data to the user through the user terminal to remind the user of manual labeling.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110144204.2A CN112936304B (en) | 2021-02-02 | 2021-02-02 | Self-evolution type service robot system and learning method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110144204.2A CN112936304B (en) | 2021-02-02 | 2021-02-02 | Self-evolution type service robot system and learning method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112936304A CN112936304A (en) | 2021-06-11 |
| CN112936304B true CN112936304B (en) | 2022-09-16 |
Family
ID=76241687
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110144204.2A Active CN112936304B (en) | 2021-02-02 | 2021-02-02 | Self-evolution type service robot system and learning method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112936304B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116415662B (en) * | 2023-06-12 | 2023-08-11 | 四川云申至诚科技有限公司 | Factory expert system based on knowledge discovery |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108764480B (en) * | 2016-08-23 | 2020-07-07 | 上海智臻智能网络科技股份有限公司 | Information processing system |
| CN110738304A (en) * | 2018-07-18 | 2020-01-31 | 科沃斯机器人股份有限公司 | Machine model updating method, device and storage medium |
| CN111444848A (en) * | 2020-03-27 | 2020-07-24 | 广州英码信息科技有限公司 | Specific scene model upgrading method and system based on federal learning |
| CN111814985B (en) * | 2020-06-30 | 2023-08-29 | 平安科技（深圳）有限公司 | Model training method under federal learning network and related equipment thereof |
| CN111831675A (en) * | 2020-07-07 | 2020-10-27 | 平安科技（深圳）有限公司 | Storage model training method and device, computer equipment and storage medium |
-
2021
- 2021-02-02 CN CN202110144204.2A patent/CN112936304B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112936304A (en) | 2021-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10794609B2 (en) | Methods and systems for personalized heating, ventilation, and air conditioning | |
| US11408629B2 (en) | Artificial intelligence device | |
| US11562297B2 (en) | Automated input-data monitoring to dynamically adapt machine-learning techniques | |
| WO2017008321A1 (en) | Smart home energy management method based on smart wearable device behavior detection | |
| CN107516127B (en) | Method and system for service robot to autonomously acquire attribution semantics of human-worn carried articles | |
| CN112936304B (en) | Self-evolution type service robot system and learning method thereof | |
| WO2019071909A1 (en) | Automatic driving system and method based on relative-entropy deep inverse reinforcement learning | |
| CN114500360B (en) | Network traffic scheduling method and system based on deep reinforcement learning | |
| JP6771105B2 (en) | Feedback-based self-adaptation subject objective weights context awareness system and how it works. | |
| CN107707657A (en) | Safety custody system based on multisensor | |
| CN109858553B (en) | Method, device and storage medium for updating driving state monitoring model | |
| WO2018120962A1 (en) | Reliability management-based uncertainty elimination context awareness system and working method thereof | |
| US20230116417A1 (en) | Systems and methods for domain-aware classification of unlabeled data | |
| CN110155073B (en) | Driving behavior pattern recognition method and system based on preference of driver | |
| KR20190116188A (en) | Apparatus and control method for recommending do-not-disturb mode based on context-awareness | |
| CN112465160A (en) | VR-based vehicle maintenance auxiliary system | |
| CN117094446A (en) | Deep learning-based milk yield prediction method for dairy cows | |
| CN114781439A (en) | Model acquisition system, gesture recognition method, device, equipment and storage medium | |
| CN108009636B (en) | Deep learning neural network evolution method, device, medium and computer equipment | |
| CN107688828A (en) | A kind of bus degree of crowding estimating and measuring method based on mobile phone sensor | |
| CN108826595A (en) | A kind of control method and system of air purifier | |
| KR20190119549A (en) | Apparatus and method for suggesting action item based on speech | |
| JP2021047751A (en) | Active learning device, active learning method, and program | |
| CN106030442B (en) | Interactive equipment selection method and device | |
| CN113360772A (en) | Interpretable recommendation model training method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |