JP7358808B2 - Robots and machine learning methods - Google Patents

Description

The present invention relates to robots and machine learning methods.

As part of work automation, robots that learn using machine learning technology and perform autonomous control are known.

One known robot that performs autonomous control by performing such machine learning is a robot that performs automatic operation by autonomous control after learning a model operation by a human operator (see, for example, Patent Document 1) ).
In order for such a robot to perform imitation learning to correctly imitate a model operation by a human operator, it is necessary to know the operation performed by the operator, the observation information known to the operator that is the basis for that operation, It is important to have the robot understand correctly as a set of learning data.
However, when performing imitation learning, for example, it is difficult to make the robot understand what the operator is looking at, so there has been a problem in that it is difficult to input such observation information as learning data.

The present invention has been made in view of the above-mentioned problems, and aims to provide a robot that performs imitation learning based on observation information grasped by an operator during a model operation.

In order to solve the above-mentioned problems, the robot of the present invention includes observation means that can observe a plurality of situations around the robot when learning, and a robot that selects the operation from among the plurality of observation results observed by the observation means. selection means for selecting one of the observation results by a switching operation of the observation means performed by a person after confirming the observation results, and in the learning , when the switching operation is performed, the selection means The observation result selected by the selection means and the movement operation of the robot by the operator input after the observation result is selected by the selection means are linked and learned.

According to the present invention, imitation learning can be performed efficiently, and the accuracy of automatic operation during autonomous control is improved.

1 is a diagram showing an example of the overall configuration of a robot control system according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of the configuration of an operating section shown in FIG. 1. FIG. 2 is a block diagram showing an example of the configuration of the control system shown in FIG. 1. FIG. FIG. 6 is a diagram showing a comparative example when performing imitation learning. It is a figure showing an example of operation of a learning control part in the present invention. It is a figure which shows an example of operation|movement of the selection means in this invention. It is a figure which shows an example of the control operation of the autonomous control part in this invention. It is a figure showing an example of composition of a 2nd embodiment in the present invention. It is a figure showing an example of operation of a selection means of a 2nd embodiment in the present invention. It is a figure showing an example of composition of a 3rd embodiment in the present invention.

As a first embodiment of the present invention, FIG. 1 shows a conceptual diagram of the overall configuration of a control system 100 for a robot 101 operated by an operator P using an operation unit 10 via a network 9.

The robot 101 has a plurality of cameras 20a and 20b which are moving image acquisition means provided as one of observation means, a movable arm 40 for picking up a designated object Q, and a movable arm 40. A driving section composed of a motor etc. as shown in FIG. The control unit 30 is a control terminal.
In this embodiment, the robot 101 will be described as a pick-up robot that picks up a designated object Q, but it learns model operations by the operator P by machine and performs autonomous operation based on the learning results. Any robot capable of performing any operation may be used.

In this embodiment, the camera 20a is attached to the arm 40, and the camera 20b is fixedly attached near the workbench and supported so that the arm 40 and the object Q can be photographed.
Further, the cameras 20a and 20b are of the same type, and the name camera 20 is used especially when the cameras 20a and 20b are not distinguished.
In particular, in this embodiment, only a camera as an imaging unit will be described as one of the observation means, but other than the camera, there are also other devices that can be used to observe sound, electromagnetic waves other than visible light, etc. It may also be a sensor or detection means that can display the information.
Furthermore, if there is a sensor that can supplement distance information such as an infrared sensor, radar, or lidar, or if data such as numerical information of the sensor can be displayed on the monitor 11, such sensors may be used instead of the camera 20, or You can also add it.

The operation unit 10 includes a movement instruction unit 12 that includes a plurality of buttons for instructing the movement direction of the arm 40, as shown in FIG. It has a moving image operation section 13 for.
The operation unit 10 communicates with the control unit 30 and the robot 101 via a wireless or wired network 9, and sends and receives data.
Specifically, a moving image taken by the robot 101 with the camera 20 is transmitted to the monitor 11, and an operation instruction given by the operator P using the movement instruction section 12 while looking at the monitor 11 is transmitted to the receiving section 22 of the robot 101. sent to.

The monitor 11 displays a moving image that is the observation result by the camera 20. In other words, the monitor 11 has a function as a display means for displaying one of the observation results selected by the video image operation section 13 from among the plurality of video images observed by the camera 20.
That is, in this embodiment, the monitor 11 has a function as a moving image display unit that displays images or moving images taken by the camera 20 that the operator P refers to for operation.
That is, the moving image displayed on the monitor 11 can be switched by the moving image operating section 13 of the operating section 10.

The movement instruction section 12 has a function as a movement direction instruction section including, for example, a forward instruction button 12a, a right movement instruction button 12b, a backward movement instruction button 12c, and a left movement instruction button 12d. In this embodiment, the movement instruction section 12 has four independent buttons, but it may be a joystick or other pointing device.
Further, in this embodiment, the receiving section 22 is shown separately from the control section 30, but the receiving section 22 may be provided as one function in the control section 30, but the configuration is not limited to this. isn't it.
Further, if the operation section 10 is a tablet terminal, the monitor 11 may be included in the operation section 10.

As shown in FIG. 3, the control unit 30 includes a machine learning control unit 31 for learning by combining a model operation performed by the operator P and a moving image displayed on the monitor 11 at the time of the model operation; It has an autonomous control section 32 for autonomously controlling the arm 40 based on the data set learned by the machine learning control section 31.
The machine learning control unit 31 receives a model operation by the operator P, that is, a movement operation command indicating which direction the model operation was made, forward, backward, left, or right, a monitor display switching command when the model operation is performed, and a command to switch the monitor display when the model operation is performed. The video frames at that time are saved as a set of data.
In addition, in this embodiment, the moving image frame is used as a data set to be stored in the machine learning control unit 31 as an observation result, but the overall state information of the robot 101, such as the identifiers of the selected cameras 20a and 20b, is also used as an observation result. It may be included as

Furthermore, the machine learning control unit 31 uses the data set of the moving image frame F as the accumulated observation result, the movement operation command M of the arm 40, and the monitor display switching command G, and inputs the moving image frame, Learn functions so that the output is a movement operation command and a monitor display switching command.
That is, the machine learning control unit 31 stores and accumulates "the moving image frame F which is the accumulated observation result" and "the movement operation command M and the monitor display switching command G which are the operated operation instructions" as a data set. Learn the functions.

Based on the control function learned by the machine learning control unit 31, the autonomous control unit 32 operates the robot 101 according to the output movement operation command when a video frame is input, and uses the output monitor display switching command. Accordingly, the operations of the cameras 20a and 20b are controlled to create a moving image frame to be input to the next control function.
Here, to show a specific example, it is assumed that the operator P looks at the frame F1 displayed on the monitor 11 in a model operation, sends the movement operation command M1, and then operates the monitor display switching command G1. Consider the case where the
When the frame F1 is input, the autonomous control unit 32 outputs a movement operation command M1, and when the arm 40 moves to a predetermined position by the movement operation command M1, it further outputs a monitor display switching command G1 to obtain the next result. Obtain frame F2.
Thereafter, using the obtained frame F2 as an input value, a movement operation command M2 and a monitor display switching command G2 are output from the data set of the machine learning control unit 31, and autonomous control is performed one after another.

Now, in conventional machine learning methods, various efforts have been made regarding what kind of input and output should be saved as a data set.
An example of a data set used for machine learning is a learning method that combines input video frames and movement operation commands as a data set.
In FIG. 4, as an example of such a data set, the field of view of the moving image frames F1' to F4' photographed by the camera 20 is shown by a solid line on the left side, and if there is an object Q outside the field of view of the camera 20, The object Q is indicated by a broken line. Further, in FIGS. 4A to 4C, movement operation commands M1' to M3' corresponding to the moving image frames F1' to F3' displayed on the left side are displayed on the right side.
In the following description, the left side of FIG. 4(a) will be referred to as shooting state A, the left side of FIG. 4(b) will be called shooting state B, and the left side of FIG. 4(c) will be called shooting state D. Express each.

A method of learning by linking the photographing state A and photographing state B shown in FIGS. 4A and 4B with the movement operation command of the arm 40 through imitation learning will be described.
First, as a model operation, the operator P executes a movement operation command M1' and a movement operation command M2 in the respective patterns of the moving image frame F1' and the moving image frame F2' as shown in FIGS. 4(a) and 4(b). ' are performed, and the machine learning control unit 31 is made to learn as a model operation.
According to such a learning method, when the autonomous control unit 32 detects the shooting state A as shown in FIG. By inputting ', the arm 40 can be appropriately guided to the position of the object Q.
Similarly, when the autonomous control unit 32 detects the shooting state B as the moving image frame F2' as shown in FIG. The arm 40 can be appropriately guided to the position of the object Q.

However, for example, as shown by the broken line in FIG. 4(c), when a shooting state C is detected in which the object Q exists outside the field of view of the camera, the object Q is detected from the moving image frame F3' in FIG. 4(c). It is difficult to read Object Q.
Even in the photographing state C, the operator P only has to directly view and operate the object Q indicated by the broken line, so that the arm 40 can be guided to the object Q on the right side out of the visual field of the camera. . That is, in the model operation, even in the photographing state C, the movement operation command M3' is output and stored as learning data.

In this way, we will explain the problem when the video frame F3', which does not actually affect the judgment of the operator P, is repeatedly associated with the movement operation command M3'. do.
For example, when a photographing state D as shown in FIG. As shown by the broken line in FIG. 4(d), the arm 40 is moved to the right by outputting the movement operation command M3' even though it is on the left outside of the camera's visual field.
That is, it is difficult to predict the optimal movement operation command from the observation results of the video frame F4' in the shooting state D.

Alternatively, if many learning patterns such as those shown in FIGS. 4(c) and 4(d) are included during the model operation, it will not be possible to distinguish between the shooting state C and the shooting state D, and the movement operation command will be Since it is not determined uniquely, learning may fail.

In order to solve such a problem, it is necessary to combine the observation information that the operator P uses as the basis for the operation, that is, the observation results that the operator P is looking at, and the observation results that the robot 101 can recognize (in the above comparative example, the observation results of the camera 20). It is good as long as the field of view) matches.
However, if the operator P simply looks at the monitor 11 during the model operation, it will be difficult to perform the model operation in FIGS. 4(c) and 4(d), and the pattern will be excluded as a learning pattern in the first place.

In order to solve this problem, in the present invention, the machine learning control unit 31 performs learning by linking the moving image frame F1 and the movement operation command M1 of the arm 40, as well as the monitor display switching command G1. conduct.
A learning method using the monitor display switching command G will be explained using the flow diagram of FIG. 5 and operation examples illustrated in FIGS. 6(a) to 6(c).
Note that the left side of FIGS. 6(a) to 6(c) shows an example of an image of the camera 20a, and the right side of FIGS. 6(a) to 6(c) shows an example of an image of the camera 20b. In addition, the transition of the screen displayed on the monitor 11 when the monitor display switching commands G1, G2, and G3 corresponding to the cameras 20a and 20b are pressed is shown in the center of FIGS. 6(a) to 6(c), respectively. Schematically indicated by arrows.
In the initial state in which the operator P has confirmed the shooting state C shown in FIG. The field of view is appropriately set (step S101).
Such monitor display switching command G1 may be, for example, to switch to either camera 20a or 20b, or to move either camera 20a or 20b to display each of the images shown in FIGS. 4(a) to 4(c). Of the photographing states, it may be moved from photographing state C to photographing state B.
Furthermore, the camera 20 may be moved by, for example, operations such as panning, tilting, and rolling, as well as operations such as changing the field of view of the camera 20.
In any case, the monitor display switching command G1 allows the position of the object Q to be placed within the moving image frame F2 (step S102).
Selection of such an image can be performed, for example, by switching from the image of the camera 20a to the image of the camera 20b using monitor display switching commands G1, G2, and G3 as shown in FIGS. 6(a) to (c). .
In the model operation of the operator P in this embodiment, as described above, when the photographing state C shown in FIG. 4(c), that is, the image shown on the left side of FIG. 6(a) is displayed, The operator P selects the moving image frame F1 taken by the camera 20b from the moving image frames F1 taken by the camera 20a and switches the display on the monitor 11 using the monitor display switching command G1, and then uses the movement operation command M1. Enter.
The machine learning control unit 31 links the monitor display switching command G1 executed when the video frame F1 was displayed with the movement operation command M1 after switching the display on the monitor 11 to a different video frame F1, and creates data used for learning. Save as a set (step S103).
In this way, since "video frame F1 that operator P wanted to switch" is guided by "monitor display switching command G1 performed when video frame F1", It functions as a "observation result".
Further, after inputting the movement operation command M1, the moving image frame photographed by the cameras 20a and 20b changes to a moving image frame F2 as shown in FIG. 6(b). Using the moving image frame F2 as a new observation result, the moving operation command M2 and the monitor display switching command G2 are also learned in a loop one after another until it is determined that the model operation has ended (step S104). For example, after the movement operation command M2, when the moving image frame F3 changes as shown in FIG. 6(c), a loop is performed using the moving image frame F3 as a new observation result. In this embodiment, the end determination in step S104 ends one learning pattern when the target object Q can be picked up.

Needless to say, the control function used by the autonomous control section 32 can be derived by storing and storing the learning results of a plurality of patterns in the machine learning control section 31 using these learning patterns as one set.
The control function obtained through this learning can take various forms, such as a convolutional neural network, a precombinant neural network, and a linear function, but any function used in so-called machine learning is sufficient and is limited to such configurations. isn't it.

As described above, the learning method performed by the machine learning control unit 31 in this embodiment includes the operation step S102 in which the operator P performs an operation, and at least one of the observation results obtained from a plurality of observation means in the operation step S102. The selection step S101 in which one is selected, the moving image frame F selected in the selection step S101, the movement operation command M operated by the operator in the operation step S102, and the monitor display switching command G are set as a set. and a learning step S103 of storing and accumulating as a data set.
With this configuration, machine learning based on model operations by the operator P can be efficiently performed.

The operation of the autonomous control unit 32 of the robot 101 after the machine learning is completed by repeatedly performing learning through model operations in this manner will be described with reference to FIG.
First, the autonomous control unit 32 checks the video frame F1 on the monitor 11 as an initial input value (step S201).
The autonomous control unit 32 outputs a monitor display switching command G1 and a movement operation command M1 as outputs for the input video frame F1 from the learning data set stored in the machine learning control unit 31 and the control function based thereon ( Step S202).
The autonomous control unit 32 checks whether all operations performed in the model operation using the monitor display switching command G1 and the movement operation command M1 have been completed (step S203).
The autonomous control unit 32 further performs loop processing to output a monitor display switching command G2 and a movement operation command M2 using the moving image frame F2 changed by the monitor display switching command G1 as a new input value to the control function. It is executed sequentially (step S204).
In this way, the robot 101 can autonomously complete the operation of picking up the object Q by imitating and learning the model operation of the operator P.

In this embodiment, the robot 101 is a robot that performs learning based on a model operation by the operator P and performs autonomous control based on the learning results, and when performing learning, the robot 101 is a robot that performs autonomous control based on the model operation performed by the operator P. It has cameras 20a and 20b as observation means capable of performing multiple observations, and a moving image operation unit 13 as selection means for selecting one of the plurality of observation results observed by the cameras 20a and 20b.
In addition, during learning of the robot 101, the machine learning control unit 31 learns by associating the movement operation command M of the operator P with the video frame F1 selected by the video image operation unit 13 when performing the operation. do.
According to this configuration, the operator P can select a suitable moving image frame F for performing the movement operation command M from among the plurality of moving image frames F using the moving image operation unit 13, so that imitation learning is possible. can be performed efficiently, improving the accuracy of automatic operation during autonomous control.

Further, in this embodiment, a plurality of cameras 20a and 20b are provided as observation means, and the moving image operation section 13 can select a moving image photographed by one of the cameras 20a and 20b as an observation result.
With this configuration, the operator P can select a suitable moving image frame F for performing the movement operation command M from among the plurality of moving image frames F using the moving image operation unit 13, so that imitation learning can be performed efficiently. This improves the accuracy of automatic operation during autonomous control.

As a second embodiment of the present invention, a case where a camera 21 that can be driven independently of the arm 40 is used will be described using FIG. 7.
Note that, in the second embodiment, the same configurations as those of the first embodiment already described are given the same numbers and the description thereof will be omitted.

The camera 21 has a camera arm 24 that can be freely moved such as panning, tilting, and rolling.
The moving image operation unit 13 can adjust the field of view of the camera 21 at any position by operating the camera arm 24 of the camera 21.
When the operator P initially obtains the moving image frame F1 as shown in FIG. is input as the monitor display switching command G1.
The machine learning control unit 31 sets the field of view of the camera 21 changed by the monitor display switching command G1 as a new video frame F2, and uses the monitor display switching command G2 and the movement operation command M2, which are the next operations by the operator P, as a new video frame F2. Store as a model operation as a set of data.
At this time, the moving image frame F2 is an image that the operator P wanted to see through the model operation, and is "an observation result arbitrarily selected by the moving image operating section 13." That is, "observation information known to the operator P" that is the basis for the movement operation command M2 of the operator P is the moving image frame F2.
The machine learning control unit 31 derives a control function from this set of data sets. The autonomous control unit 32 performs autonomous control based on the control function, and completes a series of operations of picking up the object Q so as to imitate the model operation performed by the operator P.

In the second embodiment, the robot 101 has at least one camera 21 that moves independently of the movement of the arm 40, and the video image operation unit 13 displays the video image at an arbitrary position taken by the camera 21 as an observation result. It can be selected as
In this way, even if there is only one camera 21 as the imaging unit, if it can be moved to any position, the same effect as "having multiple observation means" can be obtained, and such multiple selections of observation results can be obtained. One can be selected from among them using the moving image operation section 13.
With this configuration, the operator P can select a suitable moving image frame F for performing the movement operation command M from among the plurality of moving image frames F using the moving image operation unit 13, so that imitation learning can be performed efficiently. This improves the accuracy of automatic operation during autonomous control.

Furthermore, as a third embodiment, a configuration using the omnidirectional camera 23 attached to the tip of the arm 40 will be described.

The spherical camera 23 is attached to the tip of the arm 40 and has a function as an imaging unit capable of capturing spherical or hemispherical images.
The spherical camera 23 can also select any position of the captured spherical or semi-spherical moving image to be cropped or enlarged using the moving image operation section 13, and the monitor 11 displays the selected entire spherical moving image. Part or all of the celestial sphere image can be displayed.
Even when such a spherical camera 23 is used, the machine learning control unit 31, when the video frame F1 is input, uses the monitor display switching command G1 and the movement operation command M1 by the corresponding video operation unit 13. and stored as a set of data.
Hereinafter, the operation of the autonomous control unit 32 and the learning process will be omitted because the description overlaps with the first embodiment or the second embodiment.

Note that, for example, when using a spherical image from the spherical camera 23, the moving image operation section 13 and the monitor 11 may be integrated as a head-up display (HUD) 15, as shown in FIG.
In that case, for example, the operator P, while wearing the head-up display 15 and the pointing device 16, can perform a simple motion such as picking up the object Q located somewhere in the spherical image. The model operation can be completed immediately.

Furthermore, when the omnidirectional camera 23 and the head-up display 15 are used in this way, the function of the head-up display 15 is to change the position of the line of sight or field of view of the operator P to the entire sky captured by the omnidirectional camera 23. Since it always matches a portion of the spherical image, the selection means and the display means can be integrated, and a model operation can be performed with a simple configuration.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and unless specifically limited in the above explanation, the present invention described in the claims Various modifications and changes are possible within the scope of the spirit.

For example, in the above embodiment, the robot 101 is described as a robot that drives the arm 40 to pick up the object Q, but the present invention may also be applied to robots having other drive modes. good.

Further, in the above embodiment, only a camera was described as an observation means, but a robot that observes sound, electromagnetic waves other than visible light, etc., and performs control based on the observation results may also be used.
Further, the learning method used in the above embodiment may be executed by a program that executes each of the above-mentioned means in steps S101 to S104, or by various storage media in which the program is stored.

The effects described in the embodiments of the present invention are merely a list of the most preferred effects resulting from the present invention, and the effects of the present invention are limited to those described in the embodiments of the present invention. isn't it.

10 Operation unit 11 Video display unit (monitor)
12 Movement instruction section 13 Selection means (video image operation section)
20 Observation means (camera)
20a, 20b Observation means (camera)
21 Observation means (camera)
22 Receiving unit 23 Observation means (all-celestial camera)
24 Camera arm 30 Control unit (control terminal)
31 Learning control unit 32 Autonomous control unit 100 Control system 101 Robot F Example of observation results P Operator Q Target object

Patent No. 5361756

Claims (7)

A robot that learns based on operations by an operator and performs autonomous control based on the results of the learning,
observation means capable of observing multiple situations around the robot when performing the learning;
selection means for selecting one of the plurality of observation results observed by the observation means by a switching operation of the observation means performed by the operator after confirming the observation results ;
has
In the learning , the observation result selected by the selection means when performing the switching operation; the movement operation of the robot by the operator input after the observation result is selected by the selection means; A robot that is characterized by being linked and learned. The robot according to claim 1,
The observation means has a plurality of imaging units,
The robot is characterized in that the selection means is capable of selecting a moving image photographed by one of the imaging units as the observation result. The robot according to claim 1 or 2,
The observation means has at least one imaging unit that moves with the movement of the robot,
The robot is characterized in that the selection means is capable of selecting a moving image taken by the imaging unit at an arbitrary position as the observation result. The robot according to any one of claims 1 to 3,
an autonomous control unit that causes the robot to operate autonomously based on the learning result;
The autonomous control unit selects the observation result selected by the selection means when performing the switching operation, and the operation of the robot by the operator that is input after the observation result is selected by the selection means. A robot that performs machine learning based on a dataset consisting of a set of and . The robot according to any one of claims 1 to 4,
The robot is characterized in that the observation means includes an imaging unit capable of capturing a spherical image or a hemispherical image. The robot according to claim 5,
The robot is characterized in that the selection means is capable of selecting a predetermined range of the celestial sphere image or the half-celestial sphere image as the observation result. a selection step of selecting at least one of the observation results obtained from a plurality of observation means by a switching operation of the observation means performed by an operator after confirming the observation results; ,
an operation step of performing a movement operation of the robot by the operator input after the observation result is selected in the selection step;
a learning step of storing and accumulating the observation results selected in the selection step and the operation instructions operated by the operator in the operation step as a data set;
A machine learning method characterized by having the following.

Images