JP6897593B2 - Learning target device and operation method - Google Patents

Description

The present invention relates to a learning target device and an operation method.

Due to the sophistication of computer CPUs (Central Processing Units), storage devices, and networks, the amount of data that can be handled by a system is explosively increasing. Such a huge amount of data is called big data. Further, a large number of devices to be operated and a data source are connected to each other via a network, and various IoT (Internet of Things) systems have been developed as a mechanism for integrating these. By handling big data in the IoT system, various information processing can be performed. However, in order to carry out new information processing, new capabilities will be given to the application. In recent years, the amount of development has increased significantly, which has led to a shortage of application development resources. If new AI (Artificial Intelligence) technologies such as deep learning can be used to acquire new capabilities in applications, such a situation of lack of resources can be solved.

For some time, research on artificial intelligence, including neural networks, has been extensively conducted. For example, the technology for recognizing an object reflected in an image has undergone many improvements, and the recognition rate has been gradually improved. In particular, with the introduction of deep learning, the recognition rate of objects has increased rapidly in the last few years. Deep learning technology is used not only in image recognition but also in a wide range of fields such as voice recognition, sentence summarization, automatic translation, automatic driving, failure prediction, and sensor data analysis. By using machine learning such as deep learning, machines can acquire new abilities.

For example, as a technique related to a method of mounting a new capability on an apparatus, Patent Documents 1 and 2 propose a technique of rewriting the firmware of a printer. Further, Patent Documents 3 and 4 propose a technique related to machine learning, and Patent Document 5 proposes a character identification system using deep learning.

Japanese Unexamined Patent Publication No. 2009-134474 JP-A-2007-140952 Japanese Unexamined Patent Publication No. 2014-228792 Japanese Patent No. 5816771 Japanese Unexamined Patent Publication No. 2015-53008

The present inventors have found that the above-mentioned conventional AI technology has the following problems. That is, in the conventional AI technology, a device such as a robot to be added to a new ability is prepared at hand, and the target device is made to execute a machine learning process, so that the target device has a new ability. Was acquired. Therefore, when the device to be added to the new ability is located in a remote place, it is difficult to add the new ability to the device.

Therefore, in order to solve the above-mentioned problems, the present inventors have invented a system that adds a new ability to a device arranged in a remote place by executing a machine learning process by remote control. However, since this system remotely controls the learning target device, the present inventors have found that there is a problem that an unexpected accident or the like may occur around the learning target device.

The present invention, on the one hand, has been made in view of such circumstances, and an object of the present invention is to remotely control a learning target device located at a remote location in order to carry out machine learning. The purpose is to provide a mechanism for ensuring the safety around the device to be learned.

The present invention employs the following configuration in order to solve the above-mentioned problems.

That is, the learning target device according to one aspect of the present invention is arranged at a remote location with respect to a learning device that performs machine learning for acquiring a designated ability in response to a learning request from a client, and the learning device is concerned. It is a learning target device to be acquired by the machine learning of the above, and includes a display unit and a control unit, and the control unit executes an operation related to the designated ability. A reception step that receives a command for remote operation to be instructed from the learning device, an execution step that executes an operation related to the designated ability according to the received command for remote operation, and the operation according to the command for remote operation. During the execution, the display step of displaying on the display unit that the learning device is being operated by the remote operation is executed.

While the learning target device according to the configuration is operating in accordance with a remote control command from the learning device, the display unit displays that the learning device is being operated by remote control. In this configuration, through the display of this display unit, it is possible to notify the operator existing around the learning target device that the learning target device is remotely controlled, whereby the safety around the learning target device can be obtained. Can be secured. Therefore, according to the configuration, it is possible to provide a mechanism for ensuring the safety around the learning target device when the learning target device located at a remote location is remotely controlled in order to carry out machine learning. ..

The "learning target device" is not particularly limited as long as it is a device that can be controlled by a computer, and may be appropriately selected according to the embodiment. For example, the learning target device may be a robot system used in a production line, surgery, or the like. Further, the "ability" to be acquired may include any ability that can be implemented in the learning target device, for example, a function that can be provided by the learning target device, information processing that can be executed by the learning target device, and the like. In order to acquire the ability, the learning target device can perform new functions or information processing that have not been implemented, and the learning target device can perform the implemented functions or information processing more efficiently. Is included. Further, "located in a remote place" means that the learning device and the learning target device are physically separated from each other, for example, they are separated by a wall, or they are placed in separate buildings. Then, the person on the side of the learning device cannot see the person on the side of the learning target device, or the voice does not reach directly. Further, for example, the learning target device is installed in the factory of the learning requester, the learning device is installed in the company building of the person who undertakes the learning request, and each is arranged in a separate company building. Cases where it takes a relatively long time for a technician belonging to a company that operates a learning device to visit the place where the learning device is located, such as when the learning device and the learning device are located in different prefectures. The present invention is particularly effective. Further, the learning device is preferably configured to have a higher machine power than the learning target device. That is, the learning target device may be configured to have a lower machine power than the learning device. Machine power may be compared according to the processing speed of the CPU, the capacity of the memory, the read speed of the memory, and the like.

Further, the learning target device according to the one aspect may further include a communication unit configured to communicate with peripheral devices arranged around the device, and the control unit performs the operation according to the remote control command. During the execution, the peripheral device may be further configured to perform a notification step for notifying the communication unit that the learning device is being operated by the remote control. According to this configuration, it is possible to make the peripheral device recognize that the learning target device is operating by remote control. As a result, the peripheral device can be prevented from interfering with the operation of the learning target device by remote control for machine learning.

Further, in the learning target device according to the one aspect, the control unit notifies the peripheral device that the operation by the remote control is completed after the remote control from the learning device is completed. It may be configured to further perform the completion notification step notified by. According to this configuration, it is possible to make the peripheral device recognize that the operation by remote control has been completed in the learning target device.

Further, in the learning target device according to the one aspect, the control unit performs a completion display step of displaying on the display unit that the operation by the remote control is completed after the remote control from the learning device is completed. It may be configured to perform further. As a result, it is possible to notify the workers existing around the learning target device that the operation by remote control has been completed in the learning target device. In particular, it is possible to notify the operator existing around the learning target device that the operation of the learning device by remote control is completed and the learning target device does not suddenly start to move by remote control.

Further, in the learning target device according to the one aspect, the display unit may be a display, and the control unit causes the display to display the contents of the operation of the learning device by the remote control in the display step. You may. According to this configuration, it is possible to notify the operator existing around the learning target device of the content of the operation by remote control.

Further, in the learning target device according to the one aspect, the control unit may receive the command instructing the execution of a plurality of operations in the reception step, and in the display step, the operation being executed by the execution step. The content of the operation to be executed next may be displayed on the display. According to this configuration, by displaying the content of the operation to be executed next to the operation being executed on the display, the operator existing around the learning target device can be notified in advance of the operation of the learning target device. .. As a result, the safety around the learning target device can be enhanced.

Further, in the learning target device according to the one aspect, when the operation being executed is a dangerous operation, the control unit indicates to the display that the operation being executed is a dangerous operation in the display step. It may be displayed. According to this configuration, by clearly indicating the dangerous operation, it is possible to clearly inform the workers existing around the learning target device that the danger may be imminent, thereby ensuring the safety around the learning target device. Can be enhanced.

Further, in the learning target device according to the one aspect, the display unit may be an indicator light, and the display control unit causes the indicator light to emit light in the first display mode in the display step. The learning device may be displayed on the indicator lamp when it is being operated by the remote control, and in the completion display step, the indicator lamp is made to emit light in a second display mode different from the first display mode. Then, the indicator lamp may indicate that the operation by the remote control has been completed. According to this configuration, a mechanism for ensuring the safety around the learning target device can be realized at low cost.

The operation method according to one aspect of the present invention is arranged at a remote location from a learning device that performs machine learning for acquiring a designated ability in response to a learning request from a client, and the machine learning of the learning device. A learning target device that is a target for acquiring the designated ability, and the learning target device having a display unit issues a remote operation command instructing the execution of an operation related to the designated ability. While the reception step received from, the execution step of executing the operation related to the designated ability according to the received command of the remote operation, and the operation being executed according to the command of the remote operation, the learning device. The display step of displaying on the display unit that the operation is being performed by the remote operation is executed. According to this configuration, it is possible to provide a mechanism for ensuring the safety around the learning target device when the learning target device located at a remote location is remotely controlled in order to carry out machine learning.

According to the present invention, it is possible to provide a mechanism for ensuring the safety around the learning target device when the learning target device located at a remote location is remotely controlled in order to carry out machine learning.

FIG. 1 schematically illustrates an example of a situation in which the present invention is applied. FIG. 2 schematically illustrates an example of the hardware configuration of the learning device according to the embodiment. FIG. 3 schematically illustrates an example of the hardware configuration of the robot arm system according to the embodiment. FIG. 4 schematically illustrates an example of the operating state of the robot arm system according to the embodiment. FIG. 5 schematically illustrates an example of the software configuration of the learning device according to the embodiment. FIG. 6 schematically illustrates an example of the software configuration of the robot arm system according to the embodiment. FIG. 7 illustrates an example of the processing procedure of the learning device according to the embodiment. FIG. 8 illustrates an example of the processing procedure of the robot arm system according to the embodiment. FIG. 9 schematically illustrates an example of the configuration of the learning device according to the modified example. FIG. 10 illustrates an example of a processing procedure during machine learning of the learning device according to the modified example. FIG. 11 schematically illustrates an example of a robot arm system according to a modified example.

Hereinafter, embodiments according to one aspect of the present invention (hereinafter, also referred to as “the present embodiment”) will be described with reference to the drawings. However, the embodiments described below are merely examples of the present invention in all respects. Needless to say, various improvements and modifications can be made without departing from the scope of the present invention. That is, in carrying out the present invention, a specific configuration according to the embodiment may be appropriately adopted. For example, in the following, as an embodiment of a learning target device to be machine-learned, a robot arm system that performs a predetermined work in a factory will be illustrated. However, the application target of the present invention is not limited to the robot arm system, and may be appropriately selected depending on the embodiment. The data appearing in the present embodiment are described in natural language, but more specifically, the data is specified in a pseudo language, commands, parameters, machine language, etc. that can be recognized by a computer.

§1 Application example First, an example of a situation in which the present invention is applied will be described with reference to FIG. FIG. 1 schematically illustrates an example of an application scene of the learning device and the learning target device according to the present embodiment.

As shown in FIG. 1, the learning device 1 according to the present embodiment is machine learning for acquiring a new ability designated for a learning target device arranged in a remote location in response to a learning request from a client. It is an information processing device that performs. Specifically, the learning device 1 is arranged at a remote location and receives from the client a designation of a learning target device to be machine-learned and a designation of an ability to be acquired by the learning target device by machine learning as a learning request. The client uses the user terminal 4 to specify the learning target device and the ability to be acquired via the network 10.

In the present embodiment, it is assumed that the robot arm system 2 that performs a predetermined work in the factory is designated as a learning target device that is a target for acquiring the designated ability by machine learning of the learning device 1. The ability to be acquired may be appropriately selected from all the abilities that can be implemented in the robot arm system 2 according to the embodiment. For example, the ability to perform a new work and the work in operation can be performed more efficiently. It may be the ability to do.

By transmitting control data, the learning device 1 remotely controls the robot arm system 2 designated as the learning target device so as to execute an operation for learning related to the ability specified in the learning request. Next, the learning device 1 collects learning data for machine learning of the specified ability based on the result of the remote control of the robot arm system 2. Then, the learning device 1 performs machine learning of the learning device (neural network 6 described later) so as to acquire the designated ability by using the collected learning data. As a result, the learning device 1 can generate a learned learning device for causing the robot arm system 2 designated as the learning target device to carry out the ability specified in the learning request.

Further, in the present embodiment, the robot arm system 2 located at a remote location merely executes an operation related to the specified ability, and the machine learning process for acquiring the specified ability is a learning device. Let 1 do it. Therefore, even if the machine power of the robot arm system 2 located at a remote location is limited, it is possible to carry out the machine learning process of the ability to be acquired by the robot arm system 2.

Therefore, according to the present embodiment, a learning request is received from a general company (requester) who does not have a factor having advanced skills and a complicated system used for machine learning, and machine learning is performed according to the received learning request. It is possible to provide a technical mechanism for implementation. In particular, it is possible to provide a technical mechanism for appropriately adding a new capability to a device arranged in a factory, a warehouse, or the like in a remote place such as the robot arm system 2.

In addition, "arranged in a remote place" means that the learning device 1 and the learning target device are physically separated from each other. For example, they are separated by a wall or arranged in separate buildings. For example, the arrangement is such that the person on the side of the learning device cannot see the person on the side of the learning target device, or the voice does not reach directly. Further, for example, the learning target device is installed in the factory of the learning requester, the learning device 1 is installed in the company building of the person who contracted the learning request, and each is arranged in a separate company building. .. Therefore, the arrangement of the user terminal 4 used by the client may be appropriately selected according to the embodiment. For example, the user terminal 4 may be arranged in a local area network different from the learning device 1 and the robot arm system 2, and may be arranged so as to connect the learning device 1 and the robot arm system 2 via a network such as the Internet. Further, for example, the user terminal 4 may be arranged in the same local area network as the learning device 1, or may be arranged in the same local area network as the robot arm system 2. Further, the learning device 1 may receive a learning request from the requester without going through the user terminal 4 by directly accepting the input.

On the other hand, the robot arm system 2 according to the present embodiment is arranged at a remote location with respect to the learning device 1 that performs machine learning for acquiring a designated ability in response to a learning request from a client, and the learning is performed. This is an example of a learning target device that is a target for acquiring the ability specified by machine learning of the device 1. The robot arm system 2 according to the present embodiment includes a robot arm 30 that performs a predetermined work, and a robot controller (RC) 20 that controls the robot arm 30. Further, the robot controller may be a PLC (programmable logic controller) or the like. As a result, the robot arm system 2 receives a remote control command from the learning device 1 instructing the execution of the operation related to the designated ability, and according to the received remote control command, the operation related to the designated ability. Is configured to run. That is, the robot arm system 2 is configured so that the robot arm 30 executes the operation specified by the learning device 1 by the RC20.

Further, the robot arm system 2 includes a display 32 that performs a predetermined display. The display 32 is arranged in a place visible to workers in the factory existing around the robot arm system 2, such as in the vicinity of the robot arm 30. The display 32 is an example of the "display unit" of the present invention. The robot arm system 2 according to the present embodiment is configured to display on the display 32 that the learning device 1 is being operated by remote control while executing the operation in accordance with a command of a remote control command.

Thereby, in the present embodiment, it is possible to notify the workers in the factory that the robot arm system 2 is being remotely controlled by the learning device 1 through the display of the display 32. Therefore, according to the present embodiment, it is possible to ensure the safety around the robot arm system 2 while the remote control by the learning device 1 is being performed.

In the present embodiment, the robot arm system 2 includes a camera 31 that monitors the state of the movable range of the robot arm 30. The camera 31 is an example of a monitoring device (imaging device) that monitors the state of the movable range of the robot arm 30. Further, in the present embodiment, in addition to the robot arm system 2, a robot device 5 that can move heteronomously or autonomously by an operator's operation is performing work in the factory.

§2 Configuration example [Hardware configuration]
<Learning device>
Next, an example of the hardware configuration of the learning device 1 according to the present embodiment will be described with reference to FIG. FIG. 2 schematically illustrates an example of the hardware configuration of the learning device 1 according to the present embodiment.

As shown in FIG. 2, the learning device 1 according to the present embodiment is a computer to which the control unit 11, the storage unit 12, the communication interface, the input device 14, the output device 15, and the drive 16 are electrically connected. In FIG. 2, the communication interface is described as "communication I / F".

The control unit 11 includes a CPU, a RAM (Random Access Memory), a ROM (Read Only Memory), and the like, and is configured to execute various information processing based on a program and data. The storage unit 12 is composed of, for example, a hard disk drive, a solid state drive, or the like, and has a learning program 121 executed by the control unit 11, learning data 122 used for learning of the learner, and a robot arm having the ability specified by the client. The ability granting data 123 and the like to be given to the system 2 are stored.

The learning program 121 is a program for causing the learning device 1 to execute a machine learning process (FIG. 7) described later. The learning data 122 is data used for machine learning of the ability specified by the client, and is collected from the remotely controlled robot arm system 2. The ability-giving data 123 is data for imparting the ability acquired as a result of machine learning to the robot arm system 2. Details will be described later.

The communication interface 13 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication via the network 10. The learning device 1 can perform data communication with the robot arm system 2 and the user terminal 4 via the network 10 by the communication interface 13. The type of network 10 may be appropriately selected from, for example, the Internet, a wireless communication network, a mobile communication network, a telephone network, a dedicated network, and the like.

The input device 14 is, for example, a device for inputting a mouse, a keyboard, or the like. The output device 15 is, for example, a device for outputting a display, a speaker, or the like. The operator can operate the learning device 1 via the input device 14 and the output device 15.

The drive 16 is, for example, a CD drive, a DVD drive, or the like, and is a drive device for reading a program stored in the storage medium 91. The type of the drive 16 may be appropriately selected according to the type of the storage medium 91. The learning program 121 may be stored in the storage medium 91.

The storage medium 91 stores the information of the program or the like by electrical, magnetic, optical, mechanical or chemical action so that the information of the program or the like recorded by the computer or other device, the machine or the like can be read. It is a medium to do. The learning device 1 may acquire the learning program 121 from the storage medium 91.

Here, in FIG. 2, as an example of the storage medium 91, a disc-type storage medium such as a CD or a DVD is illustrated. However, the type of the storage medium 91 is not limited to the disc type, and may be other than the disc type. Examples of storage media other than the disk type include semiconductor memories such as flash memories.

Regarding the specific hardware configuration of the learning device 1, the components can be omitted, replaced, or added as appropriate according to the embodiment. For example, the control unit 11 may include a plurality of processors. The learning device 1 may be composed of a plurality of information processing devices. Further, the learning device 1 may be a general-purpose server device, a PC (Personal Computer), or the like, in addition to an information processing device designed exclusively for the provided service. In order to execute the machine learning process described later, it is preferable that the learning device 1 is configured to have a higher machine power than the robot arm system 2. The machine power may be specified by the processing speed of the CPU, the storage capacity of the memory, the read speed of the memory, and the like. For example, the learning device 1 may have a higher machine power than the robot arm system 2 by having a CPU that operates at a higher speed than the RC20 of the robot arm system 2. Further, when the processing speeds of the CPUs of the learning device 1 and the RC20 are the same, the RAM of the learning device 1 has a larger capacity or a higher speed than the RAM of the RC20, so that the learning device 1 is more than the robot arm system 2. The machine power may be higher.

<Robot arm system>
Next, an example of the hardware configuration of the robot arm system 2 according to the present embodiment will be described with reference to FIGS. 3 and 4. FIG. 3 schematically illustrates an example of the hardware configuration of the RC20 according to the present embodiment. FIG. 4 schematically illustrates an example of the operating state of the robot arm 30 according to the present embodiment. As shown in FIGS. 1 and 3, the robot arm system 2 according to the present embodiment includes an RC 20, a robot arm 30, a camera 31, and a display 32. Hereinafter, each component will be described.

(RC)
First, RC20 will be described. The RC20 according to the present embodiment is a computer to which the control unit 21, the storage unit 22, the external interface 23, and the communication interface 24 are electrically connected. As a result, the RC 20 is configured to control the operations of the robot arm 30, the camera 31, and the display 32. In FIG. 3, the external interface and the communication interface are described as "external I / F" and "communication I / F".

The control unit 21 includes a CPU, RAM, ROM, and the like, and is configured to execute various information processing based on programs and data. The storage unit 22 is composed of, for example, a RAM, a ROM, or the like, and stores the control program 221 or the like. The control program 221 is a program for causing the RC 20 to execute a control process (FIG. 8) of the robot arm 30 described later. The control unit 21 is configured to execute the processing of each step described later by interpreting and executing the control program 221.

The external interface 23 is an interface for connecting to an external device, and is appropriately configured according to the external device to be connected. In this embodiment, the RC 20 is connected to the robot arm 30, the camera 31, and the display 32 via each external interface 23.

The communication interface 24 is, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like, and is an interface for performing wired or wireless communication. The communication interface 24 is an example of the "communication unit" of the present invention. The RC20 uses the communication interface 24 to generate data between the learning device 1 located at a remote location and the peripheral device (for example, the self-propelled robot device 5) arranged around the robot arm system 2 in the factory. Can communicate.

Regarding the specific hardware configuration of RC20, it is possible to omit, replace, and add components as appropriate according to the embodiment. The control unit 21 may include a plurality of processors. The control unit 21 may be composed of an FPGA. The storage unit 22 may be composed of a RAM and a ROM included in the control unit 21. The storage unit 22 may be composed of an auxiliary storage device such as a hard disk drive or a solid state drive. Further, the RC20 may be a general-purpose desktop PC, a tablet PC, or the like, depending on the object to be controlled, in addition to the information processing device designed exclusively for the provided service.

(Robot arm)
Next, the robot arm 30 will be described. The robot arm 30 may be appropriately configured so that the desired work can be performed. In the example of FIG. 4, the robot arm 30 includes a base portion 301 as a starting point, two joint portions 302 as movable axes, two link portions 303 forming a skeleton, and an end effector 304 attached to the tip. There is.

Each joint portion 302 includes a drive motor such as a servo motor or a brushless motor, and each link portion 303 is configured to be rotatable or rotatable. Further, an angle sensor capable of detecting an angle such as a rotary encoder is attached to each joint portion 302. Thereby, the robot arm 30 is configured so that the angle of each joint portion 302 can be specified.

The end effector 304 is appropriately formed according to the work to be performed in the factory. The end effector 304 may be equipped with a force sensor configured to detect a force acting on the end effector 304. Thereby, the robot arm 30 can be configured to detect the force acting on the end effector 304.

The robot arm 30 has a movable range 308 according to each joint portion 302, each link portion 303, and an end effector 304. That is, the movable range 308 is a range that can be reached by the end effector 304 by driving each joint portion 302. In the present embodiment, a permission range 309 that permits the operation of the robot arm 30 is set within the movable range 308. Details will be described later.

Regarding the specific configuration of the robot arm 30, components can be omitted, replaced, or added as appropriate according to the embodiment. For example, the number of joint portions 302 and link portions 303 may be appropriately selected according to the embodiment. Further, in addition to the angle sensor, a torque sensor may be attached to each joint portion 302. Thereby, each joint portion 302 can be controlled by torque.

(display)
In this embodiment, the display 32 is used to display the status of the robot arm system 2 (robot arm 30). Therefore, the display 32 does not have to be particularly limited as long as the status can be displayed, and a known liquid crystal display, touch panel display, or the like may be used.

(camera)
In the present embodiment, the camera 31 is arranged so as to capture the situation of the movable range 308 of the robot arm system 2 (robot arm 30). As a result, the situation of the movable range 308 is reflected in the captured image captured by the camera 31. This captured image is an example of situation information indicating the situation of the movable range 308. The camera 31 may be fixed at a predetermined position, or may be configured so that the shooting direction (orientation) can be changed by a motor or the like. As the camera 31, a general digital camera, a video camera, a 360-degree camera, or the like may be used, or a camera for visible light photography or an infrared light photography may be used.

<Robot device>
The robot device 5 includes, for example, a control unit composed of a CPU or the like, a storage unit that stores programs, a communication interface that communicates with the RC20, a robot arm similar to the robot arm 30, and other robotic or autonomous movement. It is equipped with a wheel module and the like for this purpose. As a result, the robot device 5 is appropriately configured to move in the factory and perform a predetermined work. The type of the robot device 5 is not particularly limited, and of course, it does not have to be a humanoid type, and may be appropriately selected according to the work to be performed in the factory.

<User terminal>
The user terminal 4 is, for example, a computer to which a control unit composed of a CPU or the like, a storage unit for storing programs or the like, a communication interface for performing communication via a network, and an input / output device are electrically connected. .. The user terminal 4 is used by the client to request machine learning (learning request) to the service provider who operates the learning device 1. As the user terminal 4, for example, a desktop PC, a tablet PC, a mobile phone including a smartphone, or the like that can be connected to a network may be used.

[Software configuration]
<Learning device>
Next, an example of the software configuration of the learning device 1 according to the present embodiment will be described with reference to FIG. FIG. 5 schematically illustrates an example of the software configuration of the learning device 1 according to the present embodiment.

The control unit 11 of the learning device 1 expands the learning program 121 stored in the storage unit 12 into the RAM. Then, the control unit 11 interprets and executes the learning program 121 expanded in the RAM by the CPU to control each component. As a result, as shown in FIG. 5, the learning device 1 according to the present embodiment has a learning request receiving unit 110, a permission range setting unit 111, a status acquisition unit 112, a remote operation unit 113, and a learning data collecting unit as software modules. It is configured as a computer including 114, a learning processing unit 115, an ability-imparting data generation unit 116, and a distribution unit 117.

The learning request reception unit 110 receives from the requester the designation of the learning target device located at a remote location and the designation of the ability acquired by the designated learning target device by machine learning as a learning request. In the present embodiment, it is assumed that the request for machine learning of the robot arm system 2 is received from the client.

The permission range setting unit 111 sets a permission range for permitting the operation of the learning target device within the movable range of the learning target device. In the present embodiment, the permission range setting unit 111 sets the permission range 309 within the movable range 308 of the robot arm 30.

The status acquisition unit 112 acquires status information indicating the status of the movable range from the monitoring device that monitors the status of the movable range of the learning target device. In the present embodiment, the situation acquisition unit 112 acquires the captured image captured by the camera 31 as the status information from the camera 31 arranged so as to capture the movable range 308.

By transmitting the control data to the learning target device, the remote control unit 113 remotely controls the learning target device so as to execute an operation related to the ability specified in the learning request. The learning data collection unit 114 collects learning data for machine learning of a specified ability based on the result of remote control.

In the present embodiment, the remote control unit 113 remotely controls the robot arm 30 of the robot arm system 2 by transmitting control data instructing a predetermined operation to the RC 20 via the network 10. At this time, the remote control unit 113 remotely controls the robot arm system 2 so as to operate within the designated permission range 309 based on the captured image acquired from the camera 31. Then, the learning data collection unit 114 uses the target data indicating the work goal to be achieved for the designated ability and the sensor data obtained in the operation process until the work goal is achieved as input data, and sets the work goal. The learning data 122 using the control data transmitted to the RC 20 as the teacher data in the operation process until the achievement is collected is collected.

The learning processing unit 115 uses the collected learning data to perform machine learning of the learning device so as to acquire a designated ability. The ability-giving data generation unit 116 creates the ability-giving data for imparting the specified ability to the learning target device by equipping the learning target device with the learned learning device for which machine learning has been completed. The distribution unit 117 distributes the generated ability-imparting data to the learning target device.

In the present embodiment, the learning processing unit 115 performs machine learning of the neural network 6 using the learning data 122 collected from the robot arm system 2. The ability-giving data generation unit 116 generates the ability-giving data 123 for equipping the RC20 with the trained neural network 6. The distribution unit 117 distributes the generated capacity-imparting data 123 to the RC20 via the network 10.

(Learning device)
Next, the learner will be described. As shown in FIG. 5, the learner according to the present embodiment is configured by the neural network 6. The neural network 6 is a multi-layered neural network used for so-called deep learning, and includes an input layer 61, an intermediate layer (hidden layer) 62, and an output layer 63 in order from the input.

In the example of FIG. 5, the neural network 6 includes the intermediate layer 62 of one layer, the output of the input layer 61 is the input of the intermediate layer 62, and the output of the intermediate layer 62 is the input of the output layer 63. ing. However, the number of intermediate layers 62 is not limited to one, and the neural network 6 may include two or more intermediate layers 62.

Each layer 61-63 comprises one or more neurons. For example, the number of neurons in the input layer 61 can be set according to the input data used for input. The number of neurons in the intermediate layer 62 can be appropriately set according to the embodiment. Further, the number of neurons in the output layer 63 can be set according to the control data to be output. A threshold is set for each neuron, and basically, the output of each neuron is determined by whether or not the sum of the products of each input and each weight exceeds the threshold.

Neurons in adjacent layers are appropriately connected to each other, and a weight (connection load) is set for each connection. In the example of FIG. 5, each neuron is connected to all neurons in the adjacent layer, but the connection of neurons does not have to be limited to such an example and is appropriately set according to the embodiment. You can.

The learning processing unit 115 constructs the neural network 6 so as to output the control data as an output value when the target data and the sensor data included in the collected learning data 122 are input by the learning process of the neural network. The ability-giving data generation unit 116 configures the constructed neural network 6 (for example, the number of layers of the neural network, the number of neurons in each layer, the connection relationship between neurons, the transmission function of each neuron), and the weight of the connection between each neuron. , And generate capacity-imparting data 123 containing information indicating the threshold of each neuron.

<Robot arm system>
Next, an example of the software configuration of the robot arm system 2 according to the present embodiment will be described with reference to FIG. FIG. 6 schematically illustrates an example of the software configuration of the robot arm system 2 including the RC20 according to the present embodiment.

The control unit 21 of the RC20 expands the control program 221 stored in the storage unit 22 into the RAM. Then, the control unit 21 interprets and executes the control program 221 expanded in the RAM by the CPU to control each component. As a result, as shown in FIG. 6, the robot arm system 2 including the RC20 according to the present embodiment includes a remote control reception unit 211, an operation processing unit 212, a display control unit 213, and a notification unit 214 as software modules. Configured as a computer.

The remote control reception unit 211 receives a remote control command from the learning device 1 instructing the execution of an operation for learning related to the specified ability. The motion processing unit 212 executes the motion related to the designated ability according to the received remote control command. The display control unit 213 causes the display 32 to display that the learning device 1 is being operated by remote control while the operation is being executed in accordance with the remote control command. The notification unit 214 notifies the peripheral device (for example, the robot device 5) that the learning device 1 is being operated by remote control while the operation is being executed in accordance with the remote control command.

<Others>
Each software module of the learning device 1 and the robot arm system 2 (RC20) will be described in detail in an operation example described later. In this embodiment, an example in which each software module of the learning device 1 and the RC20 is realized by a general-purpose CPU is described. However, some or all of the above software modules may be implemented by one or more dedicated processors. Further, with respect to the software configurations of the learning device 1 and the RC20, software modules may be omitted, replaced, or added as appropriate according to the embodiment.

§3 Operation example [Learning device]
Next, an operation example of the learning device 1 will be described with reference to FIG. 7. FIG. 7 is a flowchart illustrating an example of the processing procedure of the learning device 1 according to the present embodiment. The processing procedure described below is only an example, and each processing may be changed as much as possible. Further, with respect to the processing procedure described below, steps can be omitted, replaced, and added as appropriate according to the embodiment.

(Step S101)
In step S101, the control unit 11 operates as the learning request receiving unit 110 and receives the learning request from the requester. For example, the client operates the user terminal 4 to specify the learning target device to be machine-learned and the ability to be acquired by the learning target device by machine learning. The input of this learning request may not be performed by the requester himself, but may be performed by a person who has heard the request contents from the requester. That is, the input of the request contents does not have to be performed by the requester himself / herself. When the designation of the learning target device and the acquisition target ability is completed, the control unit 11 proceeds to the next step S102.

In this step S101, the learning device 1 receives the designation of the learning target device and acquires the information used for the remote control of the learning target device. For example, the learning device 1 acquires the IP address of the RC20 or the like as information used for remote control of the robot arm system 2 in response to the designation of the robot arm system 2 as the learning target device.

Further, the learning device 1 may accept the designation of the ability to be the target of machine learning by presenting the requester with a list of abilities that can be acquired by machine learning according to the type of the device to be learned. A list of abilities to be subject to machine learning may be prepared in advance as a template for each device to be learned.

The ability to be the target of machine learning may be appropriately selected from all the abilities that can be implemented in the device to be learned. For example, when the robot arm 30 is used for work such as moving, attaching, processing, deburring, soldering, and welding parts, the ability to perform the work on a new object is the target of machine learning. May be specified in. In addition, when those works are already in operation, the ability to efficiently carry out the work during the operation may be designated as the target of machine learning.

Further, in the step S101, the control unit 11 may specify the ability to be acquired by machine learning and accept the input of the achievement condition of the ability. The ability achievement condition is an additional condition of the ability to be acquired by the learning target device, for example, a time condition such that a certain designated work is performed within a few seconds.

(Step S102)
In step S102, the control unit 11 operates as the permission range setting unit 111, and sets the permission range for permitting the operation of the learning target device within the movable range of the learning target device specified in step S101. In the present embodiment, the control unit 11 sets a permission range 309 that permits the operation of the robot arm 30 within the movable range 308 of the robot arm 30. When the setting of the permission range 309 is completed, the control unit 11 proceeds to the next step S103.

The permission range 309 may be set as appropriate. For example, the control unit 11 may accept the designation of the permission range 309 from the operator. In this case, the operator operates the input device 14 to set the permitted range 309 within the movable range 308. At this time, the learning device 1 may acquire a photographed image from the camera 31 that photographs the situation of the movable range 308, and output the acquired photographed image to the output device 15. As a result, the operator can specify the permission range 309 in the output captured image after omitting a place irrelevant to the ability specified in step S101. Further, the permission range 309 may be changed in real time based on the image processing result of the captured image acquired from the camera 31. For example, when it is determined by image processing that a person or an object exists within the movable range 308, the portion where the person or the object exists may be excluded from the range, and the permitted range 309 at that time may be set.

Further, for example, the control unit 11 may accept the designation of the permission range 309 from the requester. In this case, the control unit 11 may accept the designation of the permission range 309 together with the learning request in step S101. As a result, the control unit 11 can set the permission range 309 within the movable range 308 based on the input of the requester.

Further, for example, the control unit 11 may automatically set the permission range 309 within the movable range 308 based on the ability specified in step S101. In this case, the control unit 11 may specify an area related to the implementation of the capability specified in step S101, and set the specified area in the permission range 309.

(Step S103)
In step S103, the control unit 11 operates as the status acquisition unit 112, and acquires status information indicating the status of the movable range from the monitoring device that monitors the status of the movable range of the learning target device specified in step S101. .. In the present embodiment, the control unit 11 accesses the RC20 using the information acquired in step S101, and uses the camera 31 connected to the RC20 to photograph the situation of the movable range 308. As a result, the control unit 11 can acquire a captured image showing the situation of the movable range 308 as the situation information. When the captured image is acquired, the control unit 11 proceeds to the next step S104.

(Step S104)
In step S104, the control unit 11 operates as the remote control unit 113, and by transmitting the control data to the learning target device, the learning target device is remotely controlled so as to execute the operation related to the ability specified in step S101. Manipulate. In the present embodiment, the control unit 11 transmits control data for instructing a predetermined operation related to the capability specified in step S101 to the RC 20 via the network 10. The control data defines, for example, the driving amount of the driving motor of each joint portion 302. As will be described later, the RC 20 drives each joint portion 302 of the robot arm 30 based on the received control data. As a result, the control unit 11 remotely controls the robot arm system 2. After performing the remote control of the robot arm system 2, the control unit 11 proceeds to the next step S105.

The content of the operation by remote control may be appropriately determined. For example, the content of the operation by remote control may be determined by the operator. Further, for example, a plurality of templates may be prepared, each of which defines different movements of the robot arm 30. In this case, the control unit 11 may determine the content of the operation by remote control by randomly selecting the template. Further, the control unit 11 determines the content of the operation by the remote control so as to be suitable for the ability of the acquisition target specified in step S101 by using a method such as dynamic programming during the repeated remote control. May be good. Further, the control unit 11 may cause the robot arm system 2 to execute a series of operations including a plurality of steps by the remote control.

In the present embodiment, in step S102, the permission range 309 is set within the movable range 308, and in step S103, the control unit 11 captures a captured image of the situation of the movable range 308 from the camera 31. Have acquired. Therefore, in this step S104, the control unit 11 remotely controls the robot arm system 2 so that the robot arm 30 operates within the set permission range 309 based on the captured image acquired from the camera 31. That is, the control unit 11 remotely controls the robot arm system 2 while confirming whether or not the robot arm 30 is out of the permitted range 309 by using the captured image acquired from the camera 31.

In addition, the control unit 11 monitors whether or not an intruder (for example, a person, an object, etc.) has invaded the permitted range 309. Whether or not an intruder has invaded the permitted range 309 can be determined by, for example, known image processing such as template matching. When the control unit 11 determines that an intruder has entered the permitted range 309, the control unit 11 suspends (interrupts) the transmission of the remote control command to the robot arm system 2. At this time, the control unit 11 may send a command for executing an announcement to the robot arm system 2 to move the intruded object out of the permitted range 309. Then, after the intruder exits from the permitted range 309, the control unit 11 resumes the transmission of the remote control command to the robot arm system 2. As a result, safety within the permitted range 309 can be ensured.

(Step S105)
In step S105, the control unit 11 operates as a learning data collecting unit 114, and collects learning data 122 for machine learning of the specified ability based on the result of the remote control in step S104. After collecting the learning data 122, the control unit 11 proceeds to the next step S106.

The content of the learning data 122 may be appropriately determined according to the type of the learning device, the type of the learning target device, the ability of the acquisition target, and the like. In the present embodiment, the neural network 6 is used as a learning device, the robot arm system 2 is designated as a learning target device, and the robot arm 30 has the ability to perform a new work or a work in operation more efficiently. Is designated as the ability to be acquired. Further, it is assumed that the RC 20 controls the operation of the robot arm 30 based on the target to be achieved and the sensor data of the angle sensor of each joint portion 302.

In this case, the control unit 11 creates target data indicating the work target to be achieved according to the ability of the acquisition target specified in step S101. The content of the target data may be appropriately determined according to the embodiment. For example, the position, angle, moving speed, and the like of the robot arm 30 may be specified in the target data according to the completion of the target work within a predetermined time. When improving the operation of the robot arm 30, the control unit 11 acquires a photographed image of the operation of the robot arm 30 from the camera 31 and analyzes the acquired image to analyze the operation of the robot arm 30. The content of the target data may be determined so as to improve.

Next, the control unit 11 determines whether or not the robot arm system 2 has achieved the work target indicated by the target data based on the result of the remote control in step S104. When the robot arm system 2 achieves the work target indicated by the target data, the control unit 11 acquires the control data transmitted to the RC 20 in the operation process until the work target is achieved. Further, the control unit 11 acquires sensor data detected from the angle sensor of each joint portion 302 from the RC 20 in the operation process until the work target is achieved. The sensor data is an example of state data indicating the state of the learning target device (robot arm system 2). The sensor data may be acquired before the robot arm 30 is driven by the command indicated by the control data.

Then, the control unit 11 uses the control data as the teacher data, and uses the sensor data and the target data obtained immediately before performing the operation based on the control data as the teacher data as the input data, and links the target data, the sensor data, and the control data. Attach and make a set. As a result, the control unit 11 collects the learning data 122 including the target data and the sensor data as input data and the control data as teacher data. That is, in this step S105, the control unit 11 ignores the result of the remote control when the work target cannot be achieved, and from the result of the remote control when the specified ability is achieved, the machine learning of the ability is performed. Collect learning data for use.

Whether or not the work goal has been achieved may be appropriately determined. For example, the control unit 11 may acquire a captured image in which the result of the remote control is captured from the camera 31 and analyze the acquired captured image to determine whether or not the work target has been achieved. Further, the result of remote control may be obtained by the robot arm system 2 and various sensors (angle sensors and the like) provided around the robot arm system 2. In this case, the control unit 11 may determine whether or not the work target has been achieved based on the detection results of the robot arm system 2 and various sensors (angle sensors and the like) provided around the robot arm system 2. Further, in step S101, when the input of the achievement condition of the ability is accepted together with the designation of the ability to be acquired, the control unit 11 appropriately determines whether or not the achievement condition has been achieved. May be good.

(Step S106)
In step S106, the control unit 11 determines whether or not a sufficient number of learning data 122s have been collected. When it is determined that a sufficient number of learning data 122s have been collected, the control unit 11 proceeds to the next step S107. On the other hand, if it is determined that a sufficient number of learning data 122s have not been collected, the control unit 11 repeats the processes of steps S103 to S105.

The determination may be made using the threshold value. That is, the control unit 11 may determine whether or not a sufficient number of learning data 122s have been collected by comparing the number of collected learning data 122s with the threshold value. At this time, the threshold value may be set by the operator or may be set according to the ability to be the target of machine learning. The threshold setting method can be appropriately selected according to the embodiment.

(Step S107)
In step S107, the control unit 11 transmits a completion notification indicating that the remote control for machine learning has been completed to the robot arm system 2. When the transmission of the completion notification is completed, the control unit 11 proceeds to the next step S108.

(Step S108)
In step S108, the control unit 11 operates as a learning processing unit 115, and uses the learning data 122 collected in step S105 to perform machine learning of the neural network 6 so as to acquire a designated ability.

Specifically, first, the control unit 11 prepares a neural network 6 to be processed by machine learning. The configuration of the neural network 6 to be prepared, the initial value of the weight of the connection between each neuron, and the initial value of the threshold value of each neuron may be given by the template or by the input of the operator. When performing relearning, the control unit 11 determines the neural network 6 based on the configuration of the neural network to be relearned, the weight of the connection between each neuron, and the learning result data indicating the threshold value of each neuron. May be prepared.

Next, the control unit 11 learns the neural network 6 by using the target data and the sensor data included in the learning data 122 collected in step S105 as input data and the control data as training data. A gradient descent method, a stochastic gradient descent method, or the like may be used for learning the neural network 6.

For example, the control unit 11 inputs the target data and the sensor data included in the learning data 122 to the input layer 61, and performs arithmetic processing in the forward propagation direction of the neural network 6. As a result, the control unit 11 obtains an output value from the output layer 63 of the neural network 6. Next, the control unit 11 calculates an error between the output value output from the output layer 63 and the control data included in the learning data 122. Subsequently, the control unit 11 calculates the error of the connection weight between each neuron and the error of each threshold value of each neuron by using the error of the output value calculated by the error back propagation method. Then, the control unit 11 updates the weight of the connection between each neuron and the value of each threshold value of each neuron based on each calculated error.

The control unit 11 performs machine learning of the neural network 6 by repeating this series of processes for each learning data 122 until the output value output from the output layer 63 matches the corresponding control data. Thereby, when the target data and the sensor data are input, the trained neural network 6 that outputs the corresponding control data can be constructed. When the machine learning of the neural network 6 is completed, the control unit 11 proceeds to the next step S109.

(Step S109)
In step S109, the control unit 11 operates as the ability granting data generation unit 116, and equips the robot arm system 2 (RC20) with the trained neural network 6 for which machine learning has been completed, so that the robot arm has the specified ability. The ability granting data 123 for granting to the system 2 is generated. After generating the ability granting data 123, the control unit 11 proceeds to the next step S110.

The format of the ability-imparting data 123 may be appropriately determined according to the embodiment. For example, when the RC20 performs arithmetic processing using a neural network, the control unit 11 determines the configuration of the neural network 6 constructed in step S108, the weight of the connection between each neuron, and the threshold value of each neuron. The learning result data indicating the above may be generated as the ability-imparting data 123. Further, for example, when the RC20 includes an FPGA, the control unit 11 may generate data to be written in the FPGA in order to realize the neural network 6 constructed in step S108 in the FPGA as the capacity granting data 123. .. Further, for example, the control unit 11 may generate a program for causing the RC 20 to execute the arithmetic processing by the neural network 6 constructed in step S108, or patch data for modifying the program as the ability granting data 123. The ability-imparting data 123 of each format may be automatically generated by using a known automatic program generation method or the like.

(Step S110)
In step S110, the robot arm system 2 operates as the distribution unit 117 and distributes the ability granting data 123 generated in step S109 to the robot arm system 2 via the network 10. The RC20 can acquire the ability specified in step S101 by installing the received ability grant data 123. When the distribution of the capacity granting data 123 is completed, the control unit 11 ends the process related to this operation example.

[Robot arm system]
Next, an operation example of the robot arm system 2 will be described with reference to FIG. FIG. 8 is a flowchart illustrating an example of the processing procedure of the robot arm system 2 according to the present embodiment. The processing procedure described below is only an example, and each processing may be changed as much as possible. Further, with respect to the processing procedure described below, steps can be omitted, replaced, and added as appropriate according to the embodiment.

(Step S201)
In step S201, the control unit 21 of the RC20 operates as the remote control reception unit 211, and receives a remote control command from the learning device 1 instructing the execution of the operation related to the designated ability. This step S201 is an example of the "reception step" of the present invention. Specifically, the control unit 21 receives a remote control command from the learning device 1 based on the control data in step S104. At this time, the control unit 21 may receive a plurality of control data instructing the execution of the plurality of operations. Upon receiving the control data, the control unit 21 proceeds to the next step S202.

(Step S202)
In step S202, the control unit 21 operates as the operation processing unit 212, and executes the operation related to the designated ability in accordance with the remote control command received in step S201. This step S202 is an example of the "execution step" of the present invention. In the present embodiment, the control unit 21 drives the drive motor of each joint unit 302 based on the control data, so that the robot arm 30 executes an operation in response to a remote control command. The control unit 21 executes the following steps S203 and S204 while executing the operation in accordance with the remote control command in the step S202.

The learning device 1 monitors whether or not an intruder has invaded the permitted range 309, and if it is determined that the intruder has invaded the permitted range 309, the remote control is temporarily suspended. At this time, the control unit 21 may display an announcement on the display 32 for moving the intruded object out of the permitted range 309. When the RC20 is connected to a speaker (not shown), the announcement may be output from the speaker. The control unit 21 may perform this announcement in response to a command from the learning device 1.

(Step S203)
In step S203, the control unit 21 operates as the display control unit 213 and causes the display 32 to indicate that the learning device 1 is being operated by remote control. This step S203 is an example of the "display step" of the present invention. When the display control of the display 32 is completed, the control unit 21 proceeds to the next step S204.

Here, the content to be displayed on the display 32 is not particularly limited as long as it is related to the fact that the learning device 1 is being operated by remote control. For example, the control unit 21 may display “during operation by remote control” or “during learning by remote control” on the display 32. Further, for example, the control unit 21 may display the content of the operation of the learning device 1 by remote control on the display 32 by referring to the control data.

Further, for example, when a plurality of control data are received in step S201, the control unit 21 may display the content of the operation to be executed next to the operation being executed in step S202 on the display 32. At this time, the control unit 21 may display the content of the operation being executed on the display 32 together with the content of the operation to be executed next.

Further, for example, when the operation being executed in step S202 is a dangerous operation or an operation at a higher speed than usual, the control unit 21 is that the operation during the execution is a dangerous operation or an operation at a higher speed. May be displayed on the display 32. The display content indicating that the operation being executed is a dangerous operation may be appropriately determined according to the embodiment. For example, the control unit 21 may display on the display 32 that "the operation currently being executed is dangerous" or "the operation currently being executed is an operation at high speed". Further, for example, the control unit 21 may display a message on the display 32 for calling attention to a person around the robot arm 30 as a display content indicating that the operation being executed is a dangerous operation.

The method for determining whether or not the operation being executed is a dangerous operation may be appropriately selected depending on the embodiment. For example, the control unit 21 may determine whether or not the operation being executed is a dangerous operation in step S202 based on the condition defining the dangerous operation. Further, for example, the control data may include information indicating that the target operation is dangerous. In this case, the control unit 21 can determine whether or not the operation being executed in step S202 is a dangerous operation by referring to the control data received in step S201.

(Step S204)
In step S204, the control unit 21 operates as the notification unit 214 and controls the communication interface 24 to notify the peripheral device (for example, the robot device 5) that the learning device 1 is being operated by remote control. .. This step S204 is an example of the "notification step" of the present invention. When the notification is completed, the control unit 21 proceeds to the next step S205.

The peripheral device that has received the notification can recognize that the robot arm system 2 is remotely controlled by the learning device 1. As a result, the peripheral device can be prevented from interfering with the operation of the robot arm 30 by remote control. For example, the robot device 5 configured to be movable in the factory may be kept away from the vicinity of the robot arm 30 (particularly, the movable range 308 or the permitted range 309) in response to receiving the notification. it can. That is, in order not to interfere with the operation of the robot arm 30 by remote control, a movement limit is set according to the remote control so that the robot device 5 moves while avoiding the vicinity of the robot arm 30. can do.

(Step S205)
In step S205, the control unit 21 determines whether or not the remote control from the learning device 1 is completed. In the present embodiment, when the remote control is completed, the learning device 1 transmits a completion notification in step S107. Therefore, the control unit 21 determines whether or not the remote control by the learning device 1 is completed depending on whether or not the completion notification is received. When it is determined that the remote control is completed, in other words, after the remote control from the learning device 1 is completed, the control unit 21 proceeds to the next step S206. On the other hand, if it is determined that the remote control has not been completed, the control unit 21 repeats the processes of steps S201 to S204.

(Step S206)
In step S206, the control unit 21 operates as the display control unit 213, and displays on the display 32 that the operation by remote control of the learning device 1 is completed. This step S206 is an example of the "completion display step" of the present invention. The content to be displayed on the display 32 may not be particularly limited as long as the content is related to the completion of the operation by remote control of the learning device 1. For example, the control unit 21 may display “remote control end” or “remote control operation completed” on the display 32. As a result, it is possible to notify the operator existing around the robot arm system 2 that the operation of the learning device 1 by remote control is completed and that the learning device 1 does not move suddenly. When the completion display is completed, the control unit 21 proceeds to the next step S207.

(Step S207)
In step S207, the control unit 21 operates as the notification unit 214 and controls the communication interface 24 to notify the peripheral device (for example, the robot device 5) that the operation of the learning device 1 by remote control is completed. .. This step S207 is an example of the "completion notification step" of the present invention. When the notification is completed, the control unit 21 ends the process related to this operation example.

The peripheral device that has received the notification can recognize that the remote control has been completed by the learning device 1 in the robot arm system 2. Thereby, for example, the robot device 5 configured to be movable in the factory may approach a nearby area (particularly within the permission range 309) while the robot arm 30 is not operating. That is, it is possible to release the movement restriction according to the remote control so that the robot device 5 may pass in the vicinity of the robot arm 30.

[Action / Effect]
As described above, the learning device 1 according to the present embodiment receives the designation of the learning target device to be machine-learned and the ability to be acquired by the learning target device as a learning request from the requester in step S101. .. Further, the learning device 1 collects learning data 122 to be used for machine learning of the ability specified in the learning request by remotely controlling the learning target device (robot arm system 2) in steps S104 and S105. To do. Further, the learning device 1 uses the learning data 122 collected in step S108 to perform machine learning of the neural network 6 so as to acquire the ability specified in the learning request. As a result, it is possible to construct a trained neural network 6 for causing the learning target device to perform the ability specified in the learning request. Further, the learning target device (robot arm system 2) arranged at a remote location merely executes the operation related to the ability specified in step S101 by steps S201 and S202, and the machine learning of step S108. The process is executed by the learning device 1. Therefore, even if the machine power of the learning target device located at a remote location is limited, it is possible to carry out the machine learning process of the ability to be acquired by the learning target device. Therefore, according to the present embodiment, it is possible to provide a technical mechanism for appropriately adding a new capability to a device located at a remote location.

Further, while the robot arm system 2 according to the present embodiment is operating according to a remote control command from the learning device 1, it is displayed by step S203 that the robot arm system 2 is being operated by remote control of the learning device 1. It is displayed on 32. In the present embodiment, through the display of the display 32, it is possible to notify the operator existing around the robot arm system 2 that the robot arm 30 is remotely controlled, whereby the surroundings of the robot arm 30 can be notified. Safety can be ensured. Therefore, according to the present embodiment, it is possible to provide a mechanism for ensuring the safety around the learning target device when the learning target device is remotely controlled.

In step S203, the control unit 21 may display the content of the operation of the learning device 1 by remote control on the display 32. By adopting this display, it is possible to notify the operator existing around the robot arm system 2 of the content of the operation by remote control, thereby enhancing the safety around the robot arm system 2. ..

Further, in step S203, the control unit 21 may display the content of the operation to be executed next to the operation being executed in step S202 on the display 32. By adopting this display, it is possible to notify the operator existing around the robot arm system 2 in advance of the operation content of the robot arm 30. Thereby, the safety around the robot arm system 2 can be enhanced.

Further, in step S203, when the operation being executed in step S202 is a dangerous operation, the control unit 21 may display on the display 32 that the operation being executed is a dangerous operation. By adopting this display, it is possible to clearly inform the workers existing around the robot arm system 2 that danger may be imminent, thereby enhancing the safety around the robot arm system 2. ..

§4 Modifications Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in all respects. Needless to say, various improvements and modifications can be made without departing from the scope of the present invention. For example, the following changes can be made. In the following, the same reference numerals will be used for the same components as those in the above embodiment, and the same points as in the above embodiment will be omitted as appropriate. The following modifications can be combined as appropriate.

<4.1>
In the above embodiment, the robot arm system 2 has been illustrated as an example of the learning target device. However, the type of the device to be learned does not have to be limited to such an example, and may be appropriately selected depending on the embodiment.

For example, the learning target device may be a work robot such as the robot device 5 that moves in the warehouse and performs work such as carrying luggage. In this case, the learning request can be specified as the ability to acquire the procedure for efficiently transporting the luggage in the warehouse. Further, the movable range of the work robot becomes the movable range, and the movable range of the work robot can be restricted by setting the permitted range.

For example, the learning target device may be a vehicle capable of autonomous driving. In this case, in the learning request, using a test course or the like, the client can specify the ability to let the vehicle acquire automatic driving on the road itself, and as a part of the function during the automatic driving operation. It can also be specified as the ability to make the vehicle acquire automatic parking and the like. In this case, in order to set the movable range, a combination of either or both of a camera for photographing the outside of the vehicle and a laser for detecting an object outside the vehicle can be used. A display unit such as a display indicating that the vehicle is being operated by remote control may be attached to the outside of the vehicle or may be arranged at a predetermined location on the test course.

In addition, a plurality of devices may be designated as the learning target device instead of one device. For example, the robot arm system 2 may include a plurality of robot arms 30. In this case, in the learning request, it can be specified as the ability to acquire the work performed by a plurality of devices in collaboration.

<4.2>
Further, in the above embodiment, as shown in FIG. 5, a general forward-propagation neural network having a multi-layer structure is used as the neural network 6. However, the type of the neural network 6 does not have to be limited to such an example, and may be appropriately selected depending on the embodiment. For example, when an image is used as input data, a convolutional neural network including a convolutional layer and a pooling layer may be used as the neural network 6. Further, for example, when time-series data is used as input data, a recursive neural network having a recursive connection from the output side to the input side, such as an intermediate layer to an input layer, may be used for the neural network 6. .. The number of layers of the neural network 6, the number of neurons in each layer, the connection relationship between neurons, and the transfer function of each neuron may be appropriately determined according to the embodiment.

<4.3>
Further, in the above embodiment, the learner is configured by a neural network. However, the type of the learner does not have to be limited to the neural network, and may be appropriately selected depending on the embodiment. For example, as the learning device, a support vector machine, a self-organizing map, a learning device that learns by reinforcement learning, or the like may be used. When machine learning is performed using the reinforcement learning method, the machine learning process in step S108 may be performed while the remote control in step S104 is being performed.

<4.4>
Further, in the above embodiment, the camera 31 is exemplified as an example of the monitoring device for monitoring the status of the movable range 308. However, the type of the monitoring device does not have to be limited to the photographing device, and may be appropriately selected according to the embodiment. For example, the monitoring device may be a position detection system that detects the operating position of the learning target device (the position of the robot arm 30 in the above embodiment), which is composed of one or a plurality of infrared sensors. In this case, in step S103, the learning device 1 can acquire information indicating the detection result of the position detection system as status information.

Further, in the above embodiment, the camera 31 is connected to the RC20. Therefore, the learning device 1 can acquire a captured image from the camera 31 via the RC20 by using the information (for example, IP address) used for remote control of the learning target device specified in step S101. However, the method for the learning device 1 to acquire the situation information is not limited to such an example, and may be appropriately selected depending on the embodiment. For example, when the camera 31 can be connected to the network 10, the learning device 1 acquires information (for example, an IP address) used for accessing the camera 31 in the step S101, similarly to the learning target device. May be good.

When it is not necessary to monitor the movable range 308 of the robot arm system 2, steps S102 and S103 may be omitted in the processing procedure of the learning device 1. In addition, in the software configuration of the learning device 1, the permission range setting unit 111 and the status acquisition unit 112 may be omitted. Further, when monitoring within the permission range 309 is unnecessary, a series of processes from suspending the remote operation to resuming the remote operation may be omitted in the processing procedure of the learning device 1.

<4.5>
Further, the learning device 1 according to the above embodiment generates the ability granting data 123 in the above step S109. Then, the learning device 1 delivers the ability-imparting data 123 to the robot arm system 2, which is the learning target device, in step S110. However, the method of generating and distributing the ability-imparting data 123 does not have to be limited to such an example, and may be appropriately selected depending on the embodiment.

For example, the capacity-imparting data 123 may be generated by another information processing device or operator. In this case, step S109 may be omitted in the processing procedure of the learning device 1. In addition, in the software configuration of the learning device 1, the ability-imparting data generation unit 116 may be omitted.

Further, for example, the capacity imparting data 123 may be stored in a storage medium such as a CD drive, a DVD drive, or a flash memory. Then, the storage medium in which the ability-imparting data 123 is stored may be distributed to the requester. In this case, step S110 may be omitted in the processing procedure of the learning device 1. In addition, the distribution unit 117 may be omitted in the software configuration of the learning device 1.

When the ability-giving data 123 is distributed by the storage medium in this way, the client appropriately reads the ability-giving data 123 from the received storage medium and installs the read-out ability-giving data 123 in the RC20 of the robot arm system 2. To do. Thereby, the ability granting data 123 can be applied to the robot arm system 2.

<4.6>
Further, in the above embodiment, the learning device 1 is composed of one computer. However, the learning device 1 may be composed of a plurality of computers. In this case, some software modules of the learning device 1 may be mounted on each computer. For example, only the learning data collection unit 114 may be mounted on one computer. Then, when carrying out machine learning, a computer equipped with the learning data collection unit 114 may be lent to the client. Thereby, the real-time property of the collection process of the learning data 122 in step S104 can be enhanced.

<4.7>
Further, in step S101, the control unit 11 may further accept the designation of the password set for permitting remote control in the learning target device (robot arm system 2) as a learning request. In this case, in step S104, the control unit 11 may remotely control the robot arm system 2 after being authenticated by the robot arm system 2 by the designated password. This makes it possible to enhance security when remotely controlling the robot arm system 2.

<4.8>
Further, in step S101, the control unit 11 may further accept the designation of the time zone in which the remote control is permitted in the learning target device (robot arm system 2) as a learning request. In this case, the control unit 11 may execute the step S104 (remote control of the robot arm system 2) only in the designated time zone. Thereby, for example, the learning data 122 used for machine learning of the robot arm system 2 can be collected at nighttime or early morning when the robot arm system 2 is not used. Therefore, the utilization efficiency of the robot arm system 2 can be improved.

<4.9>
Further, in step S101, the control unit 11 may further accept the designation of the learning period for permitting remote control in the learning target device (robot arm system 2) as a learning request. In this case, the control unit 11 executes the above step S104 (remote control of the robot arm system 2) within the designated learning period, and uses it for the remote control of the robot arm system 2 after the lapse of the designated learning period. Information (eg, IP address) may be deleted.

<4.10>
Further, in the above embodiment, after receiving the learning request in step S101, the learning device 1 builds a neural network 6 in which the ability specified by the received learning request is acquired by machine learning in step S108. , Execute a series of processing. However, the processing mode of the learning request by the learning device 1 does not have to be limited to such an example. For example, the learning device 1 may be configured to be able to accept the discarding of the learning request.

FIG. 9 schematically illustrates an example of the software configuration of the learning device 1A according to this modification. As shown in FIG. 9, the learning device 1A according to this modified example has a discard reception unit 118 that accepts the discard of the learning request by executing the learning program 121 by the control unit 11, and when the learning request is received, the learning device 1A receives the learning request. It is configured as a computer further provided with a data deletion unit 119 that deletes information related to the learning request, including learning data collected before accepting the destruction of the learning request and information used for remote operation of the learning target device. The learning device 1A is configured in the same manner as the learning device 1 except for this point.

Next, an example of the processing procedure of the learning device 1A according to the present modification will be described with reference to FIG. FIG. 10 illustrates an example of the processing procedure related to the acceptance of discarding the learning request while the processes of steps S102 to S108 are being executed. After receiving the learning request in step S101, the control unit 11 of the learning device 1A starts the process of step S102 and starts the process of the next step S301.

(Step S301)
In step S301, the control unit 11 operates as the discard reception unit 118 and accepts the discard of the learning request. The requester who desires to discard the learning request operates the user terminal 4 to request the learning device 1 to discard the learning request made in step S101. If the discard of the learning request is received before the process of step S108 is started, the control unit 11 proceeds to the next step S302. On the other hand, if the discarding of the learning request is not accepted before the process of the step S108 is started, the control unit 11 omits the process of the next step S302 and ends the process related to the discarding of the learning request.

(Step S302)
In step S302, the control unit 11 operates as the data deletion unit 119, and the learning data 122 collected in step S105 and the information used for remote control of the robot arm system 2 (for example, the IP address) before accepting the discard of the learning request. ), And delete the information related to the learning request. The information regarding the learning request includes, for example, information indicating the content of the learning request specified in step S101, in addition to the information used for the remote control of the robot arm system 2. When the deletion of the information related to the learning request is completed, the control unit 11 ends the process related to the destruction of the learning request. According to this modification, it is possible to discard the machine learning request that is no longer needed, and thereby improve the efficiency of the resources of the learning device.

<4.11>
Further, in the above embodiment, the control unit 21 displays on the display 32 that the remote control has been completed by step S206. However, in the processing procedure of the robot arm system 2, the processing of the step S206 may be omitted.

Further, in the above embodiment, the control unit 21 notifies the peripheral device of the status of the robot arm system 2 by steps S204 and S207. However, in the processing procedure of the robot arm system 2, at least one of the steps S204 and S207 may be omitted. When both steps S204 and S207 are omitted, the notification unit 214 may be omitted in the software configuration of the robot arm system 2.

The processing order of step S203 and step S204 may be interchanged. Similarly, the processing order of step S206 and step S207 may be interchanged.

<4.12>
Further, in the above embodiment, the display 32 is used as a display unit for displaying the status of the robot arm system 2. However, the type of the display unit does not have to be limited to the display, and may be appropriately selected according to the embodiment. For example, as shown in FIG. 11, an indicator lamp may be used for the display unit.

FIG. 11 schematically illustrates the configuration of the robot arm system 2B according to this modification. In the robot arm system 2B, the RC 20 is connected to the indicator light 33 via the external interface 23. The indicator lamp 33 may be, for example, an LED (light emitting diode) lamp, a neon lamp, or the like.

In this case, in step S203, the control unit 21 may cause the indicator light 33 to indicate that the learning device 1 is being operated by remote control by causing the indicator light 33 to emit light in the first display mode. .. Then, in step S206, the indicator light 33 is caused to emit light in a second display mode different from the first display mode, so that the indicator light 33 indicates that the operation by remote control of the learning device 1 is completed. May be good.

The display mode is determined by factors that affect the viewer's vision, such as color and blinking speed. For example, in step S203, the control unit 21 may cause the indicator lamp 33 to emit red light as the first display mode. Then, in step S206, the control unit 21 may cause the indicator lamp 33 to emit blue light as a second display mode. As a result, it is possible to inexpensively configure a display unit that displays the status of the learning target device.

<4.13>
Further, in the above embodiment, the control unit 11 generates the learning data 122 by combining the sensor data, the target data, and the control data in step S105. Of these, the sensor data is an example of state data indicating the state of the learning target device. However, the type of state data does not have to be limited to sensor data, and may be appropriately selected depending on the embodiment. Further, when the state data is not required when controlling the operation of the learning target device, the state data may be omitted from the learning data. In the above embodiment, the control unit 11 may generate the learning data 122 by combining the target data and the control data in step S105.

1.1A ... Learning device,
11 ... Control unit, 12 ... Storage unit, 13 ... Communication interface,
14 ... input device, 15 ... output device, 16 ... drive,
110 ... Learning request reception department, 111 ... Permit range setting department,
112 ... Situation acquisition unit, 113 ... Remote control unit,
114 ... Learning data collection unit, 115 ... Learning processing unit,
116 ... Ability granting data generation unit, 117 ... Distribution unit,
118 ... Discard reception department, 119 ... Data deletion department,
121 ... learning program, 122 ... learning data,
123 ... Ability grant data,
2 ... Robot arm system,
20 ... RC,
21 ... Control unit, 22 ... Storage unit, 23 ... External interface,
24 ... Communication interface,
211 ... Remote control reception unit, 212 ... Operation processing unit,
213 ... Display control unit, 214 ... Notification unit,
221 ... Control program,
30 ... Robot arm,
301 ... base, 302 ... joints,
303 ... Link part, 304 ... End effector,
308 ... movable range, 309 ... permitted range,
31 ... camera, 32 ... display,
4 ... user terminal, 5 ... robot device,
6 ... Neural network,
61 ... Input layer, 62 ... Intermediate layer (hidden layer), 63 ... Output layer

Claims (7)

A target that is placed in a remote location with respect to a learning device that performs machine learning to acquire the specified ability in response to a learning request from the client, and that the specified ability is acquired by machine learning of the learning device. It is a learning target device
Display and
Control unit and
A communication unit configured to communicate with peripheral devices arranged around the learning target device, and
With
The control unit
A reception step that receives a remote control command from the learning device instructing the execution of an operation related to the specified ability, and a reception step.
An execution step that executes an operation related to the specified ability according to the received remote control command, and
A display step for displaying on the display unit that the learning device is being operated by the remote control while the operation is being executed in accordance with the remote control command.
A notification step in which the communication unit transmits a notification to the peripheral device to notify that the learning device is being operated by the remote control while the operation is being executed in accordance with the remote control command. The notification is a notification step comprising a movement restriction command instructing to restrict access to the vicinity of the learning target device.
A completion notification step in which the communication unit transmits a completion notification for notifying the peripheral device that the operation by the remote operation has been completed after the remote operation from the learning device is completed. The completion notification includes a completion notification step including a permission command for removing the movement restriction, and a completion notification.
Is configured to run,
Learning target device. The control unit is configured to further execute a completion display step of displaying on the display unit that the operation by the remote control has been completed after the remote control from the learning device is completed.
The learning target device according to claim 1. The display unit is a display.
In the display step, the control unit causes the display to display the content of the operation of the learning device by the remote control.
The device to be learned according to claim 1 or 2. The control unit
In the reception step, the command for instructing the execution of a plurality of operations is received, and the reception step receives the command.
In the display step, the content of the operation to be executed next to the operation being executed by the execution step is displayed on the display.
The learning target device according to claim 3. When the operation being executed is a dangerous operation, the control unit causes the display to indicate that the operation being executed is a dangerous operation in the display step.
The device to be learned according to claim 3 or 4. The display unit is an indicator light.
Before Symbol control section is,
In the display step, by causing the indicator light to emit light in the first display mode, the indicator light indicates that the learning device is being operated by the remote control.
In the completion display step, by causing the indicator light to emit light in a second display mode different from the first display mode, the indicator light indicates that the operation by the remote control has been completed.
The learning target device according to claim 2. A target that is placed in a remote location with respect to a learning device that performs machine learning to acquire the specified ability in response to a learning request from the client, and that the specified ability is acquired by machine learning of the learning device. A learning target device including a display unit and a communication unit configured to communicate with peripheral devices arranged around the learning target device.
A reception step that receives a remote control command from the learning device instructing the execution of an operation related to the specified ability, and a reception step.
An execution step that executes an operation related to the specified ability according to the received remote control command, and
A display step for displaying on the display unit that the learning device is being operated by the remote control while the operation is being executed in accordance with the remote control command.
A notification step in which the communication unit transmits a notification to the peripheral device to notify that the learning device is being operated by the remote control while the operation is being executed in accordance with the remote control command. The notification is a notification step comprising a movement restriction command instructing to restrict access to the vicinity of the learning target device.
A completion notification step in which the communication unit transmits a completion notification for notifying the peripheral device that the operation by the remote operation has been completed after the remote operation from the learning device is completed. The completion notification includes a completion notification step including a permission command for removing the movement restriction, and a completion notification.
To execute,
How it works.

Images