WO2022181458A1 - Information processing device, learning device, information processing system, and robot system - Google Patents

Abstract

This information processing device comprises a processing circuit. While a robot which operates according to a manipulation command is executing a task, the processing circuit executes a detection process for detecting operation-related information of the robot, an evaluation process for comparing the operation-related information with a reference so as to carry out evaluation, and a presentation process for causing a presentation device to present an evaluation result. The processing circuit executes the evaluation process with first timing at which each of the evaluation target operations of the robot to be evaluated occurs, and/or second timing at which each of the evaluation target stages of the task to be evaluated occurs. The processing circuit causes the presentation device to present evaluation results pertaining to the evaluation target operations in the order in which the evaluation target operations are executed. The processing circuit causes the presentation device to present evaluation results pertaining to the evaluation target stages in the order in which the evaluation target stages are executed.

Description

Information processing device, learning device, information processing system and robot system Cross-references to related applications

This application claims the priority and benefit of Japanese Patent Application No. 2021-30029 filed with the Japan Patent Office on February 26, 2021, and part of this application is incorporated by reference in its entirety. Quoted as an eggplant.

The present disclosure relates to information processing devices, learning devices, information processing systems, and robot systems.

Technology exists to evaluate robot operators. For example, the management system disclosed in Japanese Unexamined Patent Application Publication No. 2020-135362 makes an operator operating a robot feel as if an object at a remote location exists nearby, while allowing an operator to operate a robot near an object at a remote location. Provide an environment in which an operator operates a robot in A server provided in the management system acquires state information indicating the state of the robot or the state of the operator when the operator operates the robot, and compares the state indicated by the state information with the reference level to determine whether the robot is operated. determine a person's suitability.

In Japanese Patent Application Laid-Open No. 2020-135362, the skill level of robot operation is determined as aptitude. For example, the operator can only know the skill level of the entire operation for the work through the determined level. The operator may not be able to find a specific measure for improving the robot operation skill, and may not be able to increase the motivation for improving the skill and the willingness to operate.

The present disclosure provides an information processing device, a learning device, an information processing system, and a robot system that improve a user's willingness to operate a robot.

An information processing device according to an aspect of the present disclosure is an information processing device that evaluates an operation of the robot for causing the robot to perform a task, the processing circuit including a processing circuit, wherein the processing circuit receives an input from a user to an operation device. detection processing for detecting motion-related information, which is information related to the motion of the robot that operates in accordance with an operation command received; and a presentation process of presenting the evaluation result of the evaluation process on a presentation device that receives information presentation while the user operates the operation device, are executed during the execution of the work. The processing circuit is configured to perform a first timing each time an evaluation target motion is a motion of the robot to be evaluated, and a second timing each time an evaluation target stage is a work stage to be evaluated in the work. When the evaluation processing is executed at the first timing, the processing circuit executes the evaluation processing in the same order as the execution order of the evaluation target operation. When causing the presentation device to present the evaluation result regarding the action, and executing the evaluation process at the second timing, the processing circuit presents the evaluation result regarding the evaluation target stage in the same order as the execution order of the evaluation target stage. is presented on the presentation device.

FIG. 1 is a diagram illustrating an example of a configuration of a robot system according to an embodiment. FIG. 2 is a perspective view showing an example of the configuration of the robot according to the embodiment. FIG. 3 is a block diagram illustrating an example of the hardware configuration of the control device according to the embodiment; FIG. 4 is a block diagram illustrating an example of a functional configuration of the control device according to the embodiment; FIG. 5 is a flow chart showing an example of the operation of the robot system according to the embodiment. FIG. 6 is a plan view showing an example of the operation of the robot system according to the embodiment; FIG. 7 is a block diagram showing an example of functional configurations of an information processing device and a learning device according to a modification.

Exemplary embodiments of the present disclosure will be described below with reference to the drawings. All of the embodiments described below are comprehensive or specific examples. Among the constituent elements in the following embodiments, constituent elements that are not described in independent claims indicating the highest concept will be described as optional constituent elements. Each figure in the accompanying drawings is a schematic diagram and is not necessarily strictly illustrated. In each drawing, substantially the same components are denoted by the same reference numerals, and redundant description may be omitted or simplified. In the specification and claims, "device" may mean not only one device, but also a system of multiple devices.

[Configuration of robot system]
A configuration of a robot system 1 according to an exemplary embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of a configuration of a robot system 1 according to an embodiment. The robot system 1 includes one or more robots 100 , one or more operation terminals 200 and a server 300 . Although not limited, in this embodiment, the robot system 1 is configured to provide a service to a user P using a remotely operated robot 100 . The robot system 1 can be used in various service industries such as nursing care, medical care, cleaning, security, guidance, rescue, cooking, sales, rental, and article provision.

The robot 100 is a robot that can perform tasks such as providing services. As used herein and in the claims, "work" is defined as accomplishing, crafting, creating, or any combination of two or more of these, a productive thing in which some useful result is expected. A series of actions performed by the robot 100 can be targeted. For example, the “work” does not have to be a series of actions performed by the robot 100 during the process of teaching the robot 100 . For example, "work" may not cover a series of actions performed by robot 100 for play, games, entertainment, or a combination of two or more thereof.

Although not limited, in the present embodiment, two or more robots 100 are arranged in one service providing area AS where user P is provided with services. Furthermore, one or more operation terminals 200 are arranged in each of two or more operation areas AO located away from the service providing area AS.

The robot 100 is configured to be connected to the communication network N via wireless communication so as to enable data communication. The communication connecting the robot 100 and the communication network N may be wired communication or a combination of wired and wireless communication. The operation terminal 200 is configured to be connected to the communication network N for data communication via wired communication, wireless communication, or a combination thereof. One robot 100 and one operation terminal 200 can be connected via a communication network N so as to be capable of data communication. Any wired or wireless communication may be used.

The server 300 manages communication via the communication network N. Server 300 includes a computer device. The server 300 manages communication authentication, connection, disconnection, and the like between the robot 100 and the operation terminal 200 . For example, the server 300 stores identification information, security information, etc. of the robot 100 and the operation terminal 200 registered in the robot system 1, and uses the information to determine the qualification of connection of the operation terminal 200 to the robot system 1. Authenticate. The server 300 manages transmission and reception of data between the robot 100 and the operation terminal 200 , and the data may pass through the server 300 . Server 300 may be configured to convert data sent from a source into a data format that can be used by a destination. The server 300 may be configured to store and accumulate information, commands, data, etc. that are transmitted and received between the robot 100 and the operation terminal 200 during the operation of the robot 100 .

The communication network N is not particularly limited, and can include, for example, a local area network (LAN), a wide area network (WAN), the Internet, or a combination of two or more thereof. Communication network N includes short-range wireless communication such as Bluetooth (registered trademark) and ZigBee (registered trademark), network dedicated lines, communication carrier dedicated lines, public switched telephone network (PSTN), It may be configured to use mobile networks, internet networks, satellite communications, or a combination of two or more thereof. The mobile communication network may use a 4th generation mobile communication system, a 5th generation mobile communication system, or the like. Communication network N may include one or more networks. In this embodiment, the communication network N is the Internet.

[Configuration of operation terminal]
The configuration of the operation terminal 200 according to the embodiment will be described with reference to FIG. The operation terminal 200 is configured to receive commands, information, data, and the like input by the operator PO, and to output the received commands, information, data, and the like to other devices. The operation terminal 200 includes an operation input device 201 , a terminal computer 202 , a presentation device 203 and a communication device 204 . The four devices of the operation input device 201, the terminal computer 202, the presentation device 203, and the communication device 204 may be integrated to form one device, and each of the four devices independently forms a device and is connected to each other. or two or more devices may form one device and be connected to other devices.

The operation terminal 200 is an example of an operation device. The configuration of the operation terminal 200 is not particularly limited. It may be a teaching device, a known operating device for a robot, other operating devices, other terminal devices, devices using these devices, and improved devices thereof. The operation terminal 200 may be a dedicated device devised for the robot system 1, or may be a general-purpose device available on the general market. In this embodiment, a known general-purpose device is used as the operation terminal 200 . The device may be configured to implement the functions of the operation terminal 200 of the present disclosure by installing dedicated software.

The operation input device 201 is configured to be able to receive input from the operator PO, and to output to the terminal computer 202 signals indicating input commands, information, data, and the like. The configuration of the operation input device 201 is not particularly limited. may include a device that can be The operation input device 201 may include an imaging device that captures an image of the operator PO or the like, and a voice input device such as a microphone that receives voice input of the operator PO or the like. The operation input device 201 may be configured to output to the terminal computer 202 the captured image data and the signal indicating the input voice.

The terminal computer 202 processes commands, information, data, etc. received via the operation input device 201 and outputs them to other devices, and receives inputs of commands, information, data, etc. from other devices, and processes them. It is configured to execute instructions, process information and data, and the like.

The presentation device 203 is configured to present information to the operator PO. Although not limited, in this embodiment the presentation device 203 includes a display capable of displaying an image to the operator PO. The presentation device 203 displays an image of image data received from the terminal computer 202 . The presentation device 203 may include an audio output device such as a speaker capable of emitting audio to the operator PO. The presentation device 203 outputs the sound of the sound data received from the terminal computer 202 .

The communication device 204 has a communication interface that can be connected to the communication network N. The communication device 204 is connected to the terminal computer 202 and connects the terminal computer 202 and the communication network N so that data communication is possible. Communication devices 204 may include, for example, communication equipment such as modems, Optical Network Units (ONUs), routers, and mobile data communication equipment. Communication device 204 may include a computing device having computing capabilities and the like.

[Robot configuration]
A configuration of the robot 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a perspective view showing an example configuration of the robot 100 according to the embodiment. The robot 100 includes one carrier 110, one or more robot arms 120, one or more end effectors 130, a secondary battery module 141, a power supply circuit 142, a communication device 143, an imaging device 144, 145 and 146 , a sound collector 147 , a display device 148 , an audio output device 149 and a control device 170 . The control device 170 includes an information processing device 180 and a robot controller 190 . The information processing device 180 and the robot controller 190 may be separate devices or may be integrated. Although not limited, in this embodiment, the robot arm 120 is a robot arm that can also function for industrial purposes. The quantity of each component described above is not limited to the above quantity, and can be changed as appropriate.

The carrier 110 is configured to run on its own. Although not limited, in this embodiment, the transport vehicle 110 includes two drive wheels 111 , four auxiliary wheels 112 , and a transport drive device 113 that drives the two drive wheels 111 . The transport driving device 113 includes a servomotor as an electric actuator that is a driving source. Each servo motor is controlled by robot controller 190 . The transport driving device 113 rotates the two drive wheels 111 individually at various rotational directions and rotational speeds, thereby causing the transport vehicle 110 to travel at various speeds in the forward direction D1A, the backward direction D1B, and the turning direction. can.

Although not limited, in the present embodiment, as the robot arm 120, two robot arms 120A and 120B are mounted on the carrier 110 via the base 120C. Both of the robot arms 120A and 120B are rotatable about an axis S1 in the direction from the carrier 110 toward the base 120C, forming coaxial dual-arm robot arms. The robot arm 120A includes links 121A to 124A and arm drivers M1A to M4A that drive joints interconnecting the links 121A to 124A. The robot arm 120B includes links 121B to 124B and arm drivers M1B to M4B that drive the joints interconnecting the links 121B to 124B. For example, arm drives M1A to M4A and M1B to M4B are shown in FIG. Arm driving devices M1A to M4A and M1B to M4B include servo motors as electric actuators that are driving sources. Each servo motor is controlled by robot controller 190 . Links 121A and 121B are connected to base 120C via joints. Links 124 A and 124 B include mechanical interfaces that are connectable with end effector 130 .

The robot arms 120A and 120B as described above have a horizontal articulated arm configuration, but may have any configuration. For example, robotic arms 120A and 120B may be vertically articulated, polar, cylindrical, rectangular, or other types of robotic arms. The number of robot arms 120 arranged on the carrier 110 may be one or more. The rotation axes of the robot arms 120A and 120B may not be the same axis S1.

As the end effector 130, two end effectors 130A and 130B are detachably attached to the links 124A and 124B, respectively. End effectors 130A and 130B are configured to act on objects handled by robot 100 . End effectors 130A and 130B are operable and may include a drive. The driving device may be powered by electric power, gas pressure, hydraulic pressure, or the like. A drive device powered by electric power may include a servomotor as an electric actuator. The drives may be controlled by robot controller 190 .

The robot 100 further includes a device housing 140 on the carrier 110 . The secondary battery module 141 , the power supply circuit 142 , the communication device 143 , the information processing device 180 and the robot controller 190 are arranged inside the equipment housing 140 , but may be arranged at any position on the carrier 110 . A table 140a on which articles can be placed is arranged on the device housing 140. As shown in FIG.

The secondary battery module 141 functions as a power source for the robot 100. The secondary battery module 141 includes one or more secondary batteries. A secondary battery is a battery capable of charging and discharging power. Examples of secondary batteries include lead storage batteries, lithium ion secondary batteries, all-solid batteries, nickel-hydrogen storage batteries, nickel-cadmium storage batteries, and the like.

The power supply circuit 142 is a circuit that controls power supply and demand for the secondary battery module 141 . The power supply circuit 142 is configured to control supply and demand of electric power in accordance with commands from the information processing device 180 and the robot controller 190 . For example, the power supply circuit 142 may include devices such as converters, inverters, transformers and amplifiers. The power supply circuit 142 is connected to an external power supply such as a commercial power supply, supplies the power supplied from the external power supply to the secondary battery module 141 and stores the power, and transfers the power stored in the secondary battery module 141 to the power inside the robot 100 . to a component that consumes

The communication device 143 is a device for wireless communication, and is configured to connect to the communication network N via wireless communication. The wireless communication used by the communication device 143 is not particularly limited, but for example, mobile data communication, wireless LAN such as wireless Wi-Fi (Wireless Fidelity), Bluetooth (registered trademark) and ZigBee (registered trademark) It may be short-range wireless communication, wireless communication using a combination of two or more of these, or the like. The communication device 143 has a device compatible with wireless communication to be used.

The display device 148 includes a display capable of displaying images. The display may be configured to receive input from the user P, and may be a touch panel, for example. The display device 148 can display an image of image data sent from the information processing device 180 or the like. The display device 148 can display an image to the user P facing the robot 100 .

The sound collecting device 147 includes a microphone capable of acquiring sound from the surroundings and outputting an audio signal of the sound. The sound collector 147 is configured to output an audio signal to the information processing device 180 or the like. The information processing device 180 may be configured to convert a voice signal into voice data and transmit the data to the operation terminal 200, or may be configured to recognize voice from the voice signal. The sound collector 147 can function as an external sensor of the robot 100 capable of sensing the outside of the robot 100 . The internal world sensor of the robot 100 can sense the inside of the robot 100, and one example of the internal world sensor is a rotation sensor provided in the servo motors of the arm driving devices M1A to M4A and M1B to M4B.

The audio output device 149 includes a speaker capable of converting audio signals into sound waves and emitting them as sound. The audio output device 149 can output audio corresponding to audio signals sent from the information processing device 180 or the like. The voice output device 149 can output voice to the user P facing the robot 100 .

The imaging devices 144 , 145 and 146 each include a camera that captures digital images and are configured to send data of the captured images to the information processing device 180 . The information processing device 180 may be configured to process the image data captured by the imaging devices 144 , 145 and 146 into network transmittable data, and transmit the data to the operation terminal 200 via the communication network N. The information processing device 180 may be configured to perform image processing on the image data and use it for other processing. The camera may be a camera capable of capturing an image for detecting the three-dimensional position of the subject with respect to the camera, such as the distance to the subject. For example, the camera may be a stereo camera, a monocular camera, a TOF camera (Time-of-Flight-Camera), a pattern light projection camera such as fringe projection, a camera using a light section method, or a combination of two or more of these. etc. may be included. Imaging devices 144, 145 and 146 may function as external sensors.

The imaging device 144 is arranged at the tip of either or both of the robot arms 120A and 120B. Without limitation, in this embodiment the imaging device 144 is located on the end effector 130A of the robot arm 120A. The imaging device 145 is arranged on the display device 148 . The imaging device 145 can capture an image of the user P to whom the service is provided facing the robot 100 . Imaging device 145 may include the ability to change imaging direction, have a wide imaging field of view, or both, so that the end effectors 130A and 130B can image the object on which the end effectors 130A and 130B act. It may be configured as The imaging device 146 is arranged on the carrier 110 in the forward direction D1A.

The robot controller 190 is configured to control the motion of the robot arms 120A and 120B, the end effectors 130A and 130B, and the vehicle 110. For example, the robot controller 190 controls the operation of the arm drives M1A to M4A and M1B to M4B of the robot arms 120A and 120B, the drives of the end effectors 130A and 130B, the transport drive 113 of the transport vehicle 110, and the like. Configured. The information processing device 180 is connected to the robot controller 190 via wired communication, wireless communication, or a combination thereof, and performs arithmetic processing related to control of the robot arms 120A and 120B, the end effectors 130A and 130B, and the carrier 110. configured as

The information processing device 180 and the robot controller 190 include computer devices. The configuration of the information processing device 180 is not particularly limited. etc. The information processing device 180 is connected to the terminal computer 202 of the operation terminal 200 via the communication device 143 , the communication network N, and the communication device 204 so as to be capable of data communication. Although the configuration of the robot controller 190 is not particularly limited, the robot controller 190 may be, for example, an electronic circuit board, an electronic control unit, a microcomputer, other electronic equipment, and the like. The robot controller 190 may include a drive circuit for controlling the power supplied to the arm drives M1A to M4A and M1B to M4B, the drives of the end effectors 130A and 130B, the transport drive 113, and the like.

In the present embodiment, the information processing device 180 and the robot controller 190 control objects such as the robot arms 120A and 120B, the end effectors 130A and 130B, and the carrier 110 by manual operation control when causing the robot 100 to perform a task. It is configured to control the operation of elements, but is not so limited. The information processing device 180 and the robot controller 190 may be configured to control the operation of one or more of the control target elements by automatic operation control or a combination of automatic operation control and manual operation control. For example, the combination of automatic operation control and manual operation control may be configured such that automatic operation control and manual operation control are assigned to each control target element, and part of the work of robot 100 is performed by automatic operation control. , other parts of the work may be configured to be performed by manual operation control.

For example, the manual operation control may be a control that causes the elements to be controlled to operate in accordance with the details of the operation input by the operator PO to the operation input device 201 of the operation terminal 200 . For example, in manual operation control, the controlled element can perform an action according to the action of the operator PO who operates the operation input device 201 . The automatic operation control may be a control that autonomously operates the control target element according to the control program.

[Hardware configuration of control device]
A hardware configuration of the control device 170 according to the embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the hardware configuration of the control device 170 according to the embodiment. The information processing apparatus 180 includes a processor 1801, a memory 1802, a storage 1803, and input/output I/Fs (interfaces) 1804a to 1804f as components. Each component of the information processing device 180 is interconnected by a bus 1805, but may be connected by any other wired or wireless communication. The robot controller 190 includes a processor 1901, a memory 1902, an input/output I/F 1903, and a drive I/F 1904 as components. Robot controller 190 may include storage. Each component of robot controller 190 is interconnected by bus 1905, but may be connected by any other wired or wireless communication. The information processing device 180 and the robot controller 190 as described above include processing circuitry, but may include additional circuitry. Not all of the components included in each of the information processing device 180 and the robot controller 190 are essential.

The set of processor 1801 and memory 1802 and the set of processor 1901 and memory 1902 each form a processing circuit. The processing circuitry sends and receives commands, information, data, etc. to and from other devices. The processing circuit inputs signals from various devices and outputs control signals to the elements to be controlled.

Memories 1802 and 1902 store programs executed by the processors 1801 and 1901, various data, and the like. Memories 1802 and 1902 may include storage devices such as semiconductor memory such as volatile and non-volatile memory. Although not so limited, in this embodiment, memories 1802 and 1902 include volatile memory, RAM (Random Access Memory), and non-volatile memory, ROM (Read-Only Memory).

The storage 1803 stores various data. The storage 1803 may include storage devices such as semiconductor memory, hard disk drives (HDDs), solid state drives (SSDs), or combinations of two or more thereof.

Both processors 1801 and 1901 form a computer system together with RAM and ROM. The computer system of the information processing device 180 may realize the functions of the information processing device 180 by the processor 1801 using the RAM as a work area and executing the program recorded in the ROM. The computer system of the robot controller 190 may realize the functions of the robot controller 190 by the processor 1901 using the RAM as a work area and executing the program recorded in the ROM.

Some or all of the functions of the information processing device 180 and the robot controller 190 may be implemented by the computer system described above, or may be implemented by dedicated hardware circuits such as electronic circuits or integrated circuits. It may be implemented by a combination of hardware circuits. Each of the information processing device 180 and the robot controller 190 may be configured to execute each process by centralized control by a single device, or may be configured to execute each process by distributed control by cooperation of a plurality of devices. may be The information processing device 180 and the robot controller 190 may be configured to include at least some of the functions of each other, or may be integrated.

For example, but not limited to, processors 1801 and 1901 include CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), microprocessor, processor core, Including multiprocessors, ASICs (Application-Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), etc., logic circuits or dedicated circuits formed on IC (Integrated Circuit) chips, LSIs (Large Scale Integration), etc. Each processing may be realized. A plurality of processes may be implemented by one or more integrated circuits, or may be implemented by one integrated circuit.

The first input/output I/F 1804a of the information processing device 180 connects the information processing device 180 and the robot controller 190, enabling input/output of information, commands, data, etc. between them. The second input/output I/F 1804b connects the information processing device 180 and the communication device 143 to enable input/output of information, commands, data, and the like therebetween. The third input/output I/F 1804c connects the information processing device 180 and the imaging devices 144 to 146, enabling input/output of information, commands, data, and the like therebetween. The fourth input/output I/F 1804d connects the information processing device 180 and the sound collector 147, and enables input/output of information, commands, data, and the like therebetween. The fifth input/output I/F 1804e connects the information processing device 180 and the display device 148 to enable input/output of information, commands, data, and the like therebetween. The sixth input/output I/F 1804f connects the information processing device 180 and the audio output device 149 and enables input/output of information, commands, data, and the like therebetween.

The input/output I/F 1903 of the robot controller 190 connects the robot controller 190 and the first input/output I/F 1804a of the information processing device 180, enabling input/output of information, commands, data, etc. between them. . The drive I/F 1904 connects the robot controller 190 and the drive circuit 142a, enabling transmission and reception of signals and the like between them. The driving circuit 142a supplies electric power to the arm driving devices M1A to M4A and M1B to M4B, the driving devices of the end effectors 130A and 130B, and the transport driving device 113 according to the command values included in the signals received from the robot controller 190. configured to control. For example, the driving circuit 142a can cause each driving device to cooperate with each other to drive. Although not limited, the drive circuit 142a is formed as part of the power supply circuit 142 in this embodiment.

The robot controller 190 may be configured to servo-control the servo motor of each driving device. The robot controller 190 receives from the drive circuit 142a, as feedback information, the detected value of the rotation sensor provided in the servomotor and the command value of the current from the drive circuit 142a to the servomotor. The robot controller 190 determines command values for driving the servomotors using the feedback information, and transmits the command values to the drive circuit 142a.

Since a general-purpose device can be used as the operation terminal 200, detailed description of the hardware configuration of the operation terminal 200 is omitted. The terminal computer 202 of the operation terminal 200 includes a processor and memory, like the information processing device 180 and the like. The terminal computer 202 has an input/output I for establishing connection between the terminal computer 202 and the operation input device 201, connection between the terminal computer 202 and the communication device 204, and connection between the terminal computer 202 and the presentation device 203. /F may also be included.

[Functional Configuration of Control Device]
A functional configuration of the control device 170 according to the embodiment will be described with reference to FIG. FIG. 4 is a block diagram showing an example of the functional configuration of the control device 170 according to the embodiment. The information processing device 180 includes an operation command unit 180a, a detection processing unit 180b, an evaluation processing unit 180c, a consideration determination unit 180d, a presentation processing unit 180e, an accumulation processing unit 180f, a reference determination unit 180g, and a timer unit. 180h and storage units 181a and 181b as functional components. The functions of the functional components other than the storage units 181a and 181b are implemented by the processor 1801 or the like, and the functions of the storage units 181a and 181b are implemented by storage devices such as memory 1802, storage 1803, or a combination thereof. The functions of the storage units 181a and 181b may be implemented by separate storage devices or may be implemented by one storage device. Not all of the above functional building blocks are essential. The robot controller 190 includes drive command units 190a to 190e as functional components. Functions of the drive command units 190a to 190e are implemented by the processor 1901 and the like. Not all of the above functional building blocks are essential.

The first storage unit 181a stores motion-related reference data for various tasks. For example, motion-related reference data is denoted as "reference data" in FIG. The action-related reference data includes information on action-related references, which are references related to actions of the robot 100 . The motion-related criteria include criteria for motion-related information that is information related to the motion of the robot 100 . The motion-related information may include motion-related element information such as the motion of the robot 100, the state of the surrounding environment of the robot 100, the state of the object that the robot 100 acts on, and the state of the surrounding environment of the object.

For example, the action-related standards are standards that express the minimum required level of action-related elements, the standard level of action-related elements, the excellent level of action-related elements, and the ideal level of action-related elements. good too. Action-related criteria may include criteria for features to be evaluated, which are features for evaluating action-related elements.

For example, one task includes a series of multiple motions of the robot 100. The motion-related criteria for motion of the robot 100 may include a series of multiple motion criteria for each of the robot arms 120A and 120B, the end effectors 130A and 130B, and the vehicle 110 that form the task of interest. For example, the evaluation target features of the action-related criteria of the action of the robot 100 are the time required for the action, the order of the action, the position of the robot 100 and its parts in the action, and the posture of the robot 100 and its parts in the action. , the trajectory of the robot 100 and its parts in the motion, the velocity of the position in the motion, the velocity of the posture in the motion, the acceleration of the position in the motion, the acceleration of the posture in the motion, and , the work performance of the robot 100 in the motion, and the like.

The state of the surrounding environment of the robot 100 may include the state of robot peripheral elements, which are components such as devices, equipment, and equipment that form the surrounding environment for the robot 100 to work. Examples of robot peripheral elements include peripheral elements such as devices, facilities and equipment arranged in the work area of the robot 100 such as the service area AS, peripheral elements such as devices, facilities and equipment cooperating with the robot 100, and , and peripheral elements such as sensors, such as imaging devices, placed in the work area. The features to be evaluated for the motion-related criteria of the robot peripheral elements are the force and impact exerted by the robot 100 on the robot peripheral elements, changes in the position, posture, shape and state of the robot 100 exerted on the robot peripheral elements, and the robot relative to the robot peripheral elements. Features such as position, pose, position velocity, pose velocity, position acceleration and pose acceleration of 100 and its parts may be included. For example, in FIG. 1, the robot peripheral elements of the robot 100 are other robots 100, storage racks SR in which articles W, which are objects, are stored, and users other than the user P to whom the robot 100 provides services. .

The state of the object may include the state of the object handled by the robot 100 using the end effectors 130A and 103B. The features to be evaluated for the motion-related criteria of the object are the position, pose, position velocity, pose velocity, position acceleration and pose acceleration of the object, changes in the shape of the object, changes in the state of the object, and , the workmanship of the object, and other features. For example, in FIG. 1, the target object is an article W. FIG.

The state of the surrounding environment of the object may include the state of the elements surrounding the object, which are constituent elements such as devices, equipment, and equipment that handle the object other than the robot 100 . Examples of object peripheral elements include peripheral elements such as devices, facilities and equipment that support, hold, accommodate, load, mount and transfer objects, peripheral elements such as equipment, facilities and equipment to which objects are assembled, and , peripheral elements such as devices, equipment, and equipment that apply processing to objects. The features to be evaluated for the motion-related criteria of the object peripheral elements are the force and impact exerted by the robot 100 on the object peripheral elements, the position, posture, shape and state changes exerted by the robot 100 on the object peripheral elements, and the object Features such as the position, posture, position velocity, posture velocity, position acceleration, and posture acceleration of the robot 100 and its parts with respect to peripheral elements may be included. For example, in FIG. 1, the object peripheral elements are the shelf board of the storage shelf SR, other articles in the storage shelf SR, the user P, and the like.

The second storage unit 181b stores accumulated data, which is data accumulated by the accumulation processing unit 180f. The accumulated data includes information on the evaluation result determined by the evaluation processing unit 180c and information on the action-related criteria used to determine the evaluation result in association with each other. That is, in the accumulated data, action-related information, action-related reference information for the action-related information, and evaluation result information based on the action-related reference are associated. The second storage unit 181b is an example of a functional unit realized by the first storage device.

During manual operation control, the motion command unit 180a generates motion commands for causing the robot arms 120A and 120B, the end effectors 130A and 130B, and the transport vehicle 110 to operate according to the operation commands received from the operation terminal 200, and the robot controller 190 output. During automatic operation control, the motion command unit 180a generates motion commands for causing the robot arms 120A and 120B, the end effectors 130A and 130B, and the transport vehicle 110 to move according to the control program, and outputs them to the robot controller 190.

The operation command is a command indicating the content of the operation input to the operation input device 201 by the operator PO. The operation command includes a command for operating the robot 100 under manual operation control according to the content of the operation. The terminal computer 202 receives a signal indicating the content of the operation from the operation input device 201 , converts the signal into an operation command, and transmits the operation command to the information processing device 180 .

The motion command unit 180a issues a first motion command to the robot arm 120A, a second motion command to the robot arm 120B, a third motion command to the end effector 130A, a fourth motion command to the end effector 130B, and a transport vehicle. Generate a fifth motion command to 110 . For example, a third motion command and a fourth motion command, respectively, may be generated when end effectors 130A and 130B include drives. The first motion command and the second motion command may include target position commands and target force commands generated by each portion of the robot arms 120A and 120B. The third and fourth motion commands may include target state commands for the end effectors 130A and 130B. The fifth motion command may include a target position command for the transport vehicle 110 .

It should be noted that the target position command may include commands such as target position, position target movement speed, target attitude, and attitude target movement speed. The target force command may include commands such as target force magnitude and direction. The target force command may include a target acceleration command.

The timing unit 180h measures the elapsed time and outputs the timing result to the detection processing unit 180b, the evaluation processing unit 180c, the consideration determination unit 180d, the presentation processing unit 180e, and the like. The timing result can be used for processing in the detection processing unit 180b, the evaluation processing unit 180c, the consideration determination unit 180d, the presentation processing unit 180e, and the like. For example, the function of the clock unit 180h may be realized by a clock or the like mounted on the computer device of the information processing device 180. FIG.

The detection processing unit 180b executes detection processing for detecting motion-related information of the robot 100 that operates according to the operation command while the robot 100 is performing work. The detection processing unit 180b outputs the detection result to the evaluation processing unit 180c. In this embodiment, the detection processing unit 180b is configured to output the detection result to the evaluation processing unit 180c without setting a delay time after detecting the action-related information. may be provided.

The detection processing unit 180b detects information of action-related elements as action-related information during execution of work. The detection processing unit 180b may be configured to detect information of motion-related elements in any manner.

For example, the detection processing unit 180b detects the motion of the robot 100, specifically, the motions of the robot arms 120A and 120B, the end effectors 130A and 130B, and the carrier 110 as motion-related elements. For example, the detection processing unit 180b may be configured to process an operation command and detect a target motion of the robot 100 indicated by the operation command as a motion-related element. It may be configured to detect motion as a motion-related element. The detection processing unit 180b may be configured to acquire drive commands from the drive command units 190a to 190e of the robot controller 190, process them, and detect target motions of the robot 100 indicated by the drive commands as motion-related elements. The detection processing unit 180b may be configured to acquire feedback information from the drive command units 190a to 190e of the robot controller 190, process it, and detect the motion of the robot 100 indicated by the feedback information as a motion-related element. The detection processing unit 180b acquires captured image data from the imaging devices 144, 145 and 146 and an imaging device external to the robot 100, and processes the captured image data. It may be configured to detect as

For example, the detection processing unit 180b detects the state of robot peripheral elements forming the peripheral environment of the robot 100 as motion-related elements. For example, the detection processing unit 180b detects the state of the robot peripheral elements using the detection results received from the external sensor of the robot 100 and the information received from the robot peripheral elements. Examples of the external sensors include force sensors arranged between the end effectors 130A and 103B and the robot arms 120A and 120B, the imaging devices 144 to 146, the sound collector 147, and the service area AS. It is a sensor such as an imaging device. Examples of information received from the robot peripheral elements are the state, vibration, impact, position, attitude, position velocity, attitude velocity, etc. of the robot peripheral elements.

For example, the detection processing unit 180b detects, as an action-related element, the state of an action object, which is an object to which the robot 100 acts. For example, the detection processing unit 180b detects the state of the action object using the detection result received from the external sensor of the robot 100 and the information received from the action object. Examples of the external sensors include the force sensors of the robot arms 120A and 120B, the imaging devices 144 to 146, the sound collector 147, and sensors arranged in the service providing area AS. Examples of information received from the action object are the detection results of sensors provided by the action object, and the state, vibration, impact, position, attitude, position velocity, attitude velocity, position acceleration, and attitude of the action object. acceleration and the like.

For example, the detection processing unit 180b detects the state of an object peripheral element that forms the peripheral environment of the action object as an action-related element. For example, the detection processing unit 180b detects the state of the object peripheral element using the detection result received from the external sensor of the robot 100 and the information received from the object peripheral element. Examples of the external sensors include the force sensors of the robot arms 120A and 120B, the imaging devices 144 to 146, the sound collector 147, and sensors arranged in the service providing area AS. Examples of information received from the object surrounding elements are the state, vibration, impact, position, orientation, positional velocity, and orientational velocity of the object surrounding elements.

The detection processing unit 180b may be configured to detect motion-related elements at any timing. For example, the detection processing unit 180b detects motion-related elements at predetermined timings, such as timings at predetermined time intervals, predetermined timings for executing predetermined motions of the robot 100, and predetermined timings for predetermined work steps. may be configured to detect Although not limited, in the present embodiment, the detection processing unit 180b executes detection processing at predetermined time intervals. The detection processing unit 180b is an example of a functional unit implemented by a processing circuit.

Hereinafter, the motion of the robot 100 to be processed by the functional components of the information processing device 180 other than the detection processing unit 180b will be referred to as "processed motion". For example, a predetermined motion of the robot 100 is an example of a motion to be processed. The unit of operation to be processed may be set arbitrarily. For example, the delimitation of the motion to be processed in one unit may be a temporal delimitation such as the execution period of the motion, a delimitation of motion change such as the timing at which the motion changes, a designated delimitation, other delimitation, or two of these. It may be set by a combination of the above.

For example, when the robot 100 executes a series of operations of gripping an object placed at a point A with the end effector 130A and placing it on the table 140a, one unit of processing target operation is a series of operations to be executed. may be set to include all of, or may be set to include a part of the series of operations. For example, one unit of processing target motion includes a moving motion of the robot arm 120A that moves the end effector 130A to the target object, a gripping motion of the end effector 130A that grips the target object, and a moving motion of the end effector 130A onto the table 140a to move the target object. and one or more of the four motions of the movement motion of the robot arm 120A for placing the object and the grip releasing motion of the end effector 130A releasing the grip of the object. For example, one target action may be set for the four actions, or two or more target actions may be set.

A work stage can be set by dividing a series of actions included in the work. The unit of work steps may be set arbitrarily. For example, a division of one unit of work stage may be a temporal division such as a work execution period, a movement change division such as a timing at which the movement included in the work is switched, a designated division, other divisions, or any of these divisions. It may be set by a combination of two or more. For example, when the robot 100 receives a command requesting an object from a user P, carries an object placed at a point A, and delivers it to the user P, one unit of work stage may be set to include a portion of all operations performed in

For example, one unit work stage includes a process in which the robot 100 moves from a position near the user P to a position near the point A, and the robot 100 grips an object at the point A with the end effector 130A and places it on the robot 100. moving the robot 100 from a position near the point A to a position near the user P; and handing the object on the robot 100 to the user P using the end effector 130A. may be set to include one or more steps of For example, one work stage may be set for the four processes, or two or more work stages may be set.

The evaluation processing unit 180c performs evaluation processing for comparing and evaluating the motion-related information of the robot 100 detected by the detection processing unit 180b with the motion-related criteria of the motion-related information while the robot 100 is performing a task. Specifically, the evaluation processing unit 180c compares the information of the action-related element included in the action-related information with the action-related criteria of the action-related element, determines the evaluation result of the action-related element, and determines the evaluation result of the action-related element. An evaluation result of the action-related information is determined based on the evaluation result of the element.

The action-related information to be evaluated by the evaluation processing unit 180c is action-related information corresponding to the process-target action to be evaluated among the process-target actions included in the work. Hereinafter, the processing target motion to be evaluated is referred to as an "evaluation target motion". The evaluation target action is one or more of all the process target actions included in the work. All of the process target actions may be the evaluation target actions, or part of all the process target actions may be the evaluation target actions. Two or more evaluation target actions may be continuous with each other or may be discontinuous with each other.

For example, the evaluation processing unit 180c may determine the deviation amount, deviation direction and An evaluation result of either or both of the action-related element and the evaluation target feature may be determined based on the deviation frequency or the like. Furthermore, the evaluation processing unit 180c may determine the evaluation result of the action-related information based on the evaluation results of either or both of the action-related element and the evaluation target feature.

The evaluation processing unit 180c executes the evaluation process at either or both timings of each evaluation target motion and each evaluation target stage, which is a work stage to be evaluated in the work. All the work stages included in the work may be the evaluation target stages, or a part of all the work stages may be the evaluation target stages. Furthermore, the action-related information corresponding to all the processing target actions included in the evaluation target stage may be the evaluation target for the evaluation of the evaluation target stage. Relevant information may be subject to evaluation for evaluation in the subject to evaluation stage.

In the evaluation process at each timing of the evaluation target motion, the evaluation processing unit 180c reads the motion-related reference data of the evaluation target motion from the first storage unit 181a, and stores the motion-related reference data and the evaluation target motion corresponding to the motion-related reference data. It compares with the action-related information and determines an evaluation result of the action-related information. For example, the evaluation processing unit 180c may determine the evaluation level of action-related information. For example, the evaluation processing unit 180c compares the evaluation target feature criteria included in the action-related reference data with the evaluation target feature of the action-related element of the action-related information, and determines the evaluation level of the evaluation target feature. good too. The evaluation processing unit 180c may determine one or both of the evaluation level of the action-related element and the evaluation level of the action-related information based on the evaluation level of the evaluation target feature.

The evaluation processing unit 180c, for example, executes the evaluation processing of the action-related information corresponding to the first evaluation target action by the timing of the second evaluation target action, which is the next evaluation target action after the first evaluation target action. can. The evaluation processing unit 180c can execute and complete the evaluation processing of the action-related information corresponding to the first evaluation target action by the timing of the second evaluation target action. For example, the timing of the second evaluation target motion is the reference timing of the second evaluation target motion included in the motion-related reference data, the detection timing of the motion-related information corresponding to the second evaluation target motion, or any of these. It may be the timing of combination or the like. The reference timing and detection timing may be an operation start timing, an operation completion timing, or a predetermined timing therebetween.

The evaluation processing unit 180c may be configured to execute the evaluation process of the action-related information corresponding to the first evaluation target action by the timing of the next process target action after the first evaluation target action. For example, the timing of the next processing target motion may be the completion timing or a predetermined timing between the start timing and the completion timing of the actual motion of the robot 100 corresponding to the processing target motion. Although not limited, in the present embodiment, the evaluation processing unit 180c performs the evaluation processing without setting a delay time after receiving the detection result of the motion-related information corresponding to the first evaluation target motion from the detection processing unit 180b. Start.

In the evaluation process at each timing of the evaluation target stage, the evaluation processing unit 180c retrieves the action-related reference data of the process target action to be evaluated among the process target actions included in the evaluation target stage from the first storage unit 181a. The action-related reference data is read out, and the action-related information corresponding to the process target action to be evaluated is compared with the action-related reference data to determine the evaluation result of the action-related information. The evaluation processing unit 180c integrates the evaluation results of the motion-related information corresponding to the processing target motions of all the evaluation targets, and determines the evaluation result of the evaluation target stage based on the integration result. For example, the evaluation processing unit 180c may determine the evaluation level of the evaluation target stage. Any integration method may be used. For example, the integration method may be a method of adding evaluation results of a plurality of action-related information, a method of weighting and adding evaluation results of a plurality of action-related information, or a method of adding evaluation results of a plurality of action-related information. A method of integrating the evaluation results by other statistical processing may be used.

The evaluation processing unit 180c can, for example, execute the evaluation process of the first work stage by the timing of the second evaluation stage, which is the next evaluation stage after the first evaluation stage. The evaluation processing unit 180c can execute and complete the evaluation processing of action-related information corresponding to the process target actions of all evaluation targets included in the first evaluation target stage by the timing of the second evaluation target stage. For example, the timing of the second work stage is the reference timing of the second evaluation target stage included in the motion-related reference data, the detection timing of the second evaluation target stage, or a combination of these timings. good too. The reference timing and the detection timing may be the start timing of the first target operation in the second evaluation stage, the completion timing of the last target operation, or a predetermined timing therebetween.

The evaluation processing unit 180c may be configured to execute the evaluation process of the first evaluation target stage by the timing of the work stage following the first evaluation target stage. For example, the timing of the next work step may be the start timing of the first target operation of the work step, the completion timing of the last target operation, or a predetermined timing between these. Although not limited, in the present embodiment, the evaluation processing unit 180c collects action-related information corresponding to the last evaluation target processing target action among all evaluation target processing target actions included in the first evaluation target stage. is received from the detection processing unit 180b, the evaluation processing is started without providing a delay time. The evaluation processing unit 180c is an example of a functional unit implemented by a processing circuit.

The compensation determination unit 180d determines compensation to be given to the operator PO for the operation of the robot 100 based on the evaluation result of the evaluation processing unit 180c. The price determining unit 180d determines a price corresponding to the evaluation result of the action-related information each time the action-related information corresponding to the action to be evaluated is evaluated. The compensation determination unit 180d determines a compensation corresponding to the evaluation result of the evaluation target stage each time the evaluation process of the evaluation target stage is performed. For example, the consideration may be a reward amount, points, score, score, etc. given to the operator PO. The reward amount may be a reward amount in real currency or may be a reward amount in virtual currency.

The compensation determination unit 180d sequentially accumulates the individual compensations that have been determined during the execution of the work, and calculates the cumulative result that is the result of accumulating the compensations. Each time the price determining unit 180d determines a price, it outputs information on the price that has been decided and information on the accumulation result including the price that has been decided to the presentation processing part 180e. The consideration determining unit 180d may be configured to separately accumulate the consideration for the action-related information corresponding to the action to be evaluated and the consideration for the stage to be evaluated, thereby calculating two accumulation results. It may be configured to integrate and accumulate to calculate one integrated result.

The price determination unit 180d may be configured to add weight to the evaluation level of the action-related information corresponding to the action to be evaluated and determine the price for the action-related information. The price determining unit 180d may be configured to determine the price of the action-related information by adding weights to the evaluation levels of the evaluation target features of the action-related information and integrating the weighted evaluation levels. It may be configured to further weight the determined consideration for the action-related information. For example, the difficulty level of the operation differs depending on the motion to be evaluated. The consideration determination unit 180d may be configured to change the weighting according to the motion to be evaluated. The price determining unit 180d may be configured to change the weighting according to the feature to be evaluated.

The price determining unit 180d may be configured to determine the price by weighting the evaluation level of the evaluation target stage. The consideration determining unit 180d adds weights to the evaluation levels of the action-related information corresponding to the processing target action of the evaluation target included in the evaluation target stage, and integrates the evaluation levels after the weighting, thereby A consideration may be determined, and it may be configured to further weight the determined consideration for the evaluation target stage. The consideration determining unit 180d may be configured to change the weighting according to the evaluation target stage and the processing target action. Any method may be used as the integration method used by the consideration determining unit 180d. The consideration determining unit 180d is an example of a functional unit implemented by a processing circuit.

The presentation processing unit 180e executes presentation processing for causing the presentation device 203, which receives information presentation while the operator PO operates the operation input device 201, to present the evaluation result of the evaluation processing unit 180c during the execution of the work. do. Although not limited, in the present embodiment, the presentation processing unit 180e causes the presentation device 203 to present the price determined by the price determination unit 180d and the accumulated result of the price during execution of the work. The presentation processing unit 180 e generates data including the price received from the price determination unit 180 d and the accumulated result, and transmits the data and a presentation command for the data to the operation terminal 200 . For example, the presentation processing unit 180e may generate data including image data, audio data, character string data, or a combination of two or more of these, and transmit the generated data to the operation terminal 200, which indicates the price and the accumulated result.

When the evaluation processing is executed by the evaluation processing unit 180c at each timing of the evaluation target motion, the presentation processing unit 180e displays the evaluation result of the motion-related information corresponding to the evaluation target motion in the same order as the execution order of the evaluation target motion. is transmitted to the operation terminal 200 and presented by the presentation device 203 . Evaluation results may include either or both of an evaluation level and consideration. For example, the presentation processing unit 180e can execute the presentation processing of the evaluation result of the action-related information corresponding to the first evaluation target action by the timing of the next second evaluation target action. The timing of the second evaluation target action may be any one of the timings exemplified for the timing of the second evaluation target action of the evaluation processing unit 180c. They may be the same or different.

When the evaluation processing is executed by the evaluation processing unit 180c at each timing of the evaluation target stage, the presentation processing unit 180e transmits the evaluation result of the evaluation target stage to the operation terminal 200 in the same order as the execution order of the evaluation target stage. and causes the presentation device 203 to present it. Evaluation results may include either or both of an evaluation level and consideration. For example, the presentation processing unit 180e can execute the presentation processing of the evaluation result of the first evaluation target stage by the timing of the next second evaluation target stage. The timing of the second evaluation target stage may be any of the timings exemplified for the timing of the second evaluation work stage of the evaluation processing unit 180c, and may be the timing of the second evaluation work stage of the evaluation processing unit 180c. They may be the same or different. The presentation processing unit 180e is an example of a functional unit implemented by a processing circuit.

The accumulation processing unit 180f associates the action-related reference data, the evaluation result information of the action-related information based on the action-related reference data, and the execution date and time of the action of the robot 100 corresponding to the action-related information, and stores them as accumulated data. It is stored in the second storage unit 181b. The accumulation processing unit 180f causes the second storage unit 181b to store the accumulated data regarding the motion to be evaluated and the accumulated data regarding the stage to be evaluated.

For example, the accumulation processing unit 180f stores information on the evaluation target motion, information on the evaluation result of the motion-related information corresponding to the evaluation target motion, and information on the evaluation result of the motion-related information corresponding to the evaluation target motion as the accumulated data on the evaluation target motion. It may be stored in the second storage unit 181b in association with the motion-related reference data. The evaluation result information may include either or both of the evaluation level and the consideration.

For example, the accumulation processing unit 180f stores, as accumulated data related to the evaluation target stage, information on the evaluation target stage, information on the processing target action of the evaluation target included in the evaluation target stage, and action-related information corresponding to the processing target action. 2 or more of the evaluation result information, the evaluation result information of the evaluation target stage, and the action-related reference data used in the evaluation process of the evaluation result are associated and stored in the second storage unit 181b. You may let The evaluation result information may include either or both of the evaluation level and the consideration.

The reference determination unit 180g uses the accumulated data in the second storage unit 181b to determine new action-related reference data, and stores the action-related reference data in the first storage unit 181a. The reference determination unit 180g generates new second action-related reference data using the first action-related reference data included in the accumulated data of the second storage unit 181b and the evaluation result based on the first action-related reference data. to decide. An evaluation result based on the first action-related reference data is an evaluation result determined using the first action-related reference data.

For example, the reference determination unit 180g can accept specified conditions for determining the second action-related reference data. For example, the reference determination unit 180g receives a specified condition from the operation terminal 200. FIG. For example, the specified condition may be a condition for limiting the first action-related reference data and evaluation results used to determine the second action-related reference data. The reference determination unit 180g selects, from the data stored in the second storage unit 181b, the information of the evaluation result that satisfies the specified condition, and the first action-related reference data that is the action-related reference data used for the evaluation result. Second action-related reference data may be determined using the evaluation result information and the first action-related reference data.

For example, the specified condition may be a condition that limits the attributes of the evaluation results for which the first action-related reference data is used. The attributes of the evaluation results are the users targeted by the evaluation results, the robots targeted by the evaluation results, the tasks targeted by the evaluation results, the workplaces targeted by the evaluation results, the work environments targeted by the evaluation results, and the evaluation results. It may be the work execution date targeted by the evaluation result, and the work execution period targeted by the evaluation result. The specified condition may be a condition that limits one or more evaluation result attributes.

For example, the reference determination unit 180g can use the evaluation result of a specific user and the first action-related reference data to decide the second action-related reference data corresponding to the user. For example, if the user's evaluation level is low, the reference determination unit 180g may generate second action-related reference data with a lower reference than the first action-related reference data. When the user operates the robot 100 , the user can command the information processing device 180 via the operation terminal 200 to request evaluation using the second motion-related reference data. As a result, the user can obtain a higher evaluation from the information processing device 180, and can be motivated to operate the robot. The reference determination unit 180g is an example of a functional unit implemented by a processing circuit.

The first drive command unit 190a generates a drive command for driving the joints of the arm drive devices M1A to M4A of the robot arm 120A up to the state of the target position command and the target force command included in the first motion command. and output to the drive circuit 142a. The drive command includes a current command value for the servomotor. The first drive command section 190a generates a drive command using feedback information of the servomotor.

The second drive command unit 190b generates a drive command for driving the joints of the arm drive devices M1B to M4B of the robot arm 120B up to the state of the target position command and the target force command included in the second motion command. and output to the drive circuit 142a. The drive command includes a current command value for the servomotor. The second drive command section 190b generates a drive command using feedback information of the servomotor.

The third drive command section 190c generates a drive command for driving the drive device of the end effector 130A to the target state included in the third motion command, and outputs it to the drive circuit 142a.

The fourth drive command section 190d generates a drive command for driving the drive device of the end effector 130B to the target state included in the fourth operation command, and outputs it to the drive circuit 142a.

The fifth drive command unit 190e generates a drive command for causing the carrier drive device 113 of the carrier 110 to drive the drive wheels 111 to the state of the target position command included in the fifth operation command. output to The drive command includes a current command value for the servomotor. The fifth drive command section 190e generates a drive command using feedback information of the servomotor.

[Operation of the robot system]
An example of the operation of the robot system 1 according to the embodiment will be described with reference to FIGS. 5 and 6. FIG. FIG. 5 is a flow chart showing an example of the operation of the robot system 1 according to the embodiment. FIG. 6 is a plan view showing an example of the operation of the robot system 1 according to the embodiment. An example in which the robot 100 searches for an article W requested by the user P from the storage shelf SR and hands it over to the user P will be described below. In the examples of FIGS. 5 and 6, the information processing apparatus 180 executes the evaluation process and the presentation process each time the evaluation target motion is performed and each time the evaluation target stage is performed.

First, the operator PO inputs a request to operate the robot to the operation terminal 200, and the operation terminal 200 transmits the request to the server 300 (step S101). The server 300 searches for the robot 100 that can be operated by the operator PO, and connects the information processing device 180 of the searched robot 100 and the operation terminal 200 via the communication network N (step S102).

When the operator PO receives the connection completion notification from the server 300, the operator PO inputs a robot operation execution start command to the operation terminal 200. Operation terminal 200 transmits the command to information processing device 180 . The information processing device 180 activates the robot 100 so that it can operate according to the operation command received from the operation terminal 200 . The information processing device 180 turns ON the work execution flag for the service of the robot 100, that is, starts work (step S103).

Next, the operator PO inputs into the operation terminal 200 the specification conditions of the motion-related reference data used for evaluating the robot operation (step S104). The operation terminal 200 transmits a command for the specified condition to the information processing device 180 . For example, the specified condition may be a condition that specifies standard action-related reference data or a condition that specifies attributes of evaluation results to be used. In this example, the operator PO inputs a specified condition that designates the use of the evaluation results for the operator PO.

Next, the information processing device 180 generates new action-related reference data using the specified condition received from the operation terminal 200 and the accumulated data accumulated in the information processing device 180, and uses the action-related reference data. is determined (step S105). The action-related reference data is action-related reference data adjusted in accordance with the evaluation result of the operator PO.

Next, the operator PO searches for the user P requesting the service within the service providing area AS while checking the screen of the presentation device 203 that displays the captured image of the imaging device 145 and the like. When the operator PO finds the user P, the operator PO operates the operation terminal 200 while visually recognizing the screen of the presentation device 203 that displays the captured images of one or both of the imaging devices 145 and 146, and moves the robot 100 to the user. It is caused to travel to a position near P (step S106).

The information processing device 180 detects motion-related information of the robot 100 at predetermined time intervals while the robot 100 is running, and executes evaluation processing and presentation processing of the motion-related information each time it is detected (step S107). In this example, each piece of action-related information at a predetermined time interval corresponds to an evaluation target action. The information processing device 180 causes the presentation device 203 to display the price of the action-related information and the accumulated result of the price. For example, the information processing device 180 evaluates the evaluation target features such as the running speed, running acceleration, running trajectory, and interference with the robot peripheral elements for the motion-related information. In this example, the information processing apparatus 180 executes the detection process, the evaluation process, and the presentation process without providing a delay time between them. , it is possible to check in real time without delay from the time when the operation for the evaluation target motion is input to the operation terminal 200, that is, in real time.

When the robot 100 arrives at a position near the user P, the information processing device 180 executes evaluation processing and presentation processing of the first work stage, which is a travel process from the starting position to the near position, and Then, the result of integration is displayed on the presentation device 203 (step S108).

When the operator PO confirms that the robot 100 has arrived at a position near the user P via the screen of the presentation device 203, the operator PO makes an input to the operation terminal 200 and the display device 148 and the voice output device 149 of the robot 100. An inquiry is made to the user P about the article W to be requested via the output from either or both of them (step S109).

Next, when the user P inputs the information of the requested article W1 to either or both of the sound collector 147 and the touch panel of the display device 148, the operator PO inputs to the operation terminal 200 to receive the requested article W1. A request is made to the information processing device 180 for information on the position of the requested article W1 on the shelf SR. The information processing device 180 can store information on the service providing area AS where the robot 100 is arranged and information on the service provided, in addition to the information on the robot 100 on which the information processing device 180 is mounted. The operation terminal 200 receives the information on the position PW1 of the requested item W1 from the information processing device 180 and presents it to the operator PO (step S110). The server 300 may hold information on the service providing area AS and information on the service provided, and the operation terminal 200 may request the information on the position of the requested article W1 from the server 300 .

Next, the operator PO operates the operation terminal 200 while visually recognizing the screen of the presentation device 203 that displays the information of the position PW1 and the captured image of one or both of the imaging devices 145 and 146, and operates the robot. 100 is moved to a position near the position PW1 (step S111).

As in the process of step S107, the information processing device 180 detects motion-related information of the robot 100 at predetermined time intervals while the robot 100 is running, and executes evaluation processing and presentation processing of the motion-related information each time detection is performed. (step S112).

As in the process of step S108, when the robot 100 reaches a position near the position PW1, the information processing device 180 performs evaluation processing of the second work stage, which is a traveling process from a position near the user P to a position near the position PW1. And presentation processing is executed (step S113).

When the operator PO confirms through the screen of the presentation device 203 that the robot 100 has arrived at a position near the position PW1, the operator PO instructs the operation terminal 200 to operate the robot arms 120A and 120B and the end effectors 130A and 130B. Input (step S114). This operation is to take out the requested item W1 stored in the storage shelf SR from the storage shelf SR using the end effectors 130A and 130B and place the requested item W1 on the table 140a of the robot 100. FIG. The operator PO inputs an operation to the operation terminal 200 while viewing the screen of the presentation device 203 that displays the captured images of the imaging devices 144 and 145 .

The information processing device 180 detects the motion-related information of the robot 100 during the motion of the robot 100, and every time the motion-related information corresponding to the motion to be evaluated is detected, the motion-related information is evaluated and presented ( step S115). For example, the motions to be evaluated include the movement motions of the robot arms 120A and 120B that move the end effectors 130A and 130B to the required item W1, the gripping motion of the end effectors 130A and 130B that grip the required item W1, and the movement of the required item W1 onto the table 140a. one or more of the processing target operations including the movement and placement operation of the robot arms 120A and 120B that move and place the required article W1, and the grip release operation of the end effectors 130A and 103B that release the grip of the requested article W1. It is a target action. In this example, all the motions to be processed are motions to be evaluated.

The information processing device 180 evaluates the evaluation target feature for the action-related information corresponding to each evaluation target action. For example, the evaluation target features include the time required for the operation of the robot 100, the positions, postures, position movement speeds, posture movement speeds, position accelerations, posture accelerations, and movement trajectories of the end effectors 130A and 103B, and the required article Position, attitude and state of W1; force and impact exerted by robot 100 on requested article W1; contact, force and impact exerted by robot 100 on object peripheral elements; interference between robot 100 and robot peripheral elements may include one or more of For example, the object peripheral elements may be shelves of the storage shelf SR, other articles, and the like. For example, the acceleration of the end effectors 130A and 103B can indicate the impact or the like that the end effectors 130A and 103B give to the requested article W1.

For example, the information processing device 180 processes an image of the requested article W1 captured by either or both of the imaging devices 144 and 145, and holds the requested article W1 gripped by the end effectors 130A and 103B. It may be configured to detect a state and evaluate the holding state as the feature to be evaluated. The information processing device 180 processes the image of the requested article W1 on the table 140a captured by the imaging device 144 or the like, detects the placement state of the requested article W1, and evaluates the placement state as an evaluation target feature. It may be configured as

When the end effectors 130A and 103B detect that the end effectors 130A and 103B have released the requested item W1, the information processing apparatus 180 performs a third work, which is a take-out operation process from the arrival of the robot 100 at a position near the position PW1 to the completion of the release of the grip. The stage evaluation process and presentation process are executed, and the price and the accumulated result are displayed on the presentation device 203 (step S116).

Next, when the operator PO confirms that the requested article W1 is placed on the table 140a through the screen of the presentation device 203, an image captured by one or both of the imaging devices 145 and 146 is displayed. While viewing the screen of the presentation device 203, the operation terminal 200 is operated to cause the robot 100 to travel to a position near the user P (step S117).

As in the process of step S107, the information processing device 180 detects motion-related information of the robot 100 at predetermined time intervals while the robot 100 is running, and executes evaluation processing and presentation processing of the motion-related information each time detection is performed. (step S118).

As in the process of step S108, when the robot 100 reaches a position near the user P, the information processing device 180 performs an evaluation process of a fourth work stage, which is a traveling process from a position near the position PW1 to a position near the user P. And presentation processing is executed (step S119).

Next, when the operator PO confirms that the robot 100 has arrived at a position near the user P through the screen of the presentation device 203, the operator PO operates the operation terminal to cause the robot arms 120A and 120B and the end effectors 130A and 130B to operate. 200 (step S120). This operation is an operation of handing over the requested item W1 placed on the table 140a to the user P using the end effectors 130A and 130B. The operator PO inputs an operation to the operation terminal 200 while viewing the screen of the presentation device 203 that displays the captured images of the imaging devices 144 and 145 .

The information processing device 180 detects the motion-related information of the robot 100 during the motion of the robot 100, and every time the motion-related information corresponding to the motion to be evaluated is detected, the motion-related information is evaluated and presented ( step S121). For example, the motions to be evaluated include the movement motions of the robot arms 120A and 120B that move the end effectors 130A and 130B to the required item W1, the gripping motions of the end effectors 130A and 130B that grip the required item W1, and the user P's movement of the requested item W1. Objects to be processed include movement operations of the robot arms 120A and 120B that move forward and present, and grip releasing operations of the end effectors 130A and 103B that release the grip of the requested article W1 according to the holding state of the requested article W1 by the user P. One or more of the actions to be processed. In this example, all the motions to be processed are motions to be evaluated.

The information processing device 180 evaluates the evaluation target feature for the action-related information corresponding to each evaluation target action. For example, the evaluation target features include the time required for the operation of the robot 100, the positions, postures, position movement speeds, posture movement speeds, position accelerations, posture accelerations, and movement trajectories of the end effectors 130A and 103B, and the required article It may include one or more of the position, posture, and state of W1, the force and impact exerted by the robot 100 on the requested item W1, and the contact, force, and impact exerted by the robot 100 on an object peripheral element such as the user P. . For example, the information processing device 180 processes an image of the requested article W1 captured by either or both of the imaging devices 144 and 145, and holds the requested article W1 gripped by the end effectors 130A and 103B. It may be configured to detect a state and evaluate the holding state as the feature to be evaluated.

When the end effectors 130A and 103B detect that the end effectors 130A and 103B have released the requested item W1, the information processing apparatus 180 performs a fifth work, which is a hand-over operation process from the arrival of the robot 100 to a position near the user P to the completion of the release of the grip. The stage evaluation process and presentation process are executed, and the price and the accumulated result are displayed on the presentation device 203 (step S122).

When the operator PO ends the robot operation, the operator PO inputs an operation end command to the operation terminal 200 . When the information processing apparatus 180 receives an operation end command from the operation terminal 200 (Yes in step S123), the information processing apparatus 180 ends the series of processes. The information processing device 180 turns off the work execution flag and stops the robot 100 . The server 300 cuts off the connection between the operation terminal 200 and the information processing device 180 . The information processing apparatus 180 returns to step S106 when not receiving an operation end command (No in step S123).

In the course of the processing from steps S106 to S123, the information processing device 180 collects the action-related information corresponding to the action to be evaluated, the evaluation result for the work stage, the price, the accumulated price, and the action-related reference data used for the evaluation. are stored and accumulated in association with each other.

Through the processing of steps S101 to S123 as described above, the operator PO operates the robot 100 while confirming the evaluation of the operation of the operator PO through the consideration and the accumulated result of the consideration each time the motion to be evaluated and the work stage are performed. can be manipulated. As a result, the operator PO can find a concrete measure for improving the skill of operating the robot. The operator PO can be motivated to improve the skill of operating the robot in order to obtain more compensation.

(Modification)
This modification differs from the embodiment in that the robot system includes a learning device 400 . In the following, this modified example will be described with a focus on the points that are different from the embodiment, and the description of the points that are the same as the embodiment will be omitted as appropriate.

FIG. 7 is a block diagram showing an example of functional configurations of an information processing device 180A and a learning device 400 according to a modification. As shown in FIG. 7, the information processing device 180A further includes a data input/output unit 180i as a functional component. The data input/output unit 180 i inputs and outputs data to and from the learning device 400 . For example, the data input/output unit 180 i outputs accumulated data stored in the second storage unit 181 b to the learning device 400 . The data input/output unit 180i requests action-related reference data from the learning device 400 in response to a request from the reference determination unit 180g. The data input/output unit 180 i receives the specified condition and the information of the evaluation result satisfying the specified condition from the reference determination unit 180 g and sends the information to the learning device 400 . The learning device 400 sends the action-related reference data corresponding to the specified condition and the evaluation result to the data input/output unit 180i.

The learning device 400 includes a computer device, similar to the information processing device 180A. Although not limited, in this modification, the learning device 400 is a device separate from the information processing device 180A and the robot controller 190, and is capable of data communication with the information processing device 180A via wired communication, wireless communication, or a combination thereof. connected to Any wired or wireless communication may be used. The learning device 400 is mounted on the robot 100 together with the information processing device 180A. The learning device 400 may be incorporated in the information processing device 180A or the robot controller 190. FIG. The learning device 400 and the information processing device 180A form an information processing system 500. FIG. Such a learning device 400 may include processing circuitry including a processor and memory.

The learning device 400 may be configured to input/output data with the information processing device 180A via a storage medium. Storage media include semiconductor-based or other integrated circuits (ICs), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical disks, optical disk drives (ODDs), optical Magnetic disk, magneto-optical drive, floppy disk drive (FDD), magnetic tape, solid state drive (SSD), RAM drive, secure digital card or drive, any other suitable storage medium, or two of these Combinations of the above may be included.

In this modification, the learning device 400 is mounted on the robot 100, but may be arranged outside the robot 100 and connected to the information processing device 180A via the communication network N. The learning device 400 may be configured to connect with the information processing devices 180A of two or more robots 100 . For example, the learning device 400 may be placed in the service provision area AS, may be placed in the operation area AO, or may be placed elsewhere. For example, the learning device 400 may be arranged at the location of the server 300 and configured to perform data communication with the information processing device 180A via the server 300 . The learning device 400 may be incorporated into the server 300 .

The learning device 400 includes a data input/output unit 401, a learning data storage unit 402, a machine learning unit 403, an input data processing unit 404, and an output data processing unit 405 as functional components. The function of the learning data storage unit 402 is implemented by a memory, a storage, or a combination thereof that may be included in the learning device 400, and the functions of functional components other than the learning data storage unit 402 are implemented by a processor or the like included in the learning device 400. Realized. The machine learning unit 403 is an example of a functional unit implemented by a machine learning circuit, and the learning data storage unit 402 is an example of a functional unit implemented by a second storage device.

The data input/output unit 401 receives accumulated data from the data input/output unit 180i of the information processing device 180A, and stores the accumulated data in the learning data storage unit 402 as machine learning data. The data input/output unit 401 receives, from the data input/output unit 180 i , a request command for motion-related reference data, information on specified conditions, and information on evaluation results satisfying the specified conditions, and outputs the information to the input data processing unit 404 . The data input/output unit 401 receives the action-related reference data from the output data processing unit 405 and sends the action-related reference data to the data input/output unit 180i.

The input data processing unit 404 converts the specified condition information and evaluation result information received from the data input/output unit 401 into data that can be input to the machine learning model of the machine learning unit 403 and outputs the data to the machine learning unit 403 . The output data processing unit 405 converts the output data of the machine learning unit 403 into motion-related reference data and outputs the data to the data input/output unit 401 .

The machine learning unit 403 includes a machine learning model. The machine learning unit 403 causes the machine learning model to perform machine learning using the machine learning data, and improves the accuracy of the output data with respect to the input data in the machine learning model. Machine learning models may include neural networks, random forests, genetic programming, regression models, tree models, Bayes models, time series models, clustering models, ensemble learning models, etc., but in this modification, neural Including network. A neural network includes multiple layers of nodes, including an input layer and an output layer. The node layer contains one or more nodes. When a neural network includes an input layer, an intermediate layer, and an output layer, the neural network performs output processing from the input layer to the intermediate layer and output processing from the intermediate layer to the output layer for information input to the nodes of the input layer. are sequentially performed, and output results that match the input information are output. Each node in one layer is connected to each node in the next layer, and the connections between nodes are weighted. The information of the nodes of one layer is weighted by the connection between nodes and output to the nodes of the next layer.

The machine learning model uses condition-related information related to specified conditions and evaluation-related information related to evaluation results as input data, and outputs information related to action-related criteria corresponding to the condition-related information and the evaluation-related information. data. The condition-related information may be information indicating either or both of the specified condition and the attribute of the evaluation result, and the evaluation-related information may be information indicating the evaluation result.

The machine learning model uses the condition-related information and evaluation-related information included in the machine learning data as input data for machine learning, and uses the criterion-related information related to the action-related criterion data corresponding to the condition-related information and the evaluation-related information. Let it be teacher data. The reference-related information may be information indicating motion-related reference data. For example, in machine learning, the machine learning unit 403 matches or compares output data of a machine learning model when machine learning input data is input with teacher data corresponding to the machine learning input data. Adjust the weighting of the connections between nodes in the neural network, such as to minimize the error. When the condition-related information and the evaluation-related information are input, the weighted-adjusted machine learning model can output the reference-related information appropriate to the information.

The learning device 400 as described above can output action-related reference data more suitable for the specified condition and the evaluation result than the action-related reference data determined by the reference determination unit 180g of the embodiment. In this modification, the machine learning model of the machine learning unit 403 may include the function of the output data processing unit 405 and may be configured to output the same output data as the output data processing unit 405 does.

(Other embodiments)
Although exemplary embodiments and modifications of the present disclosure have been described above, the present disclosure is not limited to the above embodiments and modifications. That is, various modifications and improvements are possible within the scope of the present disclosure. For example, embodiments and modified examples with various modifications, and forms constructed by combining components of different embodiments and modified examples are also included within the scope of the present disclosure.

For example, in the embodiment and modifications, the work of the robot 100 targeted by the robot system 1 is work related to the service industry, but is not limited to this. The robot system 1 may target any work. For example, the robot system 1 may be intended for tasks related to industry, agriculture, and other industries.

In the embodiment and modification, the robot system 1 includes the robot 100, but the robot included in the robot system 1 may be any robot capable of performing work. For example, the robot may have a moving device with a configuration different from that of the carrier 110, and may have legs or the like. The robot does not have to be equipped with a mobile device. For example, the robot may have components other than the robot arm, and may be, for example, a humanoid robot, an animal robot, or the like.

In the embodiment and the modification, the information processing devices 180 and 180A capture the images of external elements other than the robot 100, such as robot peripheral elements, objects, and object peripheral elements. However, it is not limited to this. For example, the information processing devices 180 and 180A may detect either or both of the detection result of an external sensor, which is a sensor arranged separately from the external element, and the detection result of a mounted sensor, which is a sensor arranged on the external element. may be configured to detect the state of an external element. The external sensor may be configured to detect the position, orientation, etc. of the external element from the outside. For example, the external sensor is configured to perform detection using light waves, lasers, magnetism, radio waves, electromagnetic waves, ultrasonic waves, or a combination of two or more of these, and the like. It may also include sensors, ultrasonic sensors, various lidars (LiDAR), or combinations of two or more of these. The on-board sensor may be configured to move with the external element, detect the position, orientation, etc. of the external element. For example, onboard sensors may include acceleration sensors, angular rate sensors, magnetic sensors, GPS (Global Positioning System) receivers, or combinations of two or more of these, and the like.

The information processing devices 180 and 180A according to the embodiment and modifications may be configured to use AI (Artificial Intelligence) for processing. For example, AI can be used for the processing of determining the evaluation of the motion to be processed, the processing of determining the evaluation of the work stage, the processing of calculating the price and the accumulated result, image processing, and the like. For example, AI may include learning models that perform machine learning. For example, a learning model may include a neural network.

In the embodiment and modifications, the server 300 may be configured to have at least some of the functions of the information processing devices 180 and 180A and the learning device 400. For example, server 300 may be configured to have the functionality of one or more of the functional components included in information processors 180 and 180A shown in FIGS. Server 300 may be configured to have the functionality of one or more of the functional components included in learning device 400 shown in FIG.

In addition, each aspect example of the technology of the present disclosure is listed as follows. An information processing device according to an aspect of the present disclosure is an information processing device that evaluates an operation of the robot for causing the robot to perform a task, the processing circuit including a processing circuit, wherein the processing circuit receives an input from a user to an operation device. detection processing for detecting motion-related information, which is information related to the motion of the robot that operates in accordance with an operation command received; and a presentation process of presenting the evaluation result of the evaluation process on a presentation device that receives information presentation while the user operates the operation device, are executed during the execution of the work. The processing circuit is configured to perform a first timing each time an evaluation target motion is a motion of the robot to be evaluated, and a second timing each time an evaluation target stage is a work stage to be evaluated in the work. When the evaluation processing is executed at the first timing, the processing circuit executes the evaluation processing in the same order as the execution order of the evaluation target operation. When causing the presentation device to present the evaluation result regarding the action, and executing the evaluation process at the second timing, the processing circuit presents the evaluation result regarding the evaluation target stage in the same order as the execution order of the evaluation target stage. is presented on the presentation device.

According to the above aspect, the information processing apparatus performs the evaluation target motion and the evaluation target stage at either or both of the first timing each time the evaluation target motion is performed and the second timing each time the evaluation target stage is performed. Evaluation processing is performed on either or both of them, and the evaluation results are presented to the user via the presentation device. The user can check the evaluation of the operation at the above timing. Furthermore, the user can confirm the evaluation for the operation in the same order as the operation order. As a result, the user can recognize the evaluated operation and the evaluation result of the operation, so that the user can be motivated to improve the operation accuracy when performing the evaluation target operation next time. Therefore, the information processing device can increase the user's desire to operate the robot.

In the information processing device according to one aspect of the present disclosure, when the processing circuit executes the evaluation process at the first timing, the evaluation process related to the first evaluation target action and the first evaluation target action may be executed by the timing of the second evaluation target action following the first evaluation target action.

According to the above aspect, the information processing device executes the evaluation process and the presentation process regarding the evaluation target motion by the timing of the next evaluation target motion. Therefore, the user can accurately recognize the evaluation target action that has been evaluated and the evaluation result regarding the evaluation target action.

In the information processing device according to one aspect of the present disclosure, when the processing circuit executes the evaluation process at the second timing, the evaluation process related to the first evaluation stage and the first evaluation stage may be executed by the timing of the second evaluation target stage following the first evaluation target stage.

According to the above aspect, the information processing device executes the evaluation process and the presentation process for the evaluation target stage by the timing of the next evaluation target stage. Therefore, the user can accurately recognize the evaluation target stage that has been evaluated and the evaluation result related to the evaluation target stage.

In the information processing device according to an aspect of the present disclosure, the processing circuit further determines a compensation to be given to the user for operating the robot based on the evaluation result, The presentation device may be caused to present the results of the accumulation of the considerations being executed.

According to the above aspect, the information processing device presents to the user, via the presentation device, the price indicating the individual evaluation results and the accumulated result of the price. Thereby, the user can be motivated to operate the robot in order to obtain more compensation. For example, the consideration may be a reward amount, points, score, score, or the like.

In the information processing device according to one aspect of the present disclosure, the processing circuit may include, as the motion-related information, the operation command and a motion command generated for the robot to cause the robot to operate according to the operation command. , detecting one or more of an actual operation result of the robot, a state of the surrounding environment of the robot, a state of an object to which the robot acts, and a state of the surrounding environment of the object. good too. According to the above aspect, the information processing device can perform evaluation processing based on various information.

In the information processing device according to one aspect of the present disclosure, the processing circuit may convert two or more evaluation target features, which are features for evaluating the action-related information, into the two or more evaluation target features included in the action-related criteria. The action-related information may be evaluated based on the evaluation results of the two or more evaluation target features by comparing with a reference of the evaluation target features.

According to the above aspect, the evaluation method in the information processing device can be clear and concise. Therefore, it is possible to reduce the processing amount of the evaluation process and speed up the process. For example, the features to be evaluated may include features such as required time, position, orientation, trajectory, position velocity, orientation velocity, position acceleration, and orientation acceleration for the motion of the robot. The evaluation target features may include features such as force and impact applied by the robot to the surrounding environment of the robot, and features such as changes in the position, posture, shape and state of the surrounding environment applied by the robot. Features to be evaluated include features such as the position, posture, positional velocity, posture velocity, positional acceleration, and posture acceleration of the object to which the robot acts, and shape changes and states of the object to which the robot acts characteristics such as changes and workmanship, characteristics such as force and impact exerted by the robot on the surrounding environment of the object, and characteristics such as changes in the position, posture, shape and state of the surrounding environment given by the robot. The information processing apparatus may determine the evaluation result of the motion-related information by adding weights to the evaluation results of each of the two or more evaluation target features.

The information processing apparatus according to an aspect of the present disclosure further includes a first storage device, and the first storage device stores the information of the action-related criteria and the evaluation result of the action-related information based on the action-related criteria. and the processing circuit uses the information of the first action-related criterion and the information of the evaluation result based on the first action-related criterion stored in the first storage device to perform a second may determine information of said action-related criteria of. According to the above aspect, the information processing device can newly determine the action-related criteria according to the evaluation results accumulated in the first storage device. The first storage device may be included in the processing circuitry or separate from the processing circuitry.

In the information processing device according to an aspect of the present disclosure, the processing circuit uses information of the evaluation result that satisfies a specified condition and the evaluation result from information accumulated in the first storage device. extracting the information of the first action-related criterion, which is the action-related criterion, and extracting the information of the second action-related criterion using the extracted information of the evaluation result and the information of the first action-related criterion; may be determined.

According to the above aspect, the information processing device can determine a new action-related criterion using evaluation results that satisfy specified conditions. Thus, the information processing device can determine action-related criteria for various purposes. For example, the designated condition may be a condition that designates evaluation results corresponding to an object to be acted upon by the robot, the robot, the work, the work place, the work environment, the work execution date, the work execution period, and the like.

A learning device according to an aspect of the present disclosure uses the information of the action-related criterion and the information of the evaluation result of the action-related information based on the action-related criterion in the information processing device according to the aspect of the present disclosure for machine learning. a second storage device for storing data, and a machine learning circuit, wherein the machine learning circuit uses the information of the evaluation result included in the machine learning data as input data for machine learning, and the machine learning data Machine learning is performed using the information of the action-related reference included and used for the evaluation result as teacher data, and the machine learning circuit uses evaluation information corresponding to the information of the evaluation result as input data and corresponds to the evaluation information. The information of the action-related criteria is used as output data.

According to the above aspect, when the learning device after machine learning receives input of evaluation information corresponding to evaluation results, it can output action-related criteria suitable for the evaluation information. For example, when evaluation information corresponding to a high evaluation result is input to the learning device, the learning device outputs a simple, low-level action-related criterion that is likely to obtain a high evaluation. For example, when rating information corresponding to a low rating result is input to the learning device, the learning device outputs high-level action-related criteria that are difficult to obtain high ratings. For example, by using the learning device, the user can acquire the action-related reference of the difficulty level desired by the user and use it for operating the robot. The second storage device may be separate from the machine learning circuitry, the processing circuitry and the first storage device, may be included in the machine learning circuitry, the processing circuitry or the first storage device, and may include the first storage device. .

An information processing system according to one aspect of the present disclosure includes an information processing device according to one aspect of the present disclosure and a learning device according to one aspect of the present disclosure, wherein the learning device is the operating device or the information processing device. receives input data from and outputs output data corresponding to the input data to the information processing device. According to the above aspect, the information processing system can obtain the same effects as the information processing device and the learning device according to one aspect of the present disclosure. The information processing system can use the action-related criteria output from the learning device to perform evaluation processing in the information processing device.

A robot system according to one aspect of the present disclosure includes an information processing device according to one aspect of the present disclosure, the robot, and a robot controller that controls an operation of the robot, wherein the information processing device includes the robot controller and Communicatively connected. According to the above aspect, the robot system can obtain the same effects as the information processing device according to one aspect of the present disclosure.

A robot system according to one aspect of the present disclosure includes an information processing system according to one aspect of the present disclosure, the robot, and a robot controller that controls an operation of the robot, wherein the information processing device includes the robot controller and Communicatively connected. According to the above aspect, the robot system can obtain the same effects as the information processing system according to one aspect of the present disclosure.

The functions of the elements disclosed herein may be circuits including general purpose processors, special purpose processors, integrated circuits, ASICs, conventional circuits, and/or combinations thereof configured or programmed to perform the disclosed functions. or can be performed using processing circuitry. A processor is considered a processing circuit or circuit because it includes transistors and other circuits. In this disclosure, a circuit, unit, or means is hardware that performs or is programmed to perform the recited functions. The hardware may be the hardware disclosed herein, or other known hardware programmed or configured to perform the recited functions. A circuit, means or unit is a combination of hardware and software, where the hardware is a processor which is considered a type of circuit, the software being used to configure the hardware and/or the processor.

All numbers such as ordinal numbers and quantities used above are examples for specifically describing the technology of the present disclosure, and the present disclosure is not limited to the numbers illustrated. The connection relationship between components is an example for specifically describing the technology of the present disclosure, and the connection relationship for realizing the function of the present disclosure is not limited to this.

The division of blocks in the functional block diagram is an example. Multiple blocks may be implemented as one block, one block may be divided into multiple blocks, some functions may be moved to other blocks, and two or more of these may be combined. . A single piece of hardware or software may process the functions of multiple blocks having similar functions in parallel or in a time division manner.

As the present disclosure may be embodied in various forms without departing from the spirit of its essential characteristics, the scope of the present disclosure is defined rather by the appended claims than by the written description; The exemplary embodiments and variations are exemplary and not limiting. All changes that come within the claims and their scope, or equivalents of the claims and their scope, are intended to be covered by the claims.

Claims (12)

1. An information processing device that evaluates an operation of the robot for causing the robot to perform a task,
   including a processing circuit;
   The processing circuit is
   a detection process for detecting motion-related information, which is information related to the motion of the robot that operates in accordance with an operation command input to an operation device by a user;
   an evaluation process for comparing and evaluating the detected motion-related information with a motion-related criterion, which is a criterion related to the motion of the robot;
   a presentation process for presenting an evaluation result of the evaluation process on a presentation device that receives information presentation while the user operates the operation device;
   configured to run during execution of said work,
   The processing circuit performs either a first timing each time an evaluation target motion is performed, which is a motion of the robot to be evaluated, or a second timing each time an evaluation target stage, which is a work stage to be evaluated in the work. or executing the evaluation process at both timings,
   When executing the evaluation process at the first timing, the processing circuit causes the presentation device to present the evaluation results regarding the evaluation target motion in the same order as the execution order of the evaluation target motion,
   When the evaluation process is executed at the second timing, the processing circuit causes the presentation device to present the evaluation results regarding the evaluation target stages in the same order as the execution order of the evaluation target stages.
2. When executing the evaluation process at the first timing, the processing circuit performs the evaluation process regarding the first evaluation target action and the presentation process of the evaluation result regarding the first evaluation target action as described above. The information processing apparatus according to claim 1, which is executed by the timing of the second evaluation target action following the first evaluation target action.
3. When executing the evaluation process at the second timing, the processing circuit performs the evaluation process regarding the first evaluation target stage and the presentation process of the evaluation result regarding the first evaluation target stage as described above. The information processing apparatus according to claim 1 or 2, which is executed by the timing of the second evaluation target stage following the first evaluation target stage.
4. The processing circuit further determines a compensation to be given to the user for operating the robot based on the evaluation result, and stores the determined compensation and a result of accumulating the compensation during execution of the work. The information processing apparatus according to any one of claims 1 to 3, which is caused to be presented by a presentation apparatus.
5. The processing circuit stores, as the motion-related information, the operation command, a motion command generated for the robot to cause the robot to move according to the operation command, an actual motion result of the robot, and the robot. 5. The apparatus according to any one of claims 1 to 4, wherein one or more of a state of the surrounding environment of the robot, a state of an object on which the robot acts, and a state of the surrounding environment of the object are detected. Information processing equipment.
6. The processing circuit compares and evaluates two or more evaluation target features, which are features for evaluating the action-related information, with criteria for the two or more evaluation target features included in the action-related criteria, and The information processing apparatus according to any one of claims 1 to 5, wherein the action-related information is evaluated based on evaluation results of two or more evaluation target features.
7. further comprising a first storage device;
   the first storage device stores information on the action-related criteria and information on the evaluation result of the action-related information based on the action-related criteria;
   The processing circuitry uses the information of the first action-related criteria and the information of the evaluation result based on the first action-related criteria stored in the first storage device to determine the second of the action-related criteria. The information processing apparatus according to any one of claims 1 to 6, wherein information is determined.
8. The processing circuit selects, from information stored in the first storage device, information on the evaluation result that satisfies a specified condition, and the first action relation that is the action relation criterion used for the evaluation result. 8. The information processing apparatus according to claim 7, wherein reference information is extracted, and the extracted information of the evaluation result and the information of the first action-related reference are used to determine the information of the second action-related reference. .
9. 9. The information processing apparatus according to any one of claims 1 to 8, wherein the information of the action-related reference and the information of the evaluation result of the action-related information based on the action-related reference are stored as data for machine learning. 2 a storage device;
   a machine learning circuit;
   The machine learning circuit uses the information on the evaluation result contained in the data for machine learning as input data for machine learning, and the information on the action-related criterion contained in the data for machine learning and used for the evaluation result as a teacher. as data, machine learning,
   The learning device, wherein the machine learning circuit uses the evaluation information corresponding to the evaluation result information as input data and the action-related reference information corresponding to the evaluation information as output data.
10. an information processing apparatus according to any one of claims 1 to 8;
    A learning device according to claim 9,
    The information processing system, wherein the learning device is configured to receive input data from the operation device or the information processing device and output output data corresponding to the input data to the information processing device.
11. an information processing apparatus according to any one of claims 1 to 8;
    the robot;
    A robot controller that controls the operation of the robot,
    The robot system, wherein the information processing device is communicably connected to the robot controller.
12. an information processing system according to claim 10;
    the robot;
    A robot controller that controls the operation of the robot,
    The robot system, wherein the information processing device is communicably connected to the robot controller.

Images