WO2020189143A1 - Control device and control method - Google Patents

Control device and control method Download PDF

Info

Publication number
WO2020189143A1
WO2020189143A1 PCT/JP2020/006260 JP2020006260W WO2020189143A1 WO 2020189143 A1 WO2020189143 A1 WO 2020189143A1 JP 2020006260 W JP2020006260 W JP 2020006260W WO 2020189143 A1 WO2020189143 A1 WO 2020189143A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
robot
operator
unit
selection
Prior art date
Application number
PCT/JP2020/006260
Other languages
French (fr)
Japanese (ja)
Inventor
チャリス ラサンタ フェルナンド
高志 岩佐
山中　宏治
Original Assignee
Ｔｅｌｅｘｉｓｔｅｎｃｅ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ｔｅｌｅｘｉｓｔｅｎｃｅ株式会社 filed Critical Ｔｅｌｅｘｉｓｔｅｎｃｅ株式会社
Publication of WO2020189143A1 publication Critical patent/WO2020189143A1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/02—Hand grip control means

Definitions

  • the present invention relates to a control device and a control method for controlling a robot.
  • Patent Document 1 discloses a control device for assisting an operator who is not proficient in operating a robot so that the robot can be appropriately operated.
  • the robot By assisting the operation of the robot by the control device, the robot can be operated appropriately even when the operator performs an inappropriate operation.
  • the operator may feel uncomfortable when the robot performs an operation different from the operation content performed by the robot.
  • the work efficiency may be lowered because the robot performs an operation different from the operation content performed by the operator.
  • the selection unit selects the selection assistance process based on, for example, the operation skill of the operator.
  • the selection unit may select the selection assisting process based on the content of the operation indicated by the operation information.
  • the selection unit may select the selection assisting process based on the operation content indicated by the scenario in which the elapsed time from the reference time and the operation content of the robot are associated with each other.
  • the information acquisition unit further acquires detection information indicating a state around the robot detected by a sensor provided in the robot, and the selection unit selects the selection assist process based on the detection information. May be good.
  • the selection unit may change the selection assist process when the detection information indicates that the robot is not operating normally.
  • the information acquisition unit displays information indicating candidates for the plurality of auxiliary processes on an information terminal accessible to the operator or the administrator of the operator, and then indicates the candidates selected in the information terminal. Even if the selection candidate information is acquired and the selection unit selects the one or more auxiliary processes designated by the operator or the administrator of the operator as the selection auxiliary process based on the selection candidate information. Good.
  • the information acquisition unit displays an auxiliary screen for selecting the degree of assistance as information indicating the plurality of auxiliary processing candidates on the information terminal, and then indicates the degree of assistance input in the information terminal.
  • the level information may be acquired, and the selection unit may determine the content of the selection auxiliary processing based on the auxiliary level information.
  • the auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information, and the selection unit uses the correction information as the control information in the robot control unit as the selection auxiliary processing. You may select at least one of the correction process for controlling the robot and the instruction process for displaying the information for assisting the operation on the operation device used by the operator for the operation.
  • the auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information, and the selection unit causes the robot control unit to control the robot by using the operation information as the control information. You may switch between the non-correction mode and the correction mode in which the robot control unit controls the robot by using the correction information as the control information as the selection assist process.
  • the auxiliary processing unit may create the correction information by substituting the operation information with predetermined reference control information corresponding to the content of the operation executed by the operator.
  • the selection unit inputs the operation information acquired by the information acquisition unit into the machine learning model that outputs one or more auxiliary processes when the operation information is input, while the operator performs a predetermined operation.
  • One or more auxiliary processes output by the machine learning model may be selected.
  • the selection unit is executed as the predetermined operation before the operator performs a given operation, and the operation acquired by the information acquisition unit while performing a test operation corresponding to the content of the operation.
  • One or more auxiliary processes may be selected by inputting information into the machine learning model.
  • the selection unit may display the type of the selection assist processing on the operation device used by the operator for the operation.
  • the robot control unit may change the operating speed of the robot when the selection unit selects a new one or more auxiliary processes.
  • the control method of the second aspect of the present invention includes a step of acquiring operation information indicating the content of the operation by the operator who operates the robot, which is executed by the computer, and a plurality of assists for assisting the operation of the operator.
  • a step of selecting one or more auxiliary processes from the processing candidates, a step of assisting the operator by executing the selected auxiliary process, which is the one or more auxiliary processes, and the selection auxiliary process are executed.
  • it has a step of controlling the robot by using the control information corresponding to the type of the selection assisting process created based on the operation information.
  • the operator U operates the robot 2 by operating the operation device 3. Specifically, when the operator U operates the operation device 3, the operation device 3 transmits the operation information indicating the operated content to the control device 1 via the network N.
  • the control device 1 is a computer that remotely controls the robot 2 by transmitting control information based on the operation information to the robot 2.
  • the operation device 3 is a device for the operator U to remotely control the robot 2.
  • the operation device 3 has, for example, an operation tool worn by the operator U and a cockpit including a communication unit for notifying the control device 1 of the state of the operation tool of the operator U.
  • the operation device 3 identifies the operation of the operator U with the operation tool attached, and transmits the operation information indicating the specified operation to the control device 1.
  • the cockpit identifies the operation of the operator U based on the signal received from the sensor of the operation tool worn by the operator U.
  • the cockpit may irradiate the operator U with infrared light and specify the operation of the operator U based on the reflected light of the irradiated infrared light.
  • the cockpit may identify the operation of the operator U by analyzing an image of the operator U taken by the camera.
  • the operating device 3 may notify the operating tool worn by the operator U of the captured image and the perceptual information received from the robot 2 via the network N.
  • the captured image is an image generated by the imaging device provided in the robot 2 taking a picture of the surroundings of the robot 2, and the display of the goggles included in the operation tool worn by the operator U or the eyes of the operator U. It is displayed on contact lenses worn on.
  • the perceptual information is information indicating a state detected by a sensor provided in the robot 2, and the operating tool worn by the operator U on the hand, foot, arm, etc. changes the temperature or vibrates based on the perceptual information. To do. With these configurations, the operator U can experience the environment around the robot 2 at a place away from the robot 2 as if he / she is in the robot 2.
  • the control device 1 By assisting the operation of the operator U in various forms, the control device 1 enables the operator U to appropriately operate the robot 2 regardless of the operation skill of the operator U.
  • the control device 1 operates in any of (1) normal mode, (2) correction mode, (3) instruction mode, and (4) automatic mode, for example.
  • the control device 1 may operate in combination with any of these plurality of modes. For example, the control device 1 operates in the normal mode for some of the series of operations performed by the operator U, and operates in the correction mode for some of the other operations.
  • FIG. 1 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the normal mode.
  • the control device 1 transmits the operation information received from the operation device 3 to the robot 2 as control information without substantially changing it. Specifically, when the operation device 3 transmits the operation information to the control device 1 ((1) in FIG. 1) and the control device 1 receives the operation information ((2) in FIG. 1), the control device 1 receives the operation information. The received operation information is transmitted to the robot 2 as the control information for operating the robot 2 ((3) in FIG. 1). The robot 2 operates based on the operation information received from the control device 1 ((4) in FIG. 1). Since the robot 2 operates as operated by the operator U, the normal mode is suitable when the operator U has sufficient operation skills.
  • FIG. 2 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the correction mode.
  • the control device 1 creates correction information by correcting the operation information received from the operation device 3, and transmits the correction information to the robot 2 as control information.
  • the details of the correction information will be described later, but the correction information is information for changing the operation information to an appropriate content when the operation information is not appropriate.
  • the control device 1 corrects the operation information. Correction information is created ((2) in FIG. 2).
  • the control device 1 transmits the correction information to the robot 2 ((3) in FIG. 2), and the robot 2 operates based on the correction information ((4) in FIG. 2).
  • the correction mode is effective when the operator U cannot perform an accurate operation.
  • FIG. 3 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the instruction mode.
  • the control device 1 transmits auxiliary information including at least one of an image or a text indicating an operation method to be performed by the operator U to the operation device 3 ((1) in FIG. 3).
  • the operating device 3 presents the received auxiliary information to the operator U ((2) in FIG. 3).
  • the control device 1 presents the auxiliary information to the operator U, for example, by displaying the auxiliary information on the display of the operation device 3 worn by the operator U or the information terminal visible to the operator U.
  • the operation device 3 transmits the operation information indicating the contents operated by the operator U to the control device 1 with reference to the auxiliary information ((3) in FIG. 3).
  • the control device 1 receives the operation information ((4) in FIG. 3)
  • the control device 1 transmits the received operation information to the robot 2 as control information for operating the robot 2 ((5) in FIG. 3). )).
  • the robot 2 operates based on the operation information received from the control device 1 ((6) in FIG. 3).
  • the control device 1 may transmit the correction information obtained by correcting the operation information to the robot 2 as control information.
  • the instruction mode is effective when the operator U is not familiar with the operation method.
  • FIG. 4 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the automatic mode.
  • the control device 1 In the automatic mode, the control device 1 automatically creates control information for operating the robot 2 based on a predetermined scenario, and transmits the created control information to the robot 2.
  • the scenario is information in which the elapsed time from the reference time and the operation content of the robot 2 are associated with each other.
  • the reference time is, for example, the time when the robot 2 starts operating, but is not limited to this, and may be another time such as the time when the user who operates the robot 2 performs a predetermined operation.
  • the control device 1 acquires a scenario corresponding to the work performed by the operator U, for example, based on an instruction from the operator U or the administrator who manages the operator U ((in FIG. 4). 1)).
  • the control device 1 may acquire the scenario from another computer, for example, or may acquire the scenario from the storage medium in the control device 1.
  • the control device 1 creates control information based on the scenario ((2) in FIG. 4).
  • the robot 2 operates based on the control information ((3) in FIG. 4).
  • the automatic mode is effective when the skill of the operator U is low and the pattern for operating the robot 2 is determined.
  • FIG. 5 is a diagram showing a functional configuration of the control device 1.
  • the control device 1 has a communication unit 11, a storage unit 12, and a control unit 13.
  • the control unit 13 includes an information acquisition unit 131, a selection unit 132, an auxiliary processing unit 133, and a robot control unit 134.
  • the communication unit 11 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a LAN (Local Area Network) controller.
  • the communication unit 11 inputs the robot state information received from the robot 2 and the operation information received from the operation device 3 to the information acquisition unit 131.
  • the storage unit 12 includes a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk.
  • the storage unit 12 stores a program executed by the control unit 13.
  • the storage unit 12 stores auxiliary information used in the auxiliary mode.
  • the storage unit 12 may store the reference control information used in the correction mode. Details of the reference control information will be described later.
  • the control unit 13 includes, for example, a CPU (Central Processing Unit), and functions as an information acquisition unit 131, a selection unit 132, an auxiliary processing unit 133, and a robot control unit 134 by executing a program stored in the storage unit 12. .. Details of the information acquisition unit 131, the selection unit 132, the auxiliary processing unit 133, and the robot control unit 134 will be described later.
  • a CPU Central Processing Unit
  • FIG. 6 is a diagram showing a functional configuration of the robot 2.
  • the robot 2 has a communication unit 21, a mechanism unit 22, a sensor unit 23, a storage unit 24, and a control unit 25.
  • the control unit 25 includes an instruction receiving unit 251, a mechanism control unit 252, and a robot state transmitting unit 253.
  • the communication unit 21 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a wireless communication controller.
  • the communication unit 21 receives the control information transmitted from the operation device 3 via the network N. Further, the communication unit 21 transmits the robot state information indicating the state of the robot 2 to the control device 1 via the network N.
  • the mechanical unit 22 includes actuators for each part of the robot 2.
  • the mechanism unit 22 includes a mechanism member that moves under the control of the control unit 25, such as an actuator provided at a joint of each part such as a finger, a hand, a foot, a neck, and a head.
  • the sensor unit 23 is provided in each unit of the robot 2 and includes a device for detecting various states.
  • the sensor unit 23 is, for example, a camera provided in the eyes of the robot 2, a microphone provided in the temporal region of the robot 2, or a temperature sensor and a pressure sensor provided in the fingers of the robot 2.
  • the sensor unit 23 notifies the robot state transmission unit 253 of a signal indicating the state detected by each sensor.
  • the storage unit 24 includes a storage medium such as a ROM, a RAM, and a hard disk.
  • the storage unit 24 stores a program executed by the control unit 25.
  • the control unit 25 has, for example, a CPU, and functions as an instruction receiving unit 251, a mechanism control unit 252, and a robot state transmitting unit 253 by executing a program stored in the storage unit 24.
  • the instruction receiving unit 251 receives the control information transmitted by the control device 1 via the communication unit 21.
  • the instruction receiving unit 251 notifies the mechanism control unit 252 of the received control information.
  • the mechanism control unit 252 controls each actuator included in the mechanism unit 22 based on the control information notified from the instruction receiving unit 251.
  • the mechanism control unit 252 acquires control information including an instruction to move the hand to the right, for example, the mechanism control unit 252 moves the hand of the robot 2 to the right by controlling the actuator provided in the hand.
  • the mechanism control unit 252 may detect the position of each actuator after moving the mechanism unit 22 and notify the robot state transmission unit 253 of the detected position.
  • the robot state transmission unit 253 generates robot state information indicating the state of the robot 2 based on the signal detected by the sensor unit 23 and the state of the mechanism unit 22 notified from the mechanism control unit 252.
  • the robot state transmission unit 253 transmits the generated robot state information to the control device 1 via the communication unit 21.
  • FIG. 7 is a diagram showing a functional configuration of the operating device 3.
  • the operation device 3 includes an operation unit 31, a detection unit 32, a communication unit 33, a display unit 34, a storage unit 35, and a control unit 36.
  • the control unit 36 includes a state specifying unit 361, an instruction transmitting unit 362, a state transmitting unit 363, and a display control unit 364.
  • the operation unit 31 is a device for the operator U to operate the robot 2, and includes, for example, a member worn by the operator U and a button or a joystick operated by the operator U.
  • the operation unit 31 notifies the state specifying unit 361 of the operation content performed by the detected operator U.
  • the detection unit 32 includes a sensor for detecting the state of the operator U.
  • the detection unit 32 detects, for example, a camera that detects the position of the pupil of the operator U, a humidity sensor that detects the sweating state of the operator U, a pulse sensor that detects the heart rate of the operator U, and a blood pressure of the operator U. It includes a blood pressure sensor, a breathing sensor that detects the respiratory rate or exhaled component of the operator U, or an acceleration sensor that detects the movement of each part of the operator U.
  • the detection unit 32 notifies the state identification unit 361 of the detected state of the operator U.
  • the communication unit 33 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a wireless communication controller.
  • the communication unit 33 transmits the control information for the robot 2 generated by the control unit 36 to the robot 2 based on the operation content of the operator U detected by the operation unit 31 via the network N. Further, the communication unit 33 transmits the operator status information indicating the status of the operator U to the control device 1 via the network N.
  • the control unit 36 has, for example, a CPU, and functions as a state identification unit 361, an instruction transmission unit 362, a state transmission unit 363, and a display control unit 364 by executing a program stored in the storage unit 35.
  • the instruction transmitting unit 362 generates operation information for operating the robot 2 based on the operation content notified from the state specifying unit 361.
  • the instruction transmission unit 362 transmits the generated operation information to the control device 1 via the communication unit 33.
  • the state transmission unit 363 generates operator state information indicating the state of the operator U based on the state of the operator U notified from the state identification unit 361.
  • the state transmission unit 363 transmits the generated robot state information to the control device 1 via the communication unit 33.
  • the display control unit 364 creates display data to be displayed on the display unit 34.
  • the display control unit 364 causes the display unit 34 to display, for example, an image captured by the robot 2. Further, the display control unit 364 causes the display unit 34 to display the auxiliary information transmitted from the control device 1 for the operator U to visually recognize.
  • the information acquisition unit 131 acquires operation information indicating the content of the operation by the operator U who operates the robot 2 from the operation device 3.
  • the information acquisition unit 131 may acquire operation information via another device capable of communicating with the operation device 3.
  • the operation information is information including numerical values corresponding to the amount of movement of each part of the robot 2, the direction of movement, or the speed of movement.
  • the operation information is the movement of the operator U related to the operation of the robot 2, such as the amount of movement of each part of the operator U such as a hand, finger, foot, or head, the direction in which each part is moved, or the speed of movement. It is information indicating.
  • the operation information may be a numerical value or text input to the operation device 3 or another device by the operator U.
  • the information acquisition unit 131 inputs the acquired operation information to the selection unit 132 and the auxiliary processing unit 133.
  • the information acquisition unit 131 may further acquire detection information indicating a state around the robot detected by a sensor provided in the robot 2.
  • the sensors are, for example, an image sensor, a temperature sensor, a pressure sensor, and an acceleration sensor.
  • the information acquisition unit 131 inputs the acquired detection information to the selection unit 132 and the auxiliary processing unit 133.
  • the information acquisition unit 131 may acquire various types of information from other devices.
  • the information acquisition unit 131 acquires skill information indicating the operation skill of the operator U from, for example, a computer used by the administrator of the operator U.
  • the operation skill is used to select the mode to be executed by the selection unit 132.
  • the selection unit 132 has one or more auxiliary processes (hereinafter, may be referred to as selection auxiliary processes) from a plurality of auxiliary process candidates for assisting the operation of the operator U as the auxiliary processes to be executed by the auxiliary processing unit 133.
  • Selection auxiliary processes are, for example, a correction process executed in the above-mentioned correction mode, a guidance process executed in the guidance mode, and an automatic process executed in the automatic mode.
  • the selection unit 132 selects, for example, at least one of a correction process for causing the robot control unit 134 to control the robot 2 based on the correction information and a guidance process for displaying auxiliary information for assisting the operation on the operation device 3. ..
  • the selection unit 132 switches between a non-correction mode in which the robot control unit 134 uses the operation information as control information to control the robot and a correction mode in which the robot control unit 134 uses the correction information as control information to control the robot. May be good.
  • the selection unit 132 may display the selected auxiliary processing type on the operation device 3.
  • the operator U can operate the robot 2 while grasping the contents of the auxiliary processing.
  • the operator U can recognize that the cause is the assistance, so that the operation can be performed. It is possible to prevent the person U from feeling uncomfortable.
  • the selection unit 132 notifies the auxiliary processing unit 133 of the selection processing information indicating the selected auxiliary processing. For example, when the correction process is selected, the selection unit 132 notifies the auxiliary processing unit 133 of "01" as the selection processing information, and when the guidance processing is selected, the selection unit 132 notifies the auxiliary processing unit 133 of "10" as the selection processing information. Is notified, and when automatic processing is selected, "11" is notified to the auxiliary processing unit 133 as selection processing information. When the selection unit 132 chooses not to execute the auxiliary processing, the selection unit 132 may notify the auxiliary processing unit 133 of "00" indicating that the normal processing has been selected as the selection processing information. Further details of the operation of the selection unit 132 will be described later.
  • the auxiliary processing unit 133 assists the operator U by executing the auxiliary processing selected by the selection unit 132.
  • the auxiliary processing unit 133 corrects the operation information input from the information acquisition unit 131 to create correction information used as control information for operating the robot 2.
  • the auxiliary processing unit 133 creates correction information by replacing the operation information with predetermined reference control information corresponding to the content of the operation executed by the operator U.
  • the reference control information is, for example, operation information acquired when a person with high operation skill performs an operation.
  • the auxiliary processing unit 133 estimates the content of the operation intended by the operator U based on, for example, the operation information, and associates the content of the estimated operation with the content of the reference control information stored in the storage unit 12. Correct the operation information so that the contents of the operation information come closer.
  • the operation information indicates the operation content of grasping with two fingers.
  • the shape of the object photographed by the robot 2 is spherical, and the reference control information corresponding to the case of grasping the spherical object indicates the operation content of grasping with three fingers.
  • the auxiliary processing unit 133 corrects the operation information based on the reference control information corresponding to the case of grasping the spherical object, and creates the correction information corresponding to the operation of grasping with three fingers.
  • the auxiliary processing unit 133 displays an image for instruction to assist the operator U to be displayed on the operation device 3, a voice for instruction, and a part of the operation device 3 for instruction.
  • Create guidance information that includes at least one of the vibration information to vibrate.
  • the auxiliary processing unit 133 may select one or more images, sounds, and vibration information from a plurality of images, sounds, and vibration information for instruction prepared in advance.
  • the auxiliary processing unit 133 displays an image for instruction to be displayed on the operation device 3 based on at least one of the work content performed by the operator U, the scenario used by the operator U, or the operation information received from the operation device 3. , Voice, create vibration information.
  • the image for instruction is, for example, an image showing a position where each part such as a finger, an arm, a foot, and a head should be moved.
  • the auxiliary processing unit 133 is an image of a color (for example, a color different from the article) indicating a position where the finger should be brought into contact with the image of the article acquired from the robot 2.
  • the instructional voice is a voice that guides the operation method, such as "a little more right" or "the index finger is out of alignment”.
  • the vibration information for guidance is information for giving vibration to the operation device 3 worn by the operator U when, for example, the operator U makes a mistake in the operation.
  • the auxiliary processing unit 133 is for teaching to indicate the position where the finger should be touched, provided that the object to be grasped has a shape that is difficult to grasp when the operation skill of the operator U is high. Create an image.
  • the auxiliary processing unit 133 may determine the content of the instruction image based on the operation information transmitted from the operation device 3.
  • the auxiliary processing unit 133 indicates, for example, an operation to be performed next when it is determined that the movement of the operator U has stopped based on the operation information, or when it is determined that the movement of the operator U has become excessively slow.
  • Create a teaching image The auxiliary processing unit 133 guides the robot 2 to change the operation content when it is determined that the operation content indicated by the operation information is inappropriate based on the operation information and the detection information transmitted from the robot 2. You may create a teaching image for this.
  • the auxiliary processing unit 133 refers to a scenario showing a predetermined operation content and creates control information based on the information included in the scenario.
  • the auxiliary processing unit 133 may specify the state of the robot 2 based on the detection information transmitted from the robot 2, and create control information based on the specified state of the robot 2 and the operation content indicated by the scenario.
  • the auxiliary processing unit 133 determines the reward amount for the work of the operator U based on the content of the selected auxiliary processing, and outputs the determined reward amount to an external device (for example, the information terminal of the operator U). You may.
  • the auxiliary processing unit 133 determines that the smaller the degree of assistance, the higher the reward amount. By determining the reward amount in this way, the auxiliary processing unit 133 can motivate the operator U so that he / she can work without receiving assistance.
  • the robot control unit 134 controls the robot 2 using the control information corresponding to the type of selection assist processing created based on the operation information.
  • the robot control unit 134 transmits the control information generated based on the operation information and notified from the auxiliary processing unit 133 to the robot control unit 134 to the robot 2 while the auxiliary processing unit 133 is executing the selection auxiliary processing. This controls the robot 2.
  • the robot control unit 134 may change the operating speed of the robot 2 when the selection unit 132 selects one or more new auxiliary processes. That is, the robot control unit 134 may change the operating speed of the robot 2 when the content of the auxiliary processing is switched. For example, the robot control unit 134 temporarily stops the operation of the robot 2 at the time of switching the contents of the auxiliary processing. At this time, the robot control unit 134 may make the robot 2 emit a voice saying "switch to automatic operation" to notify people around the robot 2 that the mode is switched. Further, the robot control unit 134 may display the operation device 3 that the mode is switched. By operating the robot control unit 134 in this way, people around the robot 2 can grasp that the operation of the robot 2 changes, so that it is possible to avoid a danger such as a collision with the robot 2. ..
  • the robot control unit 134 may gradually slow down the speed of movement before pausing when switching modes. By operating the robot control unit 134 in this way, it is possible to prevent the robot 2 from dropping an article. Further, in order to prevent the operator U who is visually recognizing the surroundings of the robot 2 through the camera provided on the head of the robot 2 from feeling unwell, even when switching the mode, contrary to the operation of the operator U. The movement of the head may not be paused.
  • the robot control unit 134 may transmit control information to the robot 2 including information for causing the robot 2 to display a display corresponding to the content to be operated by the robot 2. For example, when the right arm of the robot 2 is moved to the right, the robot control unit 134 transmits control information including an instruction to light a light emitting device provided on the right arm of the robot 2. By operating the robot control unit 134 in this way, people around the robot 2 can grasp how the robot 2 moves, so that safety is improved.
  • the robot control unit 134 may transmit control information to the robot 2 including information for causing the robot 2 to display a display indicating which mode it is operating in. For example, the robot control unit 134 lights a blue lamp during operation in the normal mode, lights a red lamp during operation in the automatic mode, and lights a green lamp during operation in the instruction mode. The yellow lamp lights up during operation in the correction mode. By operating the robot control unit 134 in this way, people around the robot 2 can predict what kind of movement the robot 2 can make, so that safety is improved.
  • the robot control unit 134 may control the robot 2 so as to change the angle of view of the image captured by the robot 2 depending on the mode. For example, in the automatic mode, the robot control unit 134 captures an image on the back side of the robot 2, and in the manual mode, it captures a wider range of images than in the automatic mode. This improves operability in the manual mode.
  • the robot control unit 134 before the auxiliary processing unit 133 executes the new auxiliary processing.
  • the robot 2 may be operated so that the posture of the robot 2 matches the posture of the operator U. That is, if the posture of the operator U and the posture of the robot 2 do not match at the timing when the mode is switched, the robot control unit 134 matches the posture of the robot 2 with the posture of the operator U before changing the mode. The robot 2 may be operated so as to do so.
  • the robot control unit 134 causes the operation device 3 to display information for instructing the operator U to stand by in a specific posture when returning to the manual mode after operating in the automatic mode.
  • the robot control unit 134 causes the operation device 3 to display information permitting the restart of the operation.
  • selection operation by selection unit 132 Subsequently, the details of the operation in which the selection unit 132 selects the mode will be described.
  • the selection unit 132 may select a mode by combining various factors described below.
  • the selection unit 132 selects an auxiliary process to be executed, for example, based on the operation skill of the operator U.
  • the selection unit 132 may select an auxiliary process to be executed using the operation skill indicated by the skill information acquired by the information acquisition unit 131, and identifies the operation skill based on the operation information acquired by the information acquisition unit 131.
  • Auxiliary processing may be selected using the specified operation skill.
  • the selection unit 132 may use the operation skill stored in the storage unit 12 in association with the operator U.
  • the selection unit 132 selects the normal mode or the instruction mode when the operator U having a high operation skill operates the robot 2, and corrects when the operator U having a low operation skill operates the robot 2. Select mode or automatic mode.
  • the selection unit 132 may select an auxiliary process to be executed based on the content of the operation indicated by the operation information.
  • the selection unit 132 switches between the non-correction mode and the correction mode, for example, based on the content of the operation indicated by the operation information.
  • the selection unit 132 selects, for example, the non-correction mode for the movement of moving the arm, and selects the correction mode for the movement of grasping an object.
  • the selection unit 132 may select the non-correction mode for the movement of moving the arm in a wide place, and may select the correction mode for the movement of moving the arm in a place where there is an obstacle in the vicinity.
  • the selection unit 132 may select the automatic mode when fine movement is required.
  • the selection unit 132 may specify the operation skill of the operator U for each operation content and select an auxiliary process to be executed based on the operation skill of the operator U for each operation content. For example, when the operator U has a high skill in carrying a box but a low skill in grasping a spherical object, the selection unit 132 selects a normal mode in the operation of carrying the box and has a spherical shape. The correction mode may be selected in the operation of grasping an object.
  • the selection unit 132 may select an auxiliary process to be executed based on the difficulty level determined for each operation content.
  • the selection unit 132 executes auxiliary processing corresponding to the difficulty level of the operation by referring to a table previously stored in the storage unit 12 showing the relationship between the content of the operation and the difficulty level of the operation.
  • the selection unit 132 selects a normal mode while causing the robot 2 to perform an operation of moving an arm to a predetermined position, and then performs an operation of grasping an object having a complicated shape after moving the arm to a predetermined position.
  • the instruction mode or the correction mode is selected while the robot is executed.
  • the selection unit 132 may select the instruction mode, the correction mode, or the automatic mode while moving the arm.
  • the selection unit 132 may select an auxiliary process to be executed based on the operation content indicated by the scenario in which the elapsed time from the reference time and the operation content of the robot are associated with each other.
  • the operator U operates the robot 2 based on the scenario, the operator U needs to perform various types of operations in order to execute various operations included in one scenario.
  • the selection unit 132 switches the operation mode based on the type of each work included in the scenario, the difficulty level of each work, and the like. For example, the selection unit 132 selects a non-correction mode for some operations among a plurality of operations indicated by a plurality of operation information acquired by the information acquisition unit 131, and a correction mode for some other operations. Select. By operating the selection unit 132 in this way, when the scenario includes various operations, the operator U can appropriately perform a series of operations included in the scenario.
  • the information acquisition unit 131 displays auxiliary level information indicating the degree of assistance input in the information terminal after displaying a screen for selecting the degree of assistance on the information terminal as information indicating a plurality of candidates for auxiliary processing. You may get it.
  • the degree of assistance is represented by, for example, the ratio of the time during which the operator U receives the correction process or the instruction process in the scheduled work time.
  • the selection unit 132 inputs the operation information acquired by the information acquisition unit 131 while the operator U performs a predetermined operation into the machine learning model that outputs one or more auxiliary processes when the operation information is input, and the machine learning model.
  • One or more auxiliary processes output by may be selected as auxiliary processes to be executed.
  • the machine learning model is a model learned by using as teacher data data in which a large number of operation information and the contents of auxiliary processing selected for each operation information are associated with each other, for example, the storage unit 12 or the outside. It is stored in the computer.
  • the teacher data is, for example, data in which the operation information acquired when a large number of people have operated the robot 2 in the past and the content of the auxiliary processing selected by the operator U at the time of the operation are associated with each other.
  • the teacher data includes positive teacher data including a combination of operation information and auxiliary processing contents when a good operation result is obtained, and operation information and auxiliary processing contents when a bad operation result is obtained. It may include negative teacher data including a combination with.
  • the selection unit 132 may have a learning processing function for training the machine learning model.
  • the selection unit 132 causes the machine learning model to learn by inputting the operation information and the content of the auxiliary process into the machine learning model each time the content of the auxiliary process is determined.
  • the selection unit 132 transmits the teacher data in which the operation information and the content of the auxiliary processing are associated with the external computer to the external computer via the communication unit 11, thereby transmitting the machine. Let the learning model train.
  • the selection unit 132 may select the operation mode based on the communication quality of the network N.
  • the communication quality of the network N is poor, it is possible that the operation information indicating the content of the operation performed by the operator U may not be normally transmitted to the control device 1. Therefore, the selection unit 132 may select the automatic mode when the quality of communication between the operation device 3 and the control device 1 deteriorates to a level at which the operation of the robot 2 is hindered.
  • the selection unit 132 may change the operation mode based on the communication quality even when the operation mode is set by the operator U. In this case, the selection unit 132 may display to the operation device 3 that the mode has been changed to the automatic mode due to poor communication quality.
  • Control device 2 Robot 3 Operation device 11 Communication unit 12 Storage unit 13 Control unit 21 Communication unit 22 Mechanism unit 23 Sensor unit 24 Storage unit 25 Control unit 31 Operation unit 32 Detection unit 33 Communication unit 34 Display unit 35 Storage unit 36 Control unit 131 Information acquisition unit 132 Selection unit 133 Auxiliary processing unit 134 Robot control unit 251 Instruction reception unit 252 Mechanism control unit 253 Robot status transmission unit 361 State identification unit 362 Instruction transmission unit 363 Status transmission unit 364 Display control unit

Abstract

A control device 1 comprises: an information acquisition section 131 for acquiring operation information indicating the content of an operation by an operator U operating a robot 2; a selection section 132 for selecting one or more assisting processes from among a plurality of candidate assisting processes for assisting in the operation by the operator; an assisting process section 133 for assisting the operator U by executing a selected assisting process being the one or more assisting processes selected by the selection section 132; and a robot control section 134 for controlling the robot 2 using information corresponding to a type of the selected assisting process created on the basis of the operation information while the assisting process section 133 is executing the selected assisting process.

Description

制御装置及び制御方法Control device and control method
　この発明は、ロボットを制御するための制御装置及び制御方法に関する。 The present invention relates to a control device and a control method for controlling a robot.
　ロボットを操作する操作者の姿勢を検出し、検出した姿勢に対応する姿勢にロボットの姿勢を変化させる技術が知られている。特許文献１には、ロボットの操作に習熟していない操作者が適切にロボットを操作できるように補助するための制御装置が開示されている。 A technology is known that detects the posture of an operator who operates a robot and changes the posture of the robot to a posture corresponding to the detected posture. Patent Document 1 discloses a control device for assisting an operator who is not proficient in operating a robot so that the robot can be appropriately operated.
国際公開第２０１８／２１２２２６号International Publication No. 2018/212226
　制御装置によってロボットの操作が補助されることにより、操作者が不適切な操作を行った場合であっても、ロボットを適切に動作させることができる。しかしながら、自らが行った操作内容と異なる動作をロボットが行うことに対して操作者が違和感を抱く場合がある。例えば、操作者の操作スキルが高い場合、自らが行った操作内容と異なる動作をロボットが行うことによって作業効率が低下する場合があるという問題があった。逆に、操作者によっては、補助が足りないと感じる場合があるという問題もあった。 By assisting the operation of the robot by the control device, the robot can be operated appropriately even when the operator performs an inappropriate operation. However, the operator may feel uncomfortable when the robot performs an operation different from the operation content performed by the robot. For example, when the operation skill of the operator is high, there is a problem that the work efficiency may be lowered because the robot performs an operation different from the operation content performed by the operator. On the contrary, there is also a problem that some operators may feel that the assistance is insufficient.
　そこで、本発明はこれらの点に鑑みてなされたものであり、操作者がロボットを操作しやすくすることを目的とする。 Therefore, the present invention has been made in view of these points, and an object of the present invention is to make it easier for the operator to operate the robot.
　本発明の第１の態様の制御装置は、ロボットを操作する操作者による操作の内容を示す操作情報を取得する情報取得部と、前記操作者の操作を補助するための複数の補助処理の候補から、一以上の補助処理を選択する選択部と、前記選択部が選択した前記一以上の補助処理である選択補助処理を実行することにより前記操作者を補助する補助処理部と、前記補助処理部が前記選択補助処理を実行している間に、前記操作情報に基づいて作成された前記選択補助処理の種類に対応する制御情報を用いて前記ロボットを制御するロボット制御部と、を有する。 The control device according to the first aspect of the present invention includes an information acquisition unit that acquires operation information indicating the content of the operation by the operator who operates the robot, and a plurality of auxiliary processing candidates for assisting the operation of the operator. A selection unit that selects one or more auxiliary processes, an auxiliary processing unit that assists the operator by executing the selection auxiliary processing that is the one or more auxiliary processes selected by the selection unit, and the auxiliary processing. It has a robot control unit that controls the robot by using the control information corresponding to the type of the selection assistance process created based on the operation information while the unit is executing the selection assistance process.
　前記選択部は、例えば、前記操作者の操作スキルに基づいて前記選択補助処理を選択する。前記選択部は、前記操作情報が示す操作の内容に基づいて前記選択補助処理を選択してもよい。 The selection unit selects the selection assistance process based on, for example, the operation skill of the operator. The selection unit may select the selection assisting process based on the content of the operation indicated by the operation information.
　前記選択部は、基準時刻からの経過時間とロボットの動作内容とが関連付けられたシナリオが示す前記動作内容に基づいて前記選択補助処理を選択してもよい。 The selection unit may select the selection assisting process based on the operation content indicated by the scenario in which the elapsed time from the reference time and the operation content of the robot are associated with each other.
　前記情報取得部は、前記ロボットに設けられたセンサーが検出した前記ロボットの周辺の状態を示す検出情報をさらに取得し、前記選択部は、前記検出情報に基づいて前記選択補助処理を選択してもよい。 The information acquisition unit further acquires detection information indicating a state around the robot detected by a sensor provided in the robot, and the selection unit selects the selection assist process based on the detection information. May be good.
　前記選択部は、前記検出情報が、前記ロボットが正常に動作していないことを示している場合に、前記選択補助処理を変更してもよい。 The selection unit may change the selection assist process when the detection information indicates that the robot is not operating normally.
　前記情報取得部は、前記操作者又は前記操作者の管理者がアクセス可能な情報端末に、前記複数の補助処理の候補を示す情報を表示させた後に、前記情報端末において選択された候補を示す選択候補情報を取得し、前記選択部は、前記選択候補情報に基づいて、前記操作者又は前記操作者の管理者により指定された前記一以上の補助処理を前記選択補助処理として選択してもよい。 The information acquisition unit displays information indicating candidates for the plurality of auxiliary processes on an information terminal accessible to the operator or the administrator of the operator, and then indicates the candidates selected in the information terminal. Even if the selection candidate information is acquired and the selection unit selects the one or more auxiliary processes designated by the operator or the administrator of the operator as the selection auxiliary process based on the selection candidate information. Good.
　前記情報取得部は、前記複数の補助処理の候補を示す情報として、補助の度合いを選択するための画面を前記情報端末に表示させた後に、前記情報端末において入力された補助の度合いを示す補助レベル情報を取得し、前記選択部は、前記補助レベル情報に基づいて前記選択補助処理の内容を決定してもよい。 The information acquisition unit displays an auxiliary screen for selecting the degree of assistance as information indicating the plurality of auxiliary processing candidates on the information terminal, and then indicates the degree of assistance input in the information terminal. The level information may be acquired, and the selection unit may determine the content of the selection auxiliary processing based on the auxiliary level information.
　前記補助処理部は、前記操作情報を補正することにより前記ロボットに送信する補正情報を作成し、前記選択部は、前記選択補助処理として、前記ロボット制御部に前記補正情報を前記制御情報として用いて前記ロボットを制御させる補正処理、及び操作を補助するための情報を前記操作者が操作に用いる操作装置に表示させる指導処理の少なくともいずれかを選択してもよい。 The auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information, and the selection unit uses the correction information as the control information in the robot control unit as the selection auxiliary processing. You may select at least one of the correction process for controlling the robot and the instruction process for displaying the information for assisting the operation on the operation device used by the operator for the operation.
　前記補助処理部は、前記操作情報を補正することにより前記ロボットに送信する補正情報を作成し、前記選択部は、前記ロボット制御部に前記操作情報を前記制御情報として用いて前記ロボットを制御させる非補正モードと、前記選択補助処理として前記ロボット制御部に前記補正情報を前記制御情報として用いて前記ロボットを制御させる補正モードとを切り替えてもよい。 The auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information, and the selection unit causes the robot control unit to control the robot by using the operation information as the control information. You may switch between the non-correction mode and the correction mode in which the robot control unit controls the robot by using the correction information as the control information as the selection assist process.
　前記補助処理部は、前記補正モードにおいて、前記操作者が実行する操作の内容に対応する予め定められた基準制御情報に前記操作情報を置換することにより前記補正情報を作成してもよい。 In the correction mode, the auxiliary processing unit may create the correction information by substituting the operation information with predetermined reference control information corresponding to the content of the operation executed by the operator.
　前記選択部は、前記情報取得部が取得した複数の前記操作情報が示す複数の操作のうち一部の操作に対して前記非補正モードを選択し、他の一部の操作に対しては前記補正モードを選択してもよい。 The selection unit selects the non-correction mode for some of the operations indicated by the plurality of operation information acquired by the information acquisition unit, and the selection unit selects the non-correction mode for some of the operations. The correction mode may be selected.
　前記選択部は、前記操作情報を入力すると前記一以上の補助処理を出力する機械学習モデルに、前記操作者が所定の操作を行う間に前記情報取得部が取得した前記操作情報を入力し、前記機械学習モデルが出力した前記一以上の補助処理を選択してもよい。 The selection unit inputs the operation information acquired by the information acquisition unit into the machine learning model that outputs one or more auxiliary processes when the operation information is input, while the operator performs a predetermined operation. One or more auxiliary processes output by the machine learning model may be selected.
　前記選択部は、前記操作者が与えられた作業を行う前に前記所定の操作として実行される、前記作業の内容に対応する試験用の操作を行う間に前記情報取得部が取得した前記操作情報を前記機械学習モデルに入力することにより前記一以上の補助処理を選択してもよい。 The selection unit is executed as the predetermined operation before the operator performs a given operation, and the operation acquired by the information acquisition unit while performing a test operation corresponding to the content of the operation. One or more auxiliary processes may be selected by inputting information into the machine learning model.
　前記選択部は、前記選択補助処理の種別を前記操作者が操作に用いる操作装置に表示させてもよい。 The selection unit may display the type of the selection assist processing on the operation device used by the operator for the operation.
　前記ロボット制御部は、前記選択部が新たな前記一以上の補助処理を選択した場合に、前記ロボットの動作速度を変化させてもよい。 The robot control unit may change the operating speed of the robot when the selection unit selects a new one or more auxiliary processes.
　前記ロボット制御部は、前記選択部が新たな前記一以上の補助処理を選択した時点で前記操作者の姿勢と前記ロボットの姿勢とが一致していない場合、前記補助処理部が新たな前記一以上の補助処理を実行する前に、前記ロボットの姿勢が前記操作者の姿勢と一致するように前記ロボットを動作させてもよい。 In the robot control unit, if the posture of the operator and the posture of the robot do not match when the selection unit selects a new one or more auxiliary processes, the auxiliary processing unit is a new one. Before executing the above auxiliary processing, the robot may be operated so that the posture of the robot matches the posture of the operator.
　本発明の第２の態様の制御方法は、コンピュータが実行する、ロボットを操作する操作者による操作の内容を示す操作情報を取得するステップと、前記操作者の操作を補助するための複数の補助処理の候補から、一以上の補助処理を選択するステップと、選択した前記一以上の補助処理である選択補助処理を実行することにより前記操作者を補助するステップと、前記選択補助処理を実行している間に、前記操作情報に基づいて作成された前記選択補助処理の種類に対応する制御情報を用いて前記ロボットを制御するステップと、を有する。 The control method of the second aspect of the present invention includes a step of acquiring operation information indicating the content of the operation by the operator who operates the robot, which is executed by the computer, and a plurality of assists for assisting the operation of the operator. A step of selecting one or more auxiliary processes from the processing candidates, a step of assisting the operator by executing the selected auxiliary process, which is the one or more auxiliary processes, and the selection auxiliary process are executed. In the meantime, it has a step of controlling the robot by using the control information corresponding to the type of the selection assisting process created based on the operation information.
　本発明によれば、操作者がロボットを操作しやすくすることができるという効果を奏する。 According to the present invention, there is an effect that the operator can easily operate the robot.
制御装置が通常モードで動作する場合のロボット制御システムの動作を模式的に示す図である。It is a figure which shows typically the operation of the robot control system when the control device operates in a normal mode. 制御装置が補正モードで動作する場合のロボット制御システムの動作を模式的に示す図である。It is a figure which shows typically the operation of the robot control system when the control device operates in a correction mode. 制御装置が指導モードで動作する場合のロボット制御システムの動作を模式的に示す図である。It is a figure which shows typically the operation of the robot control system when the control device operates in a guidance mode. 制御装置が自動モードで動作する場合のロボット制御システムの動作を模式的に示す図である。It is a figure which shows typically the operation of the robot control system when the control device operates in an automatic mode. 制御装置の機能構成を示す図である。It is a figure which shows the functional structure of a control device. ロボットの機能構成を示す図である。It is a figure which shows the functional structure of a robot. 操作装置の機能構成を示す図である。It is a figure which shows the functional structure of the operation device. 動作モードを選択するための操作画面の一例を示す図である。It is a figure which shows an example of the operation screen for selecting an operation mode. 補助の度合いを入力するための操作画面の一例を示す図である。It is a figure which shows an example of the operation screen for inputting the degree of assistance. 補正処理の度合いと指導処理の度合いとを設定するための操作画面の一例を示す図である。It is a figure which shows an example of the operation screen for setting the degree of correction processing and the degree of instruction processing.
［ロボット制御システムＳの概要］
　図１から図４は、ロボット制御システムＳの概要を説明するための図である。ロボット制御システムＳは、制御装置１と、ロボット２と、操作装置３とを有する。ロボット制御システムＳは、ロボット２を操作する操作者Ｕが、ロボット２から離れた場所でロボット２を操作することができるシステムである。 [Overview of Robot Control System S]
1 to 4 are diagrams for explaining the outline of the robot control system S. The robot control system S includes a control device 1, a robot 2, and an operation device 3. The robot control system S is a system in which an operator U who operates the robot 2 can operate the robot 2 at a place away from the robot 2.
　操作者Ｕは、操作装置３を操作することによりロボット２を操作する。具体的には、操作者Ｕが操作装置３を操作すると、操作装置３は、ネットワークＮを介して、操作された内容を示す操作情報を制御装置１に送信する。制御装置１は、操作情報に基づく制御情報をロボット２に送信することによりロボット２を遠隔制御するコンピュータである。 The operator U operates the robot 2 by operating the operation device 3. Specifically, when the operator U operates the operation device 3, the operation device 3 transmits the operation information indicating the operated content to the control device 1 via the network N. The control device 1 is a computer that remotely controls the robot 2 by transmitting control information based on the operation information to the robot 2.
　操作装置３は、操作者Ｕがロボット２を遠隔操作するためのデバイスである。操作装置３は、例えば、操作者Ｕが装着する操作用具と、操作者Ｕの操作用具の状態を制御装置１に通知する通信ユニットを含むコックピットとを有する。操作装置３は、操作用具を装着した状態の操作者Ｕの動作を特定し、特定した動作を示す操作情報を制御装置１に送信する。操作装置３においては、例えば、操作者Ｕが装着した操作用具が有するセンサーから受信した信号に基づいて、コックピットが操作者Ｕの動作を特定する。コックピットは、赤外光を操作者Ｕに照射し、照射した赤外光の反射光に基づいて操作者Ｕの動作を特定してもよい。コックピットは、カメラで操作者Ｕを撮影した画像を解析することにより操作者Ｕの動作を特定してもよい。 The operation device 3 is a device for the operator U to remotely control the robot 2. The operation device 3 has, for example, an operation tool worn by the operator U and a cockpit including a communication unit for notifying the control device 1 of the state of the operation tool of the operator U. The operation device 3 identifies the operation of the operator U with the operation tool attached, and transmits the operation information indicating the specified operation to the control device 1. In the operation device 3, for example, the cockpit identifies the operation of the operator U based on the signal received from the sensor of the operation tool worn by the operator U. The cockpit may irradiate the operator U with infrared light and specify the operation of the operator U based on the reflected light of the irradiated infrared light. The cockpit may identify the operation of the operator U by analyzing an image of the operator U taken by the camera.
　また、操作装置３は、ネットワークＮを介してロボット２から受信した撮像画像及び知覚情報を操作者Ｕが装着している操作用具に通知してもよい。撮像画像は、ロボット２に設けられた撮像装置がロボット２の周囲を撮影することにより生成された画像であり、操作者Ｕが装着する操作用具に含まれるゴーグルのディスプレイ、又は操作者Ｕが目に装着するコンタクトレンズなどに表示される。知覚情報は、ロボット２に設けられたセンサーが検知した状態を示す情報であり、操作者Ｕが手、足、及び腕等に装着する操作用具は、知覚情報に基づいて温度を変化させたり振動したりする。これらの構成により、操作者Ｕは、ロボット２から離れた場所において、あたかもロボット２に入っているかのような感覚でロボット２の周辺の環境を体感することができる。 Further, the operating device 3 may notify the operating tool worn by the operator U of the captured image and the perceptual information received from the robot 2 via the network N. The captured image is an image generated by the imaging device provided in the robot 2 taking a picture of the surroundings of the robot 2, and the display of the goggles included in the operation tool worn by the operator U or the eyes of the operator U. It is displayed on contact lenses worn on. The perceptual information is information indicating a state detected by a sensor provided in the robot 2, and the operating tool worn by the operator U on the hand, foot, arm, etc. changes the temperature or vibrates based on the perceptual information. To do. With these configurations, the operator U can experience the environment around the robot 2 at a place away from the robot 2 as if he / she is in the robot 2.
　制御装置１は、さまざまな形態で操作者Ｕの操作を補助することにより、操作者Ｕの操作スキルによらず操作者Ｕがロボット２を適切に操作することを可能にする。制御装置１は、例えば、（１）通常モード、（２）補正モード、（３）指導モード、（４）自動モードのいずれかのモードで動作する。制御装置１は、これらの複数のモードのいずれかを組合せて動作してもよい。制御装置１は、例えば、操作者Ｕが行う一連の作業のうち、一部の作業に対して通常モードで動作し、他の一部の作業に対して補正モードで動作する。 By assisting the operation of the operator U in various forms, the control device 1 enables the operator U to appropriately operate the robot 2 regardless of the operation skill of the operator U. The control device 1 operates in any of (1) normal mode, (2) correction mode, (3) instruction mode, and (4) automatic mode, for example. The control device 1 may operate in combination with any of these plurality of modes. For example, the control device 1 operates in the normal mode for some of the series of operations performed by the operator U, and operates in the correction mode for some of the other operations.
　図１は、制御装置１が通常モードで動作する場合のロボット制御システムＳの動作を模式的に示す図である。通常モードにおいて、制御装置１は、操作装置３から受信した操作情報を実質的に変更することなく制御情報としてロボット２に送信する。具体的には、操作装置３が操作情報を制御装置１に送信し（図１における（１））、制御装置１が操作情報を受信すると（図１における（２））、制御装置１は、ロボット２を動作させるための制御情報として、受信した操作情報をロボット２に送信する（図１における（３））。ロボット２は、制御装置１から受信した操作情報に基づいて動作する（図１における（４））。通常モードは、操作者Ｕが操作したとおりにロボット２が動作するので、操作者Ｕが十分な操作スキルを有する場合に好適である。 FIG. 1 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the normal mode. In the normal mode, the control device 1 transmits the operation information received from the operation device 3 to the robot 2 as control information without substantially changing it. Specifically, when the operation device 3 transmits the operation information to the control device 1 ((1) in FIG. 1) and the control device 1 receives the operation information ((2) in FIG. 1), the control device 1 receives the operation information. The received operation information is transmitted to the robot 2 as the control information for operating the robot 2 ((3) in FIG. 1). The robot 2 operates based on the operation information received from the control device 1 ((4) in FIG. 1). Since the robot 2 operates as operated by the operator U, the normal mode is suitable when the operator U has sufficient operation skills.
　図２は、制御装置１が補正モードで動作する場合のロボット制御システムＳの動作を模式的に示す図である。補正モードにおいて、制御装置１は、操作装置３から受信した操作情報を補正することにより補正情報を作成し、補正情報を制御情報としてロボット２に送信する。補正情報の詳細については後述するが、補正情報は、操作情報が適切でない場合に、適切な内容に変更するための情報である。 FIG. 2 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the correction mode. In the correction mode, the control device 1 creates correction information by correcting the operation information received from the operation device 3, and transmits the correction information to the robot 2 as control information. The details of the correction information will be described later, but the correction information is information for changing the operation information to an appropriate content when the operation information is not appropriate.
　図２に示すように、操作装置３が操作情報を制御装置１に送信し（図２における（１））、制御装置１が操作情報を受信すると、制御装置１は、操作情報を補正して補正情報を作成する（図２における（２））。制御装置１は、補正情報をロボット２に送信し（図２における（３））、ロボット２は補正情報に基づいて動作する（図２における（４））。補正モードは、操作者Ｕが正確な操作をすることができない場合に有効である。 As shown in FIG. 2, when the operation device 3 transmits the operation information to the control device 1 ((1) in FIG. 2) and the control device 1 receives the operation information, the control device 1 corrects the operation information. Correction information is created ((2) in FIG. 2). The control device 1 transmits the correction information to the robot 2 ((3) in FIG. 2), and the robot 2 operates based on the correction information ((4) in FIG. 2). The correction mode is effective when the operator U cannot perform an accurate operation.
　図３は、制御装置１が指導モードで動作する場合のロボット制御システムＳの動作を模式的に示す図である。指導モードにおいて、制御装置１は、操作者Ｕが行うべき操作方法を示す画像又はテキストの少なくともいずれかを含む補助情報を操作装置３に送信する（図３における（１））。操作装置３は、受信した補助情報を操作者Ｕに提示する（図３における（２））。制御装置１は、例えば操作者Ｕが装着した操作装置３のディスプレイ又は操作者Ｕが視認可能な情報端末に補助情報を表示させることにより、補助情報を操作者Ｕに提示する。 FIG. 3 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the instruction mode. In the instruction mode, the control device 1 transmits auxiliary information including at least one of an image or a text indicating an operation method to be performed by the operator U to the operation device 3 ((1) in FIG. 3). The operating device 3 presents the received auxiliary information to the operator U ((2) in FIG. 3). The control device 1 presents the auxiliary information to the operator U, for example, by displaying the auxiliary information on the display of the operation device 3 worn by the operator U or the information terminal visible to the operator U.
　操作装置３は、補助情報を参照して操作者Ｕが操作した内容を示す操作情報を制御装置１に送信する（図３における（３））。制御装置１が操作情報を受信すると（図３における（４））、制御装置１は、ロボット２を動作させるための制御情報として、受信した操作情報をロボット２に送信する（図３における（５））。ロボット２は、制御装置１から受信した操作情報に基づいて動作する（図３における（６））。制御装置１は、操作情報を補正した補正情報を制御情報としてロボット２に送信してもよい。指導モードは、操作者Ｕが操作方法を熟知していない場合に有効である。 The operation device 3 transmits the operation information indicating the contents operated by the operator U to the control device 1 with reference to the auxiliary information ((3) in FIG. 3). When the control device 1 receives the operation information ((4) in FIG. 3), the control device 1 transmits the received operation information to the robot 2 as control information for operating the robot 2 ((5) in FIG. 3). )). The robot 2 operates based on the operation information received from the control device 1 ((6) in FIG. 3). The control device 1 may transmit the correction information obtained by correcting the operation information to the robot 2 as control information. The instruction mode is effective when the operator U is not familiar with the operation method.
　図４は、制御装置１が自動モードで動作する場合のロボット制御システムＳの動作を模式的に示す図である。自動モードにおいて、制御装置１は、予め定められたシナリオに基づいてロボット２を動作させるべく制御情報を自動的に作成し、作成した制御情報をロボット２に送信する。シナリオは、基準時刻からの経過時間とロボット２の動作内容とが関連付けられた情報である。基準時刻は、例えばロボット２が動作を開始した時刻であるが、これに限らず、ロボット２を操作するユーザが所定の操作をした時刻のような他の時刻であってもよい。 FIG. 4 is a diagram schematically showing the operation of the robot control system S when the control device 1 operates in the automatic mode. In the automatic mode, the control device 1 automatically creates control information for operating the robot 2 based on a predetermined scenario, and transmits the created control information to the robot 2. The scenario is information in which the elapsed time from the reference time and the operation content of the robot 2 are associated with each other. The reference time is, for example, the time when the robot 2 starts operating, but is not limited to this, and may be another time such as the time when the user who operates the robot 2 performs a predetermined operation.
　図４に示すように、制御装置１は、例えば操作者Ｕ又は操作者Ｕを管理する管理者からの指示に基づいて、操作者Ｕが行う作業に対応するシナリオを取得する（図４における（１））。制御装置１は、例えば他のコンピュータからシナリオを取得してもよく、制御装置１内の記憶媒体からシナリオを取得してもよい。制御装置１は、シナリオに基づいて制御情報を作成する（図４における（２））。ロボット２は、制御情報に基づいて動作する（図４における（３））。自動モードは、操作者Ｕのスキルが低く、かつロボット２を動作させるパターンが決まっている場合に有効である。 As shown in FIG. 4, the control device 1 acquires a scenario corresponding to the work performed by the operator U, for example, based on an instruction from the operator U or the administrator who manages the operator U ((in FIG. 4). 1)). The control device 1 may acquire the scenario from another computer, for example, or may acquire the scenario from the storage medium in the control device 1. The control device 1 creates control information based on the scenario ((2) in FIG. 4). The robot 2 operates based on the control information ((3) in FIG. 4). The automatic mode is effective when the skill of the operator U is low and the pattern for operating the robot 2 is determined.
［制御装置１の構成］
　図５は、制御装置１の機能構成を示す図である。制御装置１は、通信部１１と、記憶部１２と、制御部１３とを有する。制御部１３は、情報取得部１３１と、選択部１３２と、補助処理部１３３と、ロボット制御部１３４とを有する。 [Configuration of control device 1]
FIG. 5 is a diagram showing a functional configuration of the control device 1. The control device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 includes an information acquisition unit 131, a selection unit 132, an auxiliary processing unit 133, and a robot control unit 134.
　通信部１１は、ネットワークＮを介してデータを送受信するための通信インターフェースであり、例えばＬＡＮ（Local Area Network）コントローラを含む。通信部１１は、ロボット２から受信したロボット状態情報、及び操作装置３から受信した操作情報を情報取得部１３１に入力する。 The communication unit 11 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a LAN (Local Area Network) controller. The communication unit 11 inputs the robot state information received from the robot 2 and the operation information received from the operation device 3 to the information acquisition unit 131.
　記憶部１２は、ＲＯＭ（Read Only Memory）、ＲＡＭ（Random Access Memory）及びハードディスク等の記憶媒体を含む。記憶部１２は、制御部１３が実行するプログラムを記憶する。また、記憶部１２は、補助モードにおいて使用される補助情報を記憶する。記憶部１２は、補正モードにおいて使用される基準制御情報を記憶してもよい。基準制御情報の詳細については後述する。 The storage unit 12 includes a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 12 stores a program executed by the control unit 13. In addition, the storage unit 12 stores auxiliary information used in the auxiliary mode. The storage unit 12 may store the reference control information used in the correction mode. Details of the reference control information will be described later.
　制御部１３は、例えばＣＰＵ（Central Processing Unit）を含み、記憶部１２に記憶されたプログラムを実行することにより、情報取得部１３１、選択部１３２、補助処理部１３３及びロボット制御部１３４として機能する。情報取得部１３１、選択部１３２、補助処理部１３３及びロボット制御部１３４の詳細については後述する。 The control unit 13 includes, for example, a CPU (Central Processing Unit), and functions as an information acquisition unit 131, a selection unit 132, an auxiliary processing unit 133, and a robot control unit 134 by executing a program stored in the storage unit 12. .. Details of the information acquisition unit 131, the selection unit 132, the auxiliary processing unit 133, and the robot control unit 134 will be described later.
［ロボット２の機能構成］
　図６は、ロボット２の機能構成を示す図である。ロボット２は、通信部２１と、機構部２２と、センサー部２３と、記憶部２４と、制御部２５とを有する。制御部２５は、指示受信部２５１と、機構制御部２５２と、ロボット状態送信部２５３とを有する。 [Functional configuration of robot 2]
FIG. 6 is a diagram showing a functional configuration of the robot 2. The robot 2 has a communication unit 21, a mechanism unit 22, a sensor unit 23, a storage unit 24, and a control unit 25. The control unit 25 includes an instruction receiving unit 251, a mechanism control unit 252, and a robot state transmitting unit 253.
　通信部２１は、ネットワークＮを介してデータを送受信するための通信インターフェースであり、例えば無線通信コントローラを含む。通信部２１は、ネットワークＮを介して、操作装置３から送信された制御情報を受信する。また、通信部２１は、ネットワークＮを介して、ロボット２の状態を示すロボット状態情報を制御装置１に送信する。 The communication unit 21 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a wireless communication controller. The communication unit 21 receives the control information transmitted from the operation device 3 via the network N. Further, the communication unit 21 transmits the robot state information indicating the state of the robot 2 to the control device 1 via the network N.
　機構部２２は、ロボット２の各部のアクチュエータを含む。機構部２２は、例えば、指、手、足、首、頭等の各部の関節に設けられたアクチュエータのように、制御部２５の制御により動く機構部材を含む。 The mechanical unit 22 includes actuators for each part of the robot 2. The mechanism unit 22 includes a mechanism member that moves under the control of the control unit 25, such as an actuator provided at a joint of each part such as a finger, a hand, a foot, a neck, and a head.
　センサー部２３は、ロボット２の各部に設けられており、各種の状態を検知するためのデバイスを含む。センサー部２３は、例えばロボット２の目に設けられたカメラ、ロボット２の側頭部等に設けられたマイクロホン、又はロボット２の指に設けられた温度センサー及び圧力センサーである。センサー部２３は、各センサーが検出した状態を示す信号をロボット状態送信部２５３に通知する。 The sensor unit 23 is provided in each unit of the robot 2 and includes a device for detecting various states. The sensor unit 23 is, for example, a camera provided in the eyes of the robot 2, a microphone provided in the temporal region of the robot 2, or a temperature sensor and a pressure sensor provided in the fingers of the robot 2. The sensor unit 23 notifies the robot state transmission unit 253 of a signal indicating the state detected by each sensor.
　記憶部２４は、ＲＯＭ、ＲＡＭ及びハードディスク等の記憶媒体を含む。記憶部２４は、制御部２５が実行するプログラムを記憶する。
　制御部２５は、例えばＣＰＵを有しており、記憶部２４に記憶されたプログラムを実行することにより、指示受信部２５１、機構制御部２５２及びロボット状態送信部２５３として機能する。 The storage unit 24 includes a storage medium such as a ROM, a RAM, and a hard disk. The storage unit 24 stores a program executed by the control unit 25.
The control unit 25 has, for example, a CPU, and functions as an instruction receiving unit 251, a mechanism control unit 252, and a robot state transmitting unit 253 by executing a program stored in the storage unit 24.
　指示受信部２５１は、制御装置１が送信した制御情報を、通信部２１を介して受信する。指示受信部２５１は、受信した制御情報を機構制御部２５２に通知する。 The instruction receiving unit 251 receives the control information transmitted by the control device 1 via the communication unit 21. The instruction receiving unit 251 notifies the mechanism control unit 252 of the received control information.
　機構制御部２５２は、指示受信部２５１から通知された制御情報に基づいて機構部２２に含まれる各アクチュエータを制御する。機構制御部２５２は、例えば手を右に動かすという指示を含む制御情報を取得すると、手に設けられたアクチュエータを制御することにより、ロボット２の手を右に動かす。機構制御部２５２は、機構部２２を動かした後の各アクチュエータの位置を検出し、検出した位置をロボット状態送信部２５３に通知してもよい。 The mechanism control unit 252 controls each actuator included in the mechanism unit 22 based on the control information notified from the instruction receiving unit 251. When the mechanism control unit 252 acquires control information including an instruction to move the hand to the right, for example, the mechanism control unit 252 moves the hand of the robot 2 to the right by controlling the actuator provided in the hand. The mechanism control unit 252 may detect the position of each actuator after moving the mechanism unit 22 and notify the robot state transmission unit 253 of the detected position.
　ロボット状態送信部２５３は、センサー部２３が検出した信号、及び機構制御部２５２から通知された機構部２２の状態に基づいて、ロボット２の状態を示すロボット状態情報を生成する。ロボット状態送信部２５３は、生成したロボット状態情報を、通信部２１を介して制御装置１に送信する。 The robot state transmission unit 253 generates robot state information indicating the state of the robot 2 based on the signal detected by the sensor unit 23 and the state of the mechanism unit 22 notified from the mechanism control unit 252. The robot state transmission unit 253 transmits the generated robot state information to the control device 1 via the communication unit 21.
［操作装置３の機能構成］
　図７は、操作装置３の機能構成を示す図である。操作装置３は、操作部３１と、検出部３２と、通信部３３と、表示部３４と、記憶部３５と、制御部３６を有する。制御部３６は、状態特定部３６１と、指示送信部３６２と、状態送信部３６３と、表示制御部３６４とを有する。 [Functional configuration of operating device 3]
FIG. 7 is a diagram showing a functional configuration of the operating device 3. The operation device 3 includes an operation unit 31, a detection unit 32, a communication unit 33, a display unit 34, a storage unit 35, and a control unit 36. The control unit 36 includes a state specifying unit 361, an instruction transmitting unit 362, a state transmitting unit 363, and a display control unit 364.
　操作部３１は、操作者Ｕがロボット２を操作するためのデバイスであり、例えば、操作者Ｕが装着する部材、及び操作者Ｕが操作するボタン又はジョイスティックを含む。操作部３１は、検出した操作者Ｕが行った操作内容を状態特定部３６１に通知する。 The operation unit 31 is a device for the operator U to operate the robot 2, and includes, for example, a member worn by the operator U and a button or a joystick operated by the operator U. The operation unit 31 notifies the state specifying unit 361 of the operation content performed by the detected operator U.
　検出部３２は、操作者Ｕの状態を検出するためのセンサーを含む。検出部３２は、例えば操作者Ｕの瞳孔の位置を検出するカメラ、操作者Ｕの発汗状態を検出する湿度センサー、操作者Ｕの心拍数を検出する脈拍センサー、操作者Ｕの血圧を検出する血圧センサー、操作者Ｕの呼吸数若しくは呼気成分を検出する呼吸センサー、又は操作者Ｕの各部の動きを検出する加速度センサーを含む。検出部３２は、検出した操作者Ｕの状態を状態特定部３６１に通知する。 The detection unit 32 includes a sensor for detecting the state of the operator U. The detection unit 32 detects, for example, a camera that detects the position of the pupil of the operator U, a humidity sensor that detects the sweating state of the operator U, a pulse sensor that detects the heart rate of the operator U, and a blood pressure of the operator U. It includes a blood pressure sensor, a breathing sensor that detects the respiratory rate or exhaled component of the operator U, or an acceleration sensor that detects the movement of each part of the operator U. The detection unit 32 notifies the state identification unit 361 of the detected state of the operator U.
　通信部３３は、ネットワークＮを介してデータを送受信するための通信インターフェースであり、例えば無線通信コントローラを含む。通信部３３は、ネットワークＮを介して、操作部３１により検出された操作者Ｕの操作内容に基づいて制御部３６が生成したロボット２に対する制御情報をロボット２に送信する。また、通信部３３は、ネットワークＮを介して、操作者Ｕの状態を示す操作者状態情報を制御装置１に送信する。 The communication unit 33 is a communication interface for transmitting and receiving data via the network N, and includes, for example, a wireless communication controller. The communication unit 33 transmits the control information for the robot 2 generated by the control unit 36 to the robot 2 based on the operation content of the operator U detected by the operation unit 31 via the network N. Further, the communication unit 33 transmits the operator status information indicating the status of the operator U to the control device 1 via the network N.
　表示部３４は、操作者Ｕに視認させる情報を表示するディスプレイである。表示部３４は、例えば、操作者Ｕが装着するゴーグルに設けられている。 The display unit 34 is a display that displays information to be visually recognized by the operator U. The display unit 34 is provided on, for example, goggles worn by the operator U.
　記憶部３５は、ＲＯＭ、ＲＡＭ及びハードディスク等の記憶媒体を含む。記憶部３５は、制御部３６が実行するプログラムを記憶する。 The storage unit 35 includes a storage medium such as a ROM, RAM, and a hard disk. The storage unit 35 stores a program executed by the control unit 36.
　制御部３６は、例えばＣＰＵを有しており、記憶部３５に記憶されたプログラムを実行することにより、状態特定部３６１、指示送信部３６２、状態送信部３６３及び表示制御部３６４として機能する。 The control unit 36 has, for example, a CPU, and functions as a state identification unit 361, an instruction transmission unit 362, a state transmission unit 363, and a display control unit 364 by executing a program stored in the storage unit 35.
　状態特定部３６１は、操作部３１から入力された操作内容を示す信号に基づいて操作者Ｕが行った操作内容を特定し、検出部３２から入力された操作者Ｕの状態を示す信号に基づいて操作者Ｕの状態を特定する。状態特定部３６１は、操作部３１から入力された信号、及び検出部３２から入力された信号の両方に基づいて操作者Ｕの状態を特定してもよい。状態特定部３６１は、特定した操作内容を指示送信部３６２に通知し、操作者Ｕの状態を状態送信部３６３に通知する。 The state specifying unit 361 identifies the operation content performed by the operator U based on the signal indicating the operation content input from the operation unit 31, and is based on the signal indicating the state of the operator U input from the detection unit 32. To identify the state of the operator U. The state specifying unit 361 may specify the state of the operator U based on both the signal input from the operation unit 31 and the signal input from the detection unit 32. The state specifying unit 361 notifies the instruction transmitting unit 362 of the specified operation content, and notifies the state transmitting unit 363 of the state of the operator U.
　指示送信部３６２は、状態特定部３６１から通知された操作内容に基づいて、ロボット２を動作させるための操作情報を生成する。指示送信部３６２は、生成した操作情報を、通信部３３を介して制御装置１に送信する。 The instruction transmitting unit 362 generates operation information for operating the robot 2 based on the operation content notified from the state specifying unit 361. The instruction transmission unit 362 transmits the generated operation information to the control device 1 via the communication unit 33.
　状態送信部３６３は、状態特定部３６１から通知された操作者Ｕの状態に基づいて、操作者Ｕの状態を示す操作者状態情報を生成する。状態送信部３６３は、生成したロボット状態情報を、通信部３３を介して制御装置１に送信する。 The state transmission unit 363 generates operator state information indicating the state of the operator U based on the state of the operator U notified from the state identification unit 361. The state transmission unit 363 transmits the generated robot state information to the control device 1 via the communication unit 33.
　表示制御部３６４は、表示部３４に表示させる表示用データを作成する。表示制御部３６４は、例えばロボット２において撮影された画像を表示部３４に表示させる。また、表示制御部３６４は、制御装置１から送信された、操作者Ｕに視認させるための補助情報を表示部３４に表示させる。 The display control unit 364 creates display data to be displayed on the display unit 34. The display control unit 364 causes the display unit 34 to display, for example, an image captured by the robot 2. Further, the display control unit 364 causes the display unit 34 to display the auxiliary information transmitted from the control device 1 for the operator U to visually recognize.
［制御部１３の詳細］
　以下、図５を参照しながら、制御部１３の詳細について説明する。
　情報取得部１３１は、ロボット２を操作する操作者Ｕによる操作の内容を示す操作情報を操作装置３から取得する。情報取得部１３１は、操作装置３と通信可能な他の装置を介して操作情報を取得してもよい。 [Details of control unit 13]
Hereinafter, the details of the control unit 13 will be described with reference to FIG.
The information acquisition unit 131 acquires operation information indicating the content of the operation by the operator U who operates the robot 2 from the operation device 3. The information acquisition unit 131 may acquire operation information via another device capable of communicating with the operation device 3.
　操作情報は、ロボット２の各部を移動させる量、移動させる向き、又は移動させる速度に対応する数値を含む情報である。操作情報は、例えば手、指、足、頭等の操作者Ｕの各部が移動した量、各部が移動した向き、又は移動した速度のように、ロボット２の動作に関連する操作者Ｕの動きを示す情報である。操作情報は、操作者Ｕにより操作装置３又は他の装置に入力された数値又はテキストであってもよい。情報取得部１３１は、取得した操作情報を選択部１３２及び補助処理部１３３に入力する。 The operation information is information including numerical values corresponding to the amount of movement of each part of the robot 2, the direction of movement, or the speed of movement. The operation information is the movement of the operator U related to the operation of the robot 2, such as the amount of movement of each part of the operator U such as a hand, finger, foot, or head, the direction in which each part is moved, or the speed of movement. It is information indicating. The operation information may be a numerical value or text input to the operation device 3 or another device by the operator U. The information acquisition unit 131 inputs the acquired operation information to the selection unit 132 and the auxiliary processing unit 133.
　情報取得部１３１は、ロボット２に設けられたセンサーが検出したロボットの周辺の状態を示す検出情報をさらに取得してもよい。センサーは、例えば撮像素子、温度センサー、圧力センサー、及び加速度センサーである。情報取得部１３１は、取得した検出情報を選択部１３２及び補助処理部１３３に入力する。 The information acquisition unit 131 may further acquire detection information indicating a state around the robot detected by a sensor provided in the robot 2. The sensors are, for example, an image sensor, a temperature sensor, a pressure sensor, and an acceleration sensor. The information acquisition unit 131 inputs the acquired detection information to the selection unit 132 and the auxiliary processing unit 133.
　さらに、情報取得部１３１は、他の装置から各種の情報を取得してもよい。情報取得部１３１は、例えば、操作者Ｕの管理者が使用するコンピュータから、操作者Ｕの操作スキルを示すスキル情報を取得する。操作スキルは、選択部１３２が実行するモードを選択するために使用される。 Further, the information acquisition unit 131 may acquire various types of information from other devices. The information acquisition unit 131 acquires skill information indicating the operation skill of the operator U from, for example, a computer used by the administrator of the operator U. The operation skill is used to select the mode to be executed by the selection unit 132.
　選択部１３２は、補助処理部１３３に実行させる補助処理として、操作者Ｕの操作を補助するための複数の補助処理の候補から、一以上の補助処理（以下、選択補助処理という場合がある）を選択する。複数の補助処理は、例えば上記の補正モードにおいて実行される補正処理、指導モードにおいて実行される指導処理、及び自動モードにおいて実行される自動処理である。 The selection unit 132 has one or more auxiliary processes (hereinafter, may be referred to as selection auxiliary processes) from a plurality of auxiliary process candidates for assisting the operation of the operator U as the auxiliary processes to be executed by the auxiliary processing unit 133. Select. The plurality of auxiliary processes are, for example, a correction process executed in the above-mentioned correction mode, a guidance process executed in the guidance mode, and an automatic process executed in the automatic mode.
　選択部１３２は、例えば、ロボット制御部１３４に補正情報に基づいてロボット２を制御させる補正処理、及び操作を補助するための補助情報を操作装置３に表示させる指導処理の少なくともいずれかを選択する。選択部１３２は、ロボット制御部１３４に操作情報を制御情報として用いてロボットを制御させる非補正モードと、ロボット制御部１３４に補正情報を制御情報として用いてロボットを制御させる補正モードとを切り替えてもよい。 The selection unit 132 selects, for example, at least one of a correction process for causing the robot control unit 134 to control the robot 2 based on the correction information and a guidance process for displaying auxiliary information for assisting the operation on the operation device 3. .. The selection unit 132 switches between a non-correction mode in which the robot control unit 134 uses the operation information as control information to control the robot and a correction mode in which the robot control unit 134 uses the correction information as control information to control the robot. May be good.
　選択部１３２は、選択した補助処理の種別を操作装置３に表示させてもよい。選択された補助処理の種別が操作装置３に表示されることで、操作者Ｕが補助処理の内容を把握しながらロボット２を操作することができる。その結果、操作者Ｕが操作した内容とロボット２の動きとの間に差がある場合であっても、補助されていることが原因であると操作者Ｕが認識することができるので、操作者Ｕが違和感を抱くことを防止できる。 The selection unit 132 may display the selected auxiliary processing type on the operation device 3. By displaying the selected auxiliary processing type on the operation device 3, the operator U can operate the robot 2 while grasping the contents of the auxiliary processing. As a result, even if there is a difference between the content operated by the operator U and the movement of the robot 2, the operator U can recognize that the cause is the assistance, so that the operation can be performed. It is possible to prevent the person U from feeling uncomfortable.
　選択部１３２は、選択した補助処理を示す選択処理情報を補助処理部１３３に通知する。選択部１３２は、例えば補正処理が選択された場合に選択処理情報として「０１」を補助処理部１３３に通知し、指導処理が選択された場合に選択処理情報として「１０」を補助処理部１３３に通知し、自動処理が選択された場合に選択処理情報として「１１」を補助処理部１３３に通知する。選択部１３２は、補助処理を実行しないことを選択した場合、選択処理情報として、通常処理が選択されたことを示す「００」を補助処理部１３３に通知してもよい。選択部１３２の動作のさらなる詳細については後述する。 The selection unit 132 notifies the auxiliary processing unit 133 of the selection processing information indicating the selected auxiliary processing. For example, when the correction process is selected, the selection unit 132 notifies the auxiliary processing unit 133 of "01" as the selection processing information, and when the guidance processing is selected, the selection unit 132 notifies the auxiliary processing unit 133 of "10" as the selection processing information. Is notified, and when automatic processing is selected, "11" is notified to the auxiliary processing unit 133 as selection processing information. When the selection unit 132 chooses not to execute the auxiliary processing, the selection unit 132 may notify the auxiliary processing unit 133 of "00" indicating that the normal processing has been selected as the selection processing information. Further details of the operation of the selection unit 132 will be described later.
　補助処理部１３３は、選択部１３２が選択した補助処理を実行することにより操作者Ｕを補助する。補助処理部１３３は、補正処理が選択された場合、情報取得部１３１から入力された操作情報を補正することにより、ロボット２を動作させるための制御情報として用いられる補正情報を作成する。補助処理部１３３は、補正モードにおいて、操作者Ｕが実行する操作の内容に対応する予め定められた基準制御情報に操作情報を置換することにより補正情報を作成する。基準制御情報は、例えば、操作のスキルが高い人が操作をした場合に取得された操作情報である。補助処理部１３３は、例えば操作情報に基づいて、操作者Ｕが意図している操作の内容を推定し、推定した操作の内容に関連付けて記憶部１２に記憶されている基準制御情報の内容に操作情報の内容が近づくように操作情報を補正する。 The auxiliary processing unit 133 assists the operator U by executing the auxiliary processing selected by the selection unit 132. When the correction process is selected, the auxiliary processing unit 133 corrects the operation information input from the information acquisition unit 131 to create correction information used as control information for operating the robot 2. In the correction mode, the auxiliary processing unit 133 creates correction information by replacing the operation information with predetermined reference control information corresponding to the content of the operation executed by the operator U. The reference control information is, for example, operation information acquired when a person with high operation skill performs an operation. The auxiliary processing unit 133 estimates the content of the operation intended by the operator U based on, for example, the operation information, and associates the content of the estimated operation with the content of the reference control information stored in the storage unit 12. Correct the operation information so that the contents of the operation information come closer.
　補助処理部１３３は、情報取得部１３１がロボット２から取得した検出情報に基づいてロボット２の状態及びロボット２の周辺の状態の少なくともいずれかを特定し、特定した状態に適した操作になるように操作情報を補正してもよい。補助処理部１３３は、例えばロボット２が撮影したロボット２の手の近くの物体の画像データに基づいて物体の形状を特定し、特定した形状に関連付けて記憶部１２に記憶されている基準制御情報を取得する。 The auxiliary processing unit 133 identifies at least one of the state of the robot 2 and the state around the robot 2 based on the detection information acquired from the robot 2 by the information acquisition unit 131, so that the operation is suitable for the specified state. The operation information may be corrected to. The auxiliary processing unit 133 identifies the shape of the object based on the image data of the object near the hand of the robot 2 taken by the robot 2, for example, and the reference control information stored in the storage unit 12 in association with the specified shape. To get.
　例えば、操作情報が、２本の指で掴むという操作内容を示しているとする。そして、ロボット２が撮影した物体の形状が球状であり、球状の物体を掴む場合に対応する基準制御情報は３本の指で掴む操作内容を示しているとする。この場合、補助処理部１３３は、球状の物体を掴む場合に対応する基準制御情報に基づいて操作情報を補正して、３本の指で掴む操作に対応する補正情報を作成する。 For example, it is assumed that the operation information indicates the operation content of grasping with two fingers. Then, it is assumed that the shape of the object photographed by the robot 2 is spherical, and the reference control information corresponding to the case of grasping the spherical object indicates the operation content of grasping with three fingers. In this case, the auxiliary processing unit 133 corrects the operation information based on the reference control information corresponding to the case of grasping the spherical object, and creates the correction information corresponding to the operation of grasping with three fingers.
　別の例として、操作情報が、腕を高速に動かすという操作内容を示しているとする。そして、ロボット２が撮影した画像に人が写っているとする。この場合、補助処理部１３３は、人が近くにいる場合に対応する基準制御情報に基づいて操作情報を補正して、操作情報が示す速度よりも遅い速度で腕を動かすという操作に対応する補正情報を作成する。 As another example, it is assumed that the operation information indicates the operation content of moving the arm at high speed. Then, it is assumed that a person is shown in the image taken by the robot 2. In this case, the auxiliary processing unit 133 corrects the operation information based on the reference control information corresponding to the case where the person is nearby, and the correction corresponding to the operation of moving the arm at a speed slower than the speed indicated by the operation information. Create information.
　補助処理部１３３は、指導モードが選択された場合、操作装置３に表示させる操作者Ｕを補助するための指導用の画像、指導用の音声、指導用に操作装置３の一部の部位を振動させるための振動情報の少なくとも１つを含む指導情報を作成する。補助処理部１３３は、予め用意された複数の指導用の画像、音声、振動情報から一以上の画像、音声、振動情報を選択してもよい。補助処理部１３３は、例えば、操作者Ｕが行う作業内容、操作者Ｕが用いるシナリオ、又は操作装置３から受信した操作情報の少なくともいずれかに基づいて、操作装置３に表示させる指導用の画像、音声、振動情報を作成する。 When the instruction mode is selected, the auxiliary processing unit 133 displays an image for instruction to assist the operator U to be displayed on the operation device 3, a voice for instruction, and a part of the operation device 3 for instruction. Create guidance information that includes at least one of the vibration information to vibrate. The auxiliary processing unit 133 may select one or more images, sounds, and vibration information from a plurality of images, sounds, and vibration information for instruction prepared in advance. The auxiliary processing unit 133 displays an image for instruction to be displayed on the operation device 3 based on at least one of the work content performed by the operator U, the scenario used by the operator U, or the operation information received from the operation device 3. , Voice, create vibration information.
　指導用の画像は、例えば、指、腕、足、頭等の各部を移動させるべき位置を示す画像である。補助処理部１３３は、操作者Ｕが物品を掴む操作をしている場合、ロボット２から取得した物品を撮影した画像に、指を接触させるべき位置を示す色（例えば物品と異なる色）の画像を重ねた指導用の画像を作成する。指導用の音声は、「もう少し右です」、「人差し指がずれています」のように、操作方法を案内する音声である。指導用の振動情報は、例えば操作者Ｕが操作を間違えた場合に操作者Ｕが装着した操作装置３に振動を与えるための情報である。 The image for instruction is, for example, an image showing a position where each part such as a finger, an arm, a foot, and a head should be moved. When the operator U is performing an operation of grasping the article, the auxiliary processing unit 133 is an image of a color (for example, a color different from the article) indicating a position where the finger should be brought into contact with the image of the article acquired from the robot 2. Create an image for instruction with overlapping. The instructional voice is a voice that guides the operation method, such as "a little more right" or "the index finger is out of alignment". The vibration information for guidance is information for giving vibration to the operation device 3 worn by the operator U when, for example, the operator U makes a mistake in the operation.
　補助処理部１３３は、操作者Ｕの操作スキルに基づいて、指導用の画像の内容を決定してもよい。補助処理部１３３は、例えば操作者Ｕの操作スキルが低いほど多くの情報を含む指導情報を作成する。先の例の場合、補助処理部１３３は、操作者Ｕの操作スキルが最も低いレベルである場合に、例えば指を接触させるべき位置を示すとともに、注意事項を示すテキスト情報を含む指導用画像を作成する。補助処理部１３３は、操作者Ｕの操作スキルがもう少し高いレベルである場合に、操作者Ｕのスキルが最も低いレベルである場合よりも、指導用画像を提示する頻度を下げたり、指導用画像に含まれる情報を減らしたりする。具体的には、補助処理部１３３は、操作者Ｕの操作スキルが高いレベルである場合に、掴む対象の物品が掴みにくい形状であることを条件として、指を接触させるべき位置を示す指導用画像を作成する。補助処理部１３３がこのように動作することで、操作スキルが高い操作者Ｕが過度に多くの情報を視認することを防ぐとともに、操作スキルが低い操作者Ｕに対して十分な情報を提供することができる。 The auxiliary processing unit 133 may determine the content of the image for instruction based on the operation skill of the operator U. For example, the auxiliary processing unit 133 creates instruction information including more information as the operation skill of the operator U is lower. In the case of the above example, when the operation skill of the operator U is the lowest level, the auxiliary processing unit 133 indicates, for example, the position where the finger should be touched, and displays a teaching image including text information indicating precautions. create. When the operation skill of the operator U is a little higher, the auxiliary processing unit 133 lowers the frequency of presenting the instruction image or displays the instruction image as compared with the case where the skill of the operator U is the lowest level. Reduce the information contained in. Specifically, the auxiliary processing unit 133 is for teaching to indicate the position where the finger should be touched, provided that the object to be grasped has a shape that is difficult to grasp when the operation skill of the operator U is high. Create an image. By operating the auxiliary processing unit 133 in this way, the operator U having a high operation skill is prevented from visually recognizing an excessively large amount of information, and the operator U having a low operation skill is provided with sufficient information. be able to.
　補助処理部１３３は、操作装置３から送信された操作情報に基づいて指導用画像の内容を決定してもよい。補助処理部１３３は、例えば、操作情報に基づいて操作者Ｕの動きが停止したと判定した場合、又は操作者Ｕの動きが過度に遅くなったと判定した場合に、次に行うべき操作を示す指導用画像を作成する。補助処理部１３３は、操作情報とロボット２から送信される検出情報とに基づいて、操作情報が示す操作の内容が不適切であると判定した場合に、操作の内容を変更させることを案内するための指導用画像を作成してもよい。 The auxiliary processing unit 133 may determine the content of the instruction image based on the operation information transmitted from the operation device 3. The auxiliary processing unit 133 indicates, for example, an operation to be performed next when it is determined that the movement of the operator U has stopped based on the operation information, or when it is determined that the movement of the operator U has become excessively slow. Create a teaching image. The auxiliary processing unit 133 guides the robot 2 to change the operation content when it is determined that the operation content indicated by the operation information is inappropriate based on the operation information and the detection information transmitted from the robot 2. You may create a teaching image for this.
　補助処理部１３３は、自動モードが選択された場合、予め定められた動作内容を示すシナリオを参照し、シナリオに含まれる情報に基づいて制御情報を作成する。補助処理部１３３は、ロボット２から送信された検出情報に基づいてロボット２の状態を特定し、特定したロボット２の状態とシナリオが示す動作内容とに基づいて制御情報を作成してもよい。 When the automatic mode is selected, the auxiliary processing unit 133 refers to a scenario showing a predetermined operation content and creates control information based on the information included in the scenario. The auxiliary processing unit 133 may specify the state of the robot 2 based on the detection information transmitted from the robot 2, and create control information based on the specified state of the robot 2 and the operation content indicated by the scenario.
　補助処理部１３３は、通常モードが選択された場合、操作装置３から受信した操作情報をそのまま制御情報とする。補助処理部１３３は、作成した制御情報をロボット制御部１３４に通知する。 When the normal mode is selected, the auxiliary processing unit 133 uses the operation information received from the operation device 3 as the control information as it is. The auxiliary processing unit 133 notifies the robot control unit 134 of the created control information.
　なお、補助処理部１３３は、選択された補助処理の内容に基づいて、操作者Ｕの作業に対する報酬額を決定し、決定した報酬額を外部装置（例えば操作者Ｕの情報端末）に出力してもよい。補助処理部１３３は、補助の度合いが小さければ小さいほど高い報酬額に決定する。補助処理部１３３がこのように報酬額を決定することで、補助を受けないで作業をできるようになるように操作者Ｕを動機付けることができる。 The auxiliary processing unit 133 determines the reward amount for the work of the operator U based on the content of the selected auxiliary processing, and outputs the determined reward amount to an external device (for example, the information terminal of the operator U). You may. The auxiliary processing unit 133 determines that the smaller the degree of assistance, the higher the reward amount. By determining the reward amount in this way, the auxiliary processing unit 133 can motivate the operator U so that he / she can work without receiving assistance.
　ロボット制御部１３４は、操作情報に基づいて作成された選択補助処理の種類に対応する制御情報を用いてロボット２を制御する。ロボット制御部１３４は、補助処理部１３３が選択補助処理を実行している間に、操作情報に基づいて生成され補助処理部１３３からロボット制御部１３４に通知される制御情報をロボット２に送信することによりロボット２を制御する。 The robot control unit 134 controls the robot 2 using the control information corresponding to the type of selection assist processing created based on the operation information. The robot control unit 134 transmits the control information generated based on the operation information and notified from the auxiliary processing unit 133 to the robot control unit 134 to the robot 2 while the auxiliary processing unit 133 is executing the selection auxiliary processing. This controls the robot 2.
　ロボット制御部１３４は、選択部１３２が新たな一以上の補助処理を選択した場合に、ロボット２の動作速度を変化させてもよい。すなわち、ロボット制御部１３４は、補助処理の内容が切り替わった場合に、ロボット２の動作速度を変化させてもよい。例えば、ロボット制御部１３４は、補助処理の内容を切り替える時点で、ロボット２の動作を一時的に停止させる。ロボット制御部１３４は、この際に、ロボット２に「自動動作に切り替わります」という音声を発せさせてロボット２の周囲の人に、モードが切り替わることを知らせてもよい。また、ロボット制御部１３４は、モードが切り替わることを操作装置３に表示させてもよい。ロボット制御部１３４がこのように動作することで、ロボット２の周囲の人が、ロボット２の動作が変化することを把握できるので、ロボット２との衝突等の危険を回避することが可能になる。 The robot control unit 134 may change the operating speed of the robot 2 when the selection unit 132 selects one or more new auxiliary processes. That is, the robot control unit 134 may change the operating speed of the robot 2 when the content of the auxiliary processing is switched. For example, the robot control unit 134 temporarily stops the operation of the robot 2 at the time of switching the contents of the auxiliary processing. At this time, the robot control unit 134 may make the robot 2 emit a voice saying "switch to automatic operation" to notify people around the robot 2 that the mode is switched. Further, the robot control unit 134 may display the operation device 3 that the mode is switched. By operating the robot control unit 134 in this way, people around the robot 2 can grasp that the operation of the robot 2 changes, so that it is possible to avoid a danger such as a collision with the robot 2. ..
　ロボット２が物品を持っている場合、ロボット制御部１３４は、モードを切り替える際に一時停止する前に、徐々に動きの速度を遅くしてもよい。ロボット制御部１３４がこのように動作することにより、ロボット２が物品を落下してしまうことを抑制することができる。また、ロボット２の頭部に設けられたカメラを通してロボット２の周囲を視認している操作者Ｕの気分が悪くなることを防ぐために、モードを切り替える際にも、操作者Ｕの操作に反して頭部の動きを一時停止しないようにしてもよい。 When the robot 2 holds an article, the robot control unit 134 may gradually slow down the speed of movement before pausing when switching modes. By operating the robot control unit 134 in this way, it is possible to prevent the robot 2 from dropping an article. Further, in order to prevent the operator U who is visually recognizing the surroundings of the robot 2 through the camera provided on the head of the robot 2 from feeling unwell, even when switching the mode, contrary to the operation of the operator U. The movement of the head may not be paused.
　また、ロボット制御部１３４は、ロボット２に動作させる内容に対応する表示をロボット２にさせるための情報を含む制御情報をロボット２に送信してもよい。ロボット制御部１３４は、例えばロボット２の右腕を右に動かす場合に、ロボット２の右腕に設けられた発光デバイスを発光させる指示を含む制御情報を送信する。ロボット制御部１３４がこのように動作することで、ロボット２の周囲の人が、ロボット２がどのように動くかを把握することができるので、安全性が向上する。 Further, the robot control unit 134 may transmit control information to the robot 2 including information for causing the robot 2 to display a display corresponding to the content to be operated by the robot 2. For example, when the right arm of the robot 2 is moved to the right, the robot control unit 134 transmits control information including an instruction to light a light emitting device provided on the right arm of the robot 2. By operating the robot control unit 134 in this way, people around the robot 2 can grasp how the robot 2 moves, so that safety is improved.
　ロボット制御部１３４は、どのモードで動作しているかを示す表示をロボット２にさせるための情報を含む制御情報をロボット２に送信してもよい。ロボット制御部１３４は、例えば、通常モードでの動作中は青色のランプを点灯し、自動モードでの動作中は赤色のランプを点灯し、指導モードでの動作中は緑色のランプを点灯し、補正モードでの動作中は黄色のランプを点灯する。ロボット制御部１３４がこのように動作することで、ロボット２の周囲の人が、ロボット２がどのような動きをし得るかを予想することができるので、安全性が向上する。 The robot control unit 134 may transmit control information to the robot 2 including information for causing the robot 2 to display a display indicating which mode it is operating in. For example, the robot control unit 134 lights a blue lamp during operation in the normal mode, lights a red lamp during operation in the automatic mode, and lights a green lamp during operation in the instruction mode. The yellow lamp lights up during operation in the correction mode. By operating the robot control unit 134 in this way, people around the robot 2 can predict what kind of movement the robot 2 can make, so that safety is improved.
　ロボット制御部１３４は、モードによって、ロボット２が撮影する画像の画角を変化させるようにロボット２を制御してもよい。ロボット制御部１３４は、例えば自動モードの場合に、ロボット２が背面側の画像を撮影し、手動モードの場合、自動モードの場合よりも広い範囲の画像を撮影する。これにより、手動モード時の操作性が向上する。 The robot control unit 134 may control the robot 2 so as to change the angle of view of the image captured by the robot 2 depending on the mode. For example, in the automatic mode, the robot control unit 134 captures an image on the back side of the robot 2, and in the manual mode, it captures a wider range of images than in the automatic mode. This improves operability in the manual mode.
　ロボット制御部１３４は、選択部１３２が新たな補助処理を選択した時点で操作者Ｕの姿勢とロボット２の姿勢とが一致していない場合、補助処理部１３３が新たな補助処理を実行する前に、ロボット２の姿勢が操作者Ｕの姿勢と一致するようにロボット２を動作させてもよい。すなわち、ロボット制御部１３４は、モードが切り替わるタイミングで、操作者Ｕの姿勢とロボット２の姿勢とが一致していない場合、モードを変更する前にロボット２の姿勢が操作者Ｕの姿勢と一致するようにロボット２を動作させてもよい。 If the posture of the operator U and the posture of the robot 2 do not match when the selection unit 132 selects the new auxiliary processing, the robot control unit 134 before the auxiliary processing unit 133 executes the new auxiliary processing. In addition, the robot 2 may be operated so that the posture of the robot 2 matches the posture of the operator U. That is, if the posture of the operator U and the posture of the robot 2 do not match at the timing when the mode is switched, the robot control unit 134 matches the posture of the robot 2 with the posture of the operator U before changing the mode. The robot 2 may be operated so as to do so.
　例えば、ロボット制御部１３４は、自動モードで動作した後に手動モードに戻る際に、操作者Ｕが特定の姿勢で待機するように指示するための情報を操作装置３に表示させる。ロボット制御部１３４は、ロボット２の姿勢が、操作者Ｕが待機している姿勢と同じになった時点で、操作装置３に対して、操作の再開を許可する情報を表示させる。ロボット制御部１３４がこのように動作することで、操作者Ｕの姿勢とロボット２の姿勢とが異なる場合に、操作者Ｕが違和感を持った状態で操作をすることを防止できる。 For example, the robot control unit 134 causes the operation device 3 to display information for instructing the operator U to stand by in a specific posture when returning to the manual mode after operating in the automatic mode. When the posture of the robot 2 becomes the same as the posture in which the operator U is waiting, the robot control unit 134 causes the operation device 3 to display information permitting the restart of the operation. By operating the robot control unit 134 in this way, it is possible to prevent the operator U from operating in a state of discomfort when the posture of the operator U and the posture of the robot 2 are different.
［選択部１３２による選択動作］
　続いて、選択部１３２がモードを選択する動作の詳細を説明する。選択部１３２は、以下に説明する各種の要因を組み合わせることによりモードを選択してもよい。 [Selection operation by selection unit 132]
Subsequently, the details of the operation in which the selection unit 132 selects the mode will be described. The selection unit 132 may select a mode by combining various factors described below.
（スキルに基づく選択）
　選択部１３２は、例えば、操作者Ｕの操作スキルに基づいて、実行する補助処理を選択する。選択部１３２は、情報取得部１３１が取得したスキル情報が示す操作スキルを用いて実行する補助処理を選択してもよく、情報取得部１３１が取得した操作情報に基づいて操作スキルを特定し、特定した操作スキルを用いて補助処理を選択してもよい。選択部１３２は、操作者Ｕに関連付けて記憶部１２に記憶されている操作スキルを用いてもよい。 (Skill-based selection)
The selection unit 132 selects an auxiliary process to be executed, for example, based on the operation skill of the operator U. The selection unit 132 may select an auxiliary process to be executed using the operation skill indicated by the skill information acquired by the information acquisition unit 131, and identifies the operation skill based on the operation information acquired by the information acquisition unit 131. Auxiliary processing may be selected using the specified operation skill. The selection unit 132 may use the operation skill stored in the storage unit 12 in association with the operator U.
　選択部１３２は、操作者Ｕの操作スキルが高ければ高いほど、操作情報を補正しないで制御情報とする割合を多くする。選択部１３２は、例えば操作スキルが高い操作者Ｕがロボット２を操作する場合には、通常モード又は指導モードを選択し、操作スキルが低い操作者Ｕがロボット２を操作する場合には、補正モード又は自動モードを選択する。 The higher the operation skill of the operator U, the greater the ratio of the selection unit 132 to use the control information without correcting the operation information. For example, the selection unit 132 selects the normal mode or the instruction mode when the operator U having a high operation skill operates the robot 2, and corrects when the operator U having a low operation skill operates the robot 2. Select mode or automatic mode.
（操作の内容に基づく選択）
　選択部１３２は、操作情報が示す操作の内容に基づいて、実行する補助処理を選択してもよい。選択部１３２は、例えば、操作情報が示す操作の内容に基づいて、非補正モードと補正モードとを切り替える。選択部１３２は、例えば腕を動かす動作に対しては非補正モードを選択し、物を掴む動作に対しては補正モードを選択する。選択部１３２は、広い場所で腕を動かす動作に対しては非補正モードを選択し、周辺に障害物がある場所で腕を動かす動作に対しては補正モードを選択してもよい。選択部１３２は、微細な動きが必要な場合に自動モードを選択してもよい。 (Selection based on the content of the operation)
The selection unit 132 may select an auxiliary process to be executed based on the content of the operation indicated by the operation information. The selection unit 132 switches between the non-correction mode and the correction mode, for example, based on the content of the operation indicated by the operation information. The selection unit 132 selects, for example, the non-correction mode for the movement of moving the arm, and selects the correction mode for the movement of grasping an object. The selection unit 132 may select the non-correction mode for the movement of moving the arm in a wide place, and may select the correction mode for the movement of moving the arm in a place where there is an obstacle in the vicinity. The selection unit 132 may select the automatic mode when fine movement is required.
　選択部１３２は、操作の内容ごとに操作者Ｕの操作スキルを特定し、操作の内容ごとに、操作者Ｕの操作スキルに基づいて実行する補助処理を選択してもよい。例えば、箱を運ぶ操作のスキルは高いけれども、球状の物を掴む操作のスキルは低い操作者Ｕが操作をする場合、選択部１３２は、箱を運ぶ操作においては通常モードを選択し、球状の物を掴む操作においては補正モードを選択してもよい。 The selection unit 132 may specify the operation skill of the operator U for each operation content and select an auxiliary process to be executed based on the operation skill of the operator U for each operation content. For example, when the operator U has a high skill in carrying a box but a low skill in grasping a spherical object, the selection unit 132 selects a normal mode in the operation of carrying the box and has a spherical shape. The correction mode may be selected in the operation of grasping an object.
　選択部１３２は、操作の内容ごとに定められた難易度に基づいて、実行する補助処理を選択してもよい。選択部１３２は、例えば、予め記憶部１２に記憶された、操作の内容と操作の難易度との関係を示すテーブルを参照することにより、操作の難易度に対応する補助処理を実行する。 The selection unit 132 may select an auxiliary process to be executed based on the difficulty level determined for each operation content. The selection unit 132 executes auxiliary processing corresponding to the difficulty level of the operation by referring to a table previously stored in the storage unit 12 showing the relationship between the content of the operation and the difficulty level of the operation.
　一例として、選択部１３２は、所定の位置まで腕を動かす動作をロボット２に実行させる間は通常モードを選択し、所定の位置まで腕を動かした後に複雑な形状の物を掴む操作をロボット２に実行させる間は、指導モード又は補正モードを選択する。選択部１３２は、腕の周辺に障害物があり、腕の動かし方が難しい場合には、腕を動かす間に指導モード、補正モード又は自動モードを選択してもよい。このように選択部１３２が操作の難易度に基づいて動作モードを選択することにより、比較的容易な操作をする際には操作者Ｕが自分のペースで作業を進め、比較的難しい作業をする際に不適切な動作をロボット２が実行してしまうことを抑制できる。 As an example, the selection unit 132 selects a normal mode while causing the robot 2 to perform an operation of moving an arm to a predetermined position, and then performs an operation of grasping an object having a complicated shape after moving the arm to a predetermined position. The instruction mode or the correction mode is selected while the robot is executed. When there is an obstacle around the arm and it is difficult to move the arm, the selection unit 132 may select the instruction mode, the correction mode, or the automatic mode while moving the arm. By selecting the operation mode based on the difficulty level of the operation in this way, the operator U advances the work at his / her own pace when performing a relatively easy operation, and performs a relatively difficult work. It is possible to prevent the robot 2 from executing an inappropriate operation at that time.
（シナリオに基づく選択）
　選択部１３２は、基準時刻からの経過時間とロボットの動作内容とが関連付けられたシナリオが示す動作内容に基づいて実行する補助処理を選択してもよい。操作者Ｕがシナリオに基づいてロボット２を動作させる場合、一つのシナリオに含まれる各種の作業を実行するために、操作者Ｕはさまざまな種別の操作を行う必要がある。 (Selection based on scenario)
The selection unit 132 may select an auxiliary process to be executed based on the operation content indicated by the scenario in which the elapsed time from the reference time and the operation content of the robot are associated with each other. When the operator U operates the robot 2 based on the scenario, the operator U needs to perform various types of operations in order to execute various operations included in one scenario.
　そこで、選択部１３２は、操作者Ｕがシナリオに基づいて操作している間に、シナリオに含まれる各作業の種別又は各作業の難易度等に基づいて動作モードを切り替える。選択部１３２は、例えば情報取得部１３１が取得した複数の操作情報が示す複数の操作のうち一部の操作に対して非補正モードを選択し、他の一部の操作に対しては補正モードを選択する。選択部１３２がこのように動作することで、シナリオに各種の作業が含まれている場合に、操作者Ｕはシナリオに含まれる一連の作業を適切に遂行することができる。 Therefore, while the operator U is operating based on the scenario, the selection unit 132 switches the operation mode based on the type of each work included in the scenario, the difficulty level of each work, and the like. For example, the selection unit 132 selects a non-correction mode for some operations among a plurality of operations indicated by a plurality of operation information acquired by the information acquisition unit 131, and a correction mode for some other operations. Select. By operating the selection unit 132 in this way, when the scenario includes various operations, the operator U can appropriately perform a series of operations included in the scenario.
（ロボット２の周辺の状況に基づく選択）
　選択部１３２は、ロボット２の周辺の状況に基づいて実行する補助処理を選択してもよい。選択部１３２は、例えば情報取得部１３１を介して、ロボット２の周辺の状況をロボット２が検出した結果を含む検出情報を取得し、取得した検出情報に基づいて操作の難易度を判定する。選択部１３２は、上述したように、操作の難易度に基づいて動作モードを選択する。選択部１３２は、例えば、ロボット２が正常に動作していないことを検出情報が示している場合に、選択した補助処理を変更してもよい。 (Selection based on the situation around Robot 2)
The selection unit 132 may select an auxiliary process to be executed based on the situation around the robot 2. The selection unit 132 acquires detection information including the result of the robot 2 detecting the situation around the robot 2 via, for example, the information acquisition unit 131, and determines the difficulty level of the operation based on the acquired detection information. As described above, the selection unit 132 selects the operation mode based on the difficulty level of the operation. The selection unit 132 may change the selected auxiliary process, for example, when the detection information indicates that the robot 2 is not operating normally.
　具体的には、選択部１３２は、通常モードで動作中に、ロボット２が掴もうとしている物品を落下しそうになっている場合、又はロボット２が近傍の人や物に衝突しそうになっている場合のように、そのまま動作を継続すると問題が生じ得ることを示す検出情報を取得した場合に、補正モードに切り替える。選択部１３２は、通常モードで動作中に、ロボット２が掴んでいる物品を落下した場合、物品を壊した場合、ロボット２の一部の部位が本来の位置からずれている場合、周囲の物体と衝突した場合に、補正モード又は自動モードに切り替えてもよい。また、選択部１３２は、指導モードで動作中に問題が生じ得ることを示す検出情報を取得した場合に補正モードに切り替えてもよい。選択部１３２がこのように動作することで、ロボット制御部１３４は、想定外の状況が発生した場合にもロボット２を適切に動作させることができる。 Specifically, the selection unit 132 is about to drop an article that the robot 2 is trying to grab while operating in the normal mode, or the robot 2 is about to collide with a nearby person or object. When the detection information indicating that a problem may occur if the operation is continued as it is is acquired as in the case, the mode is switched to the correction mode. The selection unit 132 is a surrounding object when the article held by the robot 2 is dropped, the article is broken, or a part of the robot 2 is displaced from the original position during operation in the normal mode. In the event of a collision with, the mode may be switched to the correction mode or the automatic mode. Further, the selection unit 132 may switch to the correction mode when the detection information indicating that a problem may occur during the operation in the instruction mode is acquired. By operating the selection unit 132 in this way, the robot control unit 134 can appropriately operate the robot 2 even when an unexpected situation occurs.
（操作者Ｕの状態に基づく選択）
　操作者Ｕが、操作中に気分が悪くなったり集中力が低下したりして事故を起こしやすい状態になっている場合、操作者Ｕが行った操作に従ってロボット２を動作させると危険である。そこで、選択部１３２は、例えば操作が不安定になったり、操作ミスが生じたりした場合に、操作者Ｕの状態が事故を起こしやすい状態であると判定する。選択部１３２は、操作者Ｕの状態が事故を起こしやすい状態であると判定した場合に、自動モードに切り替えたり、補正モードにおける補正処理の度合いを強めたりしてもよい。また、選択部１３２は、操作者Ｕの状態が事故を起こしやすい状態であると判定した場合に管理者に通報し、管理者に遠隔制御させてもよい。 (Selection based on the state of operator U)
When the operator U feels sick or loses his / her concentration during the operation and is in a state where an accident is likely to occur, it is dangerous to operate the robot 2 according to the operation performed by the operator U. Therefore, the selection unit 132 determines that the state of the operator U is likely to cause an accident, for example, when the operation becomes unstable or an operation error occurs. When the selection unit 132 determines that the state of the operator U is in a state in which an accident is likely to occur, the selection unit 132 may switch to the automatic mode or increase the degree of correction processing in the correction mode. Further, the selection unit 132 may notify the administrator when it is determined that the state of the operator U is in a state in which an accident is likely to occur, and the administrator may remotely control the operation.
（操作者Ｕ又は管理者の指示に基づく選択）
　選択部１３２は、操作者Ｕ又は操作者Ｕ若しくはロボット２を管理する管理者の指示に基づいて、実行する指導モードを選択してもよい。このようにするために、情報取得部１３１は、操作者Ｕ又は操作者Ｕの管理者がアクセス可能な情報端末に、複数の補助処理の候補を示す情報を表示させた後に、情報端末において選択された候補を示す選択候補情報を取得する。情報取得部１３１は、例えば、通常モード、指導モード、補正モード、自動モードから一つのモードを選択するための操作画面を情報端末に表示させる。 (Selection based on the instructions of operator U or administrator)
The selection unit 132 may select the instruction mode to be executed based on the instruction of the operator U, the operator U, or the administrator who manages the robot 2. In order to do so, the information acquisition unit 131 selects on the information terminal after displaying information indicating a plurality of auxiliary processing candidates on the information terminal accessible to the operator U or the administrator of the operator U. Acquires selection candidate information indicating the selected candidates. The information acquisition unit 131 causes the information terminal to display an operation screen for selecting one mode from, for example, a normal mode, a guidance mode, a correction mode, and an automatic mode.
　情報取得部１３１は、操作者Ｕが装着している操作装置３のディスプレイに操作画面を表示させてもよい。情報取得部１３１は、例えば、操作者Ｕが操作を一時的に停止している間に操作装置３に表示される一時停止画面のメニューの中に、補助処理を選択するためのメニューを含める。一時停止画面のメニューに、補助処理を選択するためのメニューが設けられていることにより、操作者Ｕは、操作を一時的に停止している間に、補助処理の内容を変更することができる。 The information acquisition unit 131 may display the operation screen on the display of the operation device 3 worn by the operator U. The information acquisition unit 131 includes, for example, a menu for selecting an auxiliary process in the menu of the pause screen displayed on the operation device 3 while the operator U temporarily stops the operation. Since the menu on the pause screen is provided with a menu for selecting an auxiliary process, the operator U can change the content of the auxiliary process while the operation is temporarily stopped. ..
　図８は、動作モードを選択するための操作画面の一例を示す図である。操作者Ｕは、操作画面におけるラジオボタンを選択することにより、所望の動作モードを選択することができる。 FIG. 8 is a diagram showing an example of an operation screen for selecting an operation mode. The operator U can select a desired operation mode by selecting a radio button on the operation screen.
　情報取得部１３１は、選択候補情報を取得すると、選択されたモードを示す情報を選択部１３２に通知する。選択部１３２は、情報取得部１３１が取得した選択候補情報に基づいて、操作者Ｕ又は操作者Ｕの管理者により指定された補助処理を、実行する補助処理として選択する。 When the information acquisition unit 131 acquires the selection candidate information, the information acquisition unit 131 notifies the selection unit 132 of information indicating the selected mode. Based on the selection candidate information acquired by the information acquisition unit 131, the selection unit 132 selects the auxiliary process specified by the operator U or the administrator of the operator U as the auxiliary process to be executed.
　情報取得部１３１は、複数の補助処理の候補を示す情報として、補助の度合いを選択するための画面を情報端末に表示させた後に、情報端末において入力された補助の度合いを示す補助レベル情報を取得してもよい。補助の度合いは、例えば、操作者Ｕが予定している作業時間において、補正処理を受けたり指導処理を受けたりする時間の割合により表される。 The information acquisition unit 131 displays auxiliary level information indicating the degree of assistance input in the information terminal after displaying a screen for selecting the degree of assistance on the information terminal as information indicating a plurality of candidates for auxiliary processing. You may get it. The degree of assistance is represented by, for example, the ratio of the time during which the operator U receives the correction process or the instruction process in the scheduled work time.
　図９は、補助の度合いを入力するための操作画面の一例を示す図である。情報取得部１３１は、図９（ａ）に示すように、補助の度合いの選択肢として「強」、「中」、「弱」というテキスト情報を含む画面を表示させてもよく、図９（ｂ）に示すように、０％から１００％までの任意の数値を選択するための画面を表示させてもよい。 FIG. 9 is a diagram showing an example of an operation screen for inputting the degree of assistance. As shown in FIG. 9A, the information acquisition unit 131 may display a screen including text information of “strong”, “medium”, and “weak” as options for the degree of assistance, and FIG. 9B ) May be displayed, a screen for selecting an arbitrary numerical value from 0% to 100% may be displayed.
　選択部１３２は、補助レベル情報に基づいて選択補助処理の内容を決定する。選択部１３２は、例えば「強」が選択された場合、常に補助情報を操作装置３に表示させ、操作情報を補正するように補助処理を選択する。選択部１３２は、「弱」が選択された場合、補助情報を操作装置３に表示させず、危険が伴う操作において誤操作が行われた場合のみ操作情報が補正されるように補助処理を選択する。選択部１３２は、補助の度合いの割合が数値で入力された場合、作業の含まれる複数の操作において補助情報を操作装置３に表示させ、かつ全ての操作情報を補正する状態を１００％として、所定の期間内の平均的な補助の程度が選択された数値になるように補助の内容を決定する。 The selection unit 132 determines the content of the selection assistance process based on the assistance level information. When, for example, "strong" is selected, the selection unit 132 always displays the auxiliary information on the operation device 3 and selects the auxiliary process so as to correct the operation information. When "weak" is selected, the selection unit 132 does not display the auxiliary information on the operation device 3, and selects the auxiliary process so that the operation information is corrected only when an erroneous operation is performed in a dangerous operation. .. When the ratio of the degree of assistance is input numerically, the selection unit 132 sets the state in which the auxiliary information is displayed on the operation device 3 in a plurality of operations including the work and all the operation information is corrected as 100%. The content of the assistance is determined so that the average degree of assistance within a predetermined period becomes the selected numerical value.
　情報取得部１３１は、補正処理の度合いと指導処理の度合いとを操作者Ｕが選択できるようにしてもよい。図１０は、補正処理の度合いと指導処理の度合いとを設定するための操作画面の一例を示す図である。図１０における横軸は補正処理の度合いに対応しており、縦軸は指導処理の度合いに対応している。操作者Ｕは、図１０に示す二次元領域における所望の位置を選択することで、所望の度合いの補正処理と指導処理を受けながらロボット２を操作することが可能になる。 The information acquisition unit 131 may allow the operator U to select the degree of correction processing and the degree of guidance processing. FIG. 10 is a diagram showing an example of an operation screen for setting the degree of correction processing and the degree of guidance processing. The horizontal axis in FIG. 10 corresponds to the degree of correction processing, and the vertical axis corresponds to the degree of guidance processing. By selecting a desired position in the two-dimensional region shown in FIG. 10, the operator U can operate the robot 2 while receiving a desired degree of correction processing and guidance processing.
　選択部１３２は、例えば横軸上の最も右の位置が操作者Ｕにより選択された場合には、自動処理を選択する。選択部１３２は、縦軸上の最も上の位置が操作者Ｕにより選択された場合には、補正をすることなく、常に指導情報を操作者Ｕに提示する処理を選択する。このように、情報取得部１３１が補正処理の度合いと指導処理の度合いとを設定するための操作画面を表示させることで、操作者Ｕが、所望の内容の補助を受けることができる。 The selection unit 132 selects automatic processing, for example, when the rightmost position on the horizontal axis is selected by the operator U. When the uppermost position on the vertical axis is selected by the operator U, the selection unit 132 selects a process of always presenting guidance information to the operator U without making corrections. In this way, by displaying the operation screen for setting the degree of correction processing and the degree of guidance processing by the information acquisition unit 131, the operator U can receive assistance of desired contents.
　情報取得部１３１は、操作の内容ごとに補助の度合いを変えられるように、操作の内容ごとに補助レベル情報を取得してもよい。例えば、情報取得部１３１は、腕を動かす動作、物を掴む動作、移動する動作のそれぞれに対する補助の度合いを入力するための画面を情報端末に表示させ、それぞれの動作に対する補助の度合いを示す補助レベル情報を取得する。選択部１３２は、情報取得部１３１が取得した補助レベル情報が示す補助の度合いに基づいて、作業に含まれる動作の種別ごとに補助処理の内容を切り替える。 The information acquisition unit 131 may acquire assistance level information for each operation content so that the degree of assistance can be changed for each operation content. For example, the information acquisition unit 131 causes the information terminal to display a screen for inputting the degree of assistance for each of the movement of the arm, the movement of grasping an object, and the movement of movement, and the assistance indicating the degree of assistance for each movement. Get level information. The selection unit 132 switches the content of the auxiliary processing for each type of operation included in the work, based on the degree of assistance indicated by the auxiliary level information acquired by the information acquisition unit 131.
　補助処理部１３３は、補助レベル情報が示す補助の度合いに基づいて、補助情報の量を変えたり、補正をする必要があると判定する条件を変えたりしてもよい。補助処理部１３３は、例えば補助の度合いを大きくする必要がある場合、補助の度合いを小さくてよい場合に比べて操作情報と基準制御情報との差が小さい時点で操作情報を補正する。情報取得部１３１、選択部１３２及び補助処理部１３３がこのように動作することで、ロボット制御部１３４は、操作者Ｕが得意とする操作においては補助の度合いを小さくし、操作者Ｕが苦手な操作においては補助の度合いを大きくすることができる。 The auxiliary processing unit 133 may change the amount of auxiliary information or change the condition for determining that correction is necessary, based on the degree of assistance indicated by the auxiliary level information. For example, when it is necessary to increase the degree of assistance, the auxiliary processing unit 133 corrects the operation information when the difference between the operation information and the reference control information is smaller than in the case where the degree of assistance may be small. By operating the information acquisition unit 131, the selection unit 132, and the auxiliary processing unit 133 in this way, the robot control unit 134 reduces the degree of assistance in the operations that the operator U is good at, and the operator U is not good at it. The degree of assistance can be increased in various operations.
（機械学習モデルを用いた選択）
　選択部１３２は、操作情報を入力すると一以上の補助処理を出力する機械学習モデルに、操作者Ｕが所定の操作を行う間に情報取得部１３１が取得した操作情報を入力し、機械学習モデルが出力した一以上の補助処理を、実行する補助処理として選択してもよい。機械学習モデルは、多数の操作情報と、各操作情報に対して選択された補助処理の内容とが関連付けられたデータを教師データとして使用することにより学習したモデルであり、例えば記憶部１２又は外部のコンピュータに記憶されている。教師データは、例えば過去に多数の人がロボット２の操作をした際に取得された操作情報と、当該操作時に操作者Ｕが選択した補助処理の内容とが関連付けられたデータである。教師データには、良好な操作結果が得られた際の操作情報と補助処理の内容との組み合わせを含む正の教師データと、良好でない操作結果が得られた際の操作情報と補助処理の内容との組み合わせを含む負の教師データとを含んでもよい。 (Selection using a machine learning model)
The selection unit 132 inputs the operation information acquired by the information acquisition unit 131 while the operator U performs a predetermined operation into the machine learning model that outputs one or more auxiliary processes when the operation information is input, and the machine learning model. One or more auxiliary processes output by may be selected as auxiliary processes to be executed. The machine learning model is a model learned by using as teacher data data in which a large number of operation information and the contents of auxiliary processing selected for each operation information are associated with each other, for example, the storage unit 12 or the outside. It is stored in the computer. The teacher data is, for example, data in which the operation information acquired when a large number of people have operated the robot 2 in the past and the content of the auxiliary processing selected by the operator U at the time of the operation are associated with each other. The teacher data includes positive teacher data including a combination of operation information and auxiliary processing contents when a good operation result is obtained, and operation information and auxiliary processing contents when a bad operation result is obtained. It may include negative teacher data including a combination with.
　選択部１３２が機械学習モデルに学習させる学習処理機能を有してもよい。この場合、選択部１３２は、補助処理の内容を決定するたびに操作情報と補助処理の内容とを関連付けて機械学習モデルに入力することにより、機械学習モデルに学習させる。機械学習モデルが外部のコンピュータに記憶されている場合、選択部１３２は、通信部１１を介して、外部のコンピュータに操作情報と補助処理の内容とを関連付けた教師データを送信することにより、機械学習モデルに学習させる。 The selection unit 132 may have a learning processing function for training the machine learning model. In this case, the selection unit 132 causes the machine learning model to learn by inputting the operation information and the content of the auxiliary process into the machine learning model each time the content of the auxiliary process is determined. When the machine learning model is stored in an external computer, the selection unit 132 transmits the teacher data in which the operation information and the content of the auxiliary processing are associated with the external computer to the external computer via the communication unit 11, thereby transmitting the machine. Let the learning model train.
　選択部１３２は、操作者Ｕが与えられた作業を行う前に所定の操作として実行される、作業の内容に対応する試験用の操作を行う間に情報取得部１３１が取得した操作情報を機械学習モデルに入力することにより一以上の補助処理を選択してもよい。この場合、機械学習モデルは、試験用の操作が行われた際の操作情報と補助処理の内容とを教師データとして学習する。選択部１３２がこのように動作することで、同一の作業をする場合に取得された教師データに基づいて学習した機械学習モデルに、機械学習モデルが学習する際に用いられた作業の内容と同一の作業をする間に取得された操作情報を入力することができるので、より適切な補助処理の内容が選択される確率が高まる。 The selection unit 132 uses the operation information acquired by the information acquisition unit 131 while performing a test operation corresponding to the content of the work, which is executed as a predetermined operation before the operator U performs the given work. One or more auxiliary processes may be selected by inputting into the training model. In this case, the machine learning model learns the operation information when the test operation is performed and the contents of the auxiliary processing as teacher data. By operating the selection unit 132 in this way, the machine learning model learned based on the teacher data acquired when performing the same work is the same as the content of the work used when the machine learning model learns. Since the operation information acquired during the work of can be input, the probability that a more appropriate auxiliary processing content is selected increases.
　選択部１３２は、作業の内容に関連付けられた、それぞれ異なる複数の機械学習モデルから、操作者Ｕが実行する予定の作業の内容に対応する一つの機械学習モデルを選択し、選択した機械学習モデルに操作情報を入力することにより、実行する補助処理を選択してもよい。選択部１３２がこのように動作する場合にも、機械学習モデルが学習する際に用いられた作業の内容と同一の作業をする間に取得された操作情報を入力することができるので、より適切な補助処理の内容が選択される確率が高まる。 The selection unit 132 selects one machine learning model corresponding to the content of the work to be executed by the operator U from a plurality of different machine learning models associated with the content of the work, and selects the machine learning model. By inputting the operation information in, the auxiliary process to be executed may be selected. Even when the selection unit 132 operates in this way, it is more appropriate because the operation information acquired while performing the same work as the content of the work used when the machine learning model learns can be input. The probability that the content of the auxiliary processing will be selected increases.
（ネットワークの通信品質に基づく選択）
　選択部１３２は、ネットワークＮの通信品質に基づいて動作モードを選択してもよい。ネットワークＮの通信品質が悪い場合、操作者Ｕが行った操作の内容を示す操作情報が制御装置１に正常に伝送されないということが生じ得る。そこで、選択部１３２は、操作装置３と制御装置１との間の通信の品質が、ロボット２の操作に支障が生じるレベルまで悪化した場合に、自動モードを選択してもよい。選択部１３２は、操作者Ｕにより動作モードが設定されている場合であっても、通信品質に基づいて動作モードを変更してもよい。この場合、選択部１３２は、通信品質が悪いために自動モードに変更されたことを操作装置３に表示させてもよい。 (Selection based on network communication quality)
The selection unit 132 may select the operation mode based on the communication quality of the network N. When the communication quality of the network N is poor, it is possible that the operation information indicating the content of the operation performed by the operator U may not be normally transmitted to the control device 1. Therefore, the selection unit 132 may select the automatic mode when the quality of communication between the operation device 3 and the control device 1 deteriorates to a level at which the operation of the robot 2 is hindered. The selection unit 132 may change the operation mode based on the communication quality even when the operation mode is set by the operator U. In this case, the selection unit 132 may display to the operation device 3 that the mode has been changed to the automatic mode due to poor communication quality.
［制御装置１による効果］
　以上説明したように、制御装置１は、操作者Ｕの操作を補助するための複数の補助処理の候補から、一以上の補助処理を選択する選択部１３２と、選択部１３２が選択した補助処理を実行することにより操作者Ｕを補助する補助処理部１３３と、を有する。制御装置１が選択部１３２及び補助処理部１３３を有することで、操作者Ｕの操作スキル、操作の内容、操作者Ｕの好み、ロボット２の周辺の状況等の各種の状況によって異なる内容の補助をすることができるので、操作者Ｕがロボット２を適切に操作しやすくなる。 [Effect of control device 1]
As described above, the control device 1 has a selection unit 132 that selects one or more auxiliary processes from a plurality of auxiliary process candidates for assisting the operation of the operator U, and an auxiliary process selected by the selection unit 132. Has an auxiliary processing unit 133 that assists the operator U by executing the above. Since the control device 1 has the selection unit 132 and the auxiliary processing unit 133, the auxiliary content differs depending on various situations such as the operation skill of the operator U, the content of the operation, the preference of the operator U, and the situation around the robot 2. Therefore, the operator U can easily operate the robot 2 appropriately.
　以上、本発明を実施の形態を用いて説明したが、本発明の技術的範囲は上記実施の形態に記載の範囲には限定されず、その要旨の範囲内で種々の変形及び変更が可能である。例えば、装置の全部又は一部は、任意の単位で機能的又は物理的に分散・統合して構成することができる。また、複数の実施の形態の任意の組み合わせによって生じる新たな実施の形態も、本発明の実施の形態に含まれる。組み合わせによって生じる新たな実施の形態の効果は、もとの実施の形態の効果を併せ持つ。 Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist. is there. For example, all or a part of the device can be functionally or physically distributed / integrated in any unit. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment.
１　制御装置
２　ロボット
３　操作装置
１１　通信部
１２　記憶部
１３　制御部
２１　通信部
２２　機構部
２３　センサー部
２４　記憶部
２５　制御部
３１　操作部
３２　検出部
３３　通信部
３４　表示部
３５　記憶部
３６　制御部
１３１　情報取得部
１３２　選択部
１３３　補助処理部
１３４　ロボット制御部
２５１　指示受信部
２５２　機構制御部
２５３　ロボット状態送信部
３６１　状態特定部
３６２　指示送信部
３６３　状態送信部
３６４　表示制御部 1 Control device 2 Robot 3 Operation device 11 Communication unit 12 Storage unit 13 Control unit 21 Communication unit 22 Mechanism unit 23 Sensor unit 24 Storage unit 25 Control unit 31 Operation unit 32 Detection unit 33 Communication unit 34 Display unit 35 Storage unit 36 Control unit 131 Information acquisition unit 132 Selection unit 133 Auxiliary processing unit 134 Robot control unit 251 Instruction reception unit 252 Mechanism control unit 253 Robot status transmission unit 361 State identification unit 362 Instruction transmission unit 363 Status transmission unit 364 Display control unit

Claims (18)

  1. 　ロボットを操作する操作者による操作の内容を示す操作情報を取得する情報取得部と、
    　前記操作者の操作を補助するための複数の補助処理の候補から、一以上の補助処理を選択する選択部と、
    　前記選択部が選択した前記一以上の補助処理である選択補助処理を実行することにより前記操作者を補助する補助処理部と、
    　前記補助処理部が前記選択補助処理を実行している間に、前記操作情報に基づいて作成された前記選択補助処理の種類に対応する制御情報を用いて前記ロボットを制御するロボット制御部と、
    　を有する制御装置。     An information acquisition unit that acquires operation information indicating the details of operations by the operator who operates the robot,
    A selection unit that selects one or more auxiliary processes from a plurality of auxiliary process candidates for assisting the operation of the operator.
    An auxiliary processing unit that assists the operator by executing a selection auxiliary processing that is one or more auxiliary processes selected by the selection unit, and
    A robot control unit that controls the robot using control information corresponding to the type of the selection auxiliary processing created based on the operation information while the auxiliary processing unit is executing the selection auxiliary processing.
    Control device having.
  2. 　前記選択部は、前記操作者の操作スキルに基づいて前記選択補助処理を選択する、
    　請求項１に記載の制御装置。     The selection unit selects the selection assisting process based on the operation skill of the operator.
    The control device according to claim 1.
  3. 　前記選択部は、前記操作情報が示す操作の内容に基づいて前記選択補助処理を選択する、
    　請求項１又は２に記載の制御装置。     The selection unit selects the selection assisting process based on the content of the operation indicated by the operation information.
    The control device according to claim 1 or 2.
  4. 　前記選択部は、基準時刻からの経過時間とロボットの動作内容とが関連付けられたシナリオが示す前記動作内容に基づいて前記選択補助処理を選択する、
    　請求項１から３のいずれか一項に記載の制御装置。     The selection unit selects the selection assisting process based on the operation content indicated by the scenario in which the elapsed time from the reference time and the operation content of the robot are associated with each other.
    The control device according to any one of claims 1 to 3.
  5. 　前記情報取得部は、前記ロボットに設けられたセンサーが検出した前記ロボットの周辺の状態を示す検出情報をさらに取得し、
    　前記選択部は、前記検出情報に基づいて前記選択補助処理を選択する、
    　請求項１から４のいずれか一項に記載の制御装置。     The information acquisition unit further acquires detection information indicating a state around the robot detected by a sensor provided in the robot.
    The selection unit selects the selection assisting process based on the detection information.
    The control device according to any one of claims 1 to 4.
  6. 　前記選択部は、前記検出情報が、前記ロボットが正常に動作していないことを示している場合に、前記選択補助処理を変更する、
    　請求項５に記載の制御装置。     The selection unit changes the selection assisting process when the detection information indicates that the robot is not operating normally.
    The control device according to claim 5.
  7. 　前記情報取得部は、前記操作者又は前記操作者の管理者がアクセス可能な情報端末に、前記複数の補助処理の候補を示す情報を表示させた後に、前記情報端末において選択された候補を示す選択候補情報を取得し、
    　前記選択部は、前記選択候補情報に基づいて、前記操作者又は前記操作者の管理者により指定された前記一以上の補助処理を前記選択補助処理として選択する、
    　請求項１から６のいずれか一項に記載の制御装置。     The information acquisition unit displays information indicating candidates for the plurality of auxiliary processes on an information terminal accessible to the operator or the administrator of the operator, and then indicates the candidates selected in the information terminal. Get selection candidate information,
    Based on the selection candidate information, the selection unit selects the one or more auxiliary processes designated by the operator or the administrator of the operator as the selection auxiliary processes.
    The control device according to any one of claims 1 to 6.
  8. 　前記情報取得部は、前記複数の補助処理の候補を示す情報として、補助の度合いを選択するための画面を前記情報端末に表示させた後に、前記情報端末において入力された補助の度合いを示す補助レベル情報を取得し、
    　前記選択部は、前記補助レベル情報に基づいて前記選択補助処理の内容を決定する、
    　請求項７に記載の制御装置。     The information acquisition unit displays an auxiliary screen for selecting the degree of assistance as information indicating the plurality of auxiliary processing candidates on the information terminal, and then indicates the degree of assistance input in the information terminal. Get level information,
    The selection unit determines the content of the selection assistance process based on the assistance level information.
    The control device according to claim 7.
  9. 　前記補助処理部は、前記操作情報を補正することにより前記ロボットに送信する補正情報を作成し、
    　前記選択部は、前記選択補助処理として、前記ロボット制御部に前記補正情報を前記制御情報として用いて前記ロボットを制御させる補正処理、及び操作を補助するための情報を前記操作者が操作に用いる操作装置に表示させる指導処理の少なくともいずれかを選択する、
    　請求項１から８のいずれか一項に記載の制御装置。     The auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information.
    The selection unit uses the correction processing for causing the robot control unit to control the robot by using the correction information as the control information, and the information for assisting the operation for the operation. Select at least one of the instruction processes to be displayed on the operating device,
    The control device according to any one of claims 1 to 8.
  10. 　前記補助処理部は、前記操作情報を補正することにより前記ロボットに送信する補正情報を作成し、
    　前記選択部は、前記ロボット制御部に前記操作情報を前記制御情報として用いて前記ロボットを制御させる非補正モードと、前記選択補助処理として前記ロボット制御部に前記補正情報を前記制御情報として用いて前記ロボットを制御させる補正モードとを切り替える、
    　請求項１から９のいずれか一項に記載の制御装置。     The auxiliary processing unit creates correction information to be transmitted to the robot by correcting the operation information.
    The selection unit uses the non-correction mode in which the robot control unit uses the operation information as the control information to control the robot, and the robot control unit uses the correction information as the control information as the selection assist processing. Switching between the correction mode for controlling the robot,
    The control device according to any one of claims 1 to 9.
  11. 　前記補助処理部は、前記補正モードにおいて、前記操作者が実行する操作の内容に対応する予め定められた基準制御情報に前記操作情報を置換することにより前記補正情報を作成する、
    　請求項１０に記載の制御装置。     In the correction mode, the auxiliary processing unit creates the correction information by substituting the operation information with predetermined reference control information corresponding to the content of the operation executed by the operator.
    The control device according to claim 10.
  12. 　前記選択部は、前記情報取得部が取得した複数の前記操作情報が示す複数の操作のうち一部の操作に対して前記非補正モードを選択し、他の一部の操作に対しては前記補正モードを選択する、
    　請求項１０又は１１に記載の制御装置。     The selection unit selects the non-correction mode for some of the operations indicated by the plurality of operation information acquired by the information acquisition unit, and the selection unit selects the non-correction mode for some of the operations. Select the correction mode,
    The control device according to claim 10 or 11.
  13. 　前記選択部は、前記操作情報を入力すると前記一以上の補助処理を出力する機械学習モデルに、前記操作者が所定の操作を行う間に前記情報取得部が取得した前記操作情報を入力し、前記機械学習モデルが出力した前記一以上の補助処理を選択する、
    　請求項１から１２のいずれか一項に記載の制御装置。     The selection unit inputs the operation information acquired by the information acquisition unit into the machine learning model that outputs one or more auxiliary processes when the operation information is input, while the operator performs a predetermined operation. Select one or more auxiliary processes output by the machine learning model.
    The control device according to any one of claims 1 to 12.
  14. 　前記選択部は、前記操作者が与えられた作業を行う前に前記所定の操作として実行される、前記作業の内容に対応する試験用の操作を行う間に前記情報取得部が取得した前記操作情報を前記機械学習モデルに入力することにより前記一以上の補助処理を選択する、
    　請求項１３に記載の制御装置。     The selection unit is executed as the predetermined operation before the operator performs a given operation, and the operation acquired by the information acquisition unit while performing a test operation corresponding to the content of the operation. Select one or more auxiliary processes by inputting information into the machine learning model.
    The control device according to claim 13.
  15. 　前記選択部は、前記選択補助処理の種別を前記操作者が操作に用いる操作装置に表示させる、
    　請求項１から１４のいずれか一項に記載の制御装置。     The selection unit causes the operation device used by the operator to display the type of the selection assist processing.
    The control device according to any one of claims 1 to 14.
  16. 　前記ロボット制御部は、前記選択部が新たな前記一以上の補助処理を選択した場合に、前記ロボットの動作速度を変化させる、
    　請求項１から１５のいずれか一項に記載の制御装置。     The robot control unit changes the operating speed of the robot when the selection unit selects a new one or more auxiliary processes.
    The control device according to any one of claims 1 to 15.
  17. 　前記ロボット制御部は、前記選択部が新たな前記一以上の補助処理を選択した時点で前記操作者の姿勢と前記ロボットの姿勢とが一致していない場合、前記補助処理部が新たな前記一以上の補助処理を実行する前に、前記ロボットの姿勢が前記操作者の姿勢と一致するように前記ロボットを動作させる、
    　請求項１から１６のいずれか一項に記載の制御装置。     In the robot control unit, if the posture of the operator and the posture of the robot do not match when the selection unit selects a new one or more auxiliary processes, the auxiliary processing unit is a new one. Before executing the above auxiliary processing, the robot is operated so that the posture of the robot matches the posture of the operator.
    The control device according to any one of claims 1 to 16.
  18. 　コンピュータが実行する、
    　ロボットを操作する操作者による操作の内容を示す操作情報を取得するステップと、
    　前記操作者の操作を補助するための複数の補助処理の候補から、一以上の補助処理を選択するステップと、
    　選択した前記一以上の補助処理である選択補助処理を実行することにより前記操作者を補助するステップと、
    　前記選択補助処理を実行している間に、前記操作情報に基づいて作成された前記選択補助処理の種類に対応する制御情報を用いて前記ロボットを制御するステップと、
    　を有する制御方法。
          Computer runs,
    Steps to acquire operation information indicating the contents of operations by the operator who operates the robot, and
    A step of selecting one or more auxiliary processes from a plurality of auxiliary process candidates for assisting the operation of the operator, and
    A step of assisting the operator by executing the selected auxiliary process, which is one or more of the selected auxiliary processes, and
    A step of controlling the robot using the control information corresponding to the type of the selection assist processing created based on the operation information while executing the selection assistance processing.
    Control method having.
PCT/JP2020/006260 2019-03-20 2020-02-18 Control device and control method WO2020189143A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019052443 2019-03-20
JP2019-052443 2019-03-20

Publications (1)

Publication Number Publication Date
WO2020189143A1 true WO2020189143A1 (en) 2020-09-24

Family

ID=72520169

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/006260 WO2020189143A1 (en) 2019-03-20 2020-02-18 Control device and control method

Country Status (1)

Country Link
WO (1) WO2020189143A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022196011A1 (en) * 2021-03-15 2022-09-22 オムロン株式会社 Wearable instrument
CN115225682A (en) * 2021-04-15 2022-10-21 本田技研工业株式会社 Management server, remote operation system, remote operation method, and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018096902A1 (en) * 2016-11-22 2018-05-31 パナソニックＩｐマネジメント株式会社 Picking system and method for controlling same
WO2018212226A1 (en) * 2017-05-17 2018-11-22 Ｔｅｌｅｘｉｓｔｅｎｃｅ株式会社 Control device, robot control method, and robot control system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018096902A1 (en) * 2016-11-22 2018-05-31 パナソニックＩｐマネジメント株式会社 Picking system and method for controlling same
WO2018212226A1 (en) * 2017-05-17 2018-11-22 Ｔｅｌｅｘｉｓｔｅｎｃｅ株式会社 Control device, robot control method, and robot control system

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022196011A1 (en) * 2021-03-15 2022-09-22 オムロン株式会社 Wearable instrument
CN115225682A (en) * 2021-04-15 2022-10-21 本田技研工业株式会社 Management server, remote operation system, remote operation method, and storage medium
CN115225682B (en) * 2021-04-15 2024-01-19 本田技研工业株式会社 Management server, remote operation system, remote operation method, and storage medium

Similar Documents

Publication Publication Date Title
EP3809241B1 (en) System and method for enabling communication through eye feedback
WO2020189143A1 (en) Control device and control method
CN111091732B (en) Cardiopulmonary resuscitation (CPR) instructor based on AR technology and guiding method
CN117441362A (en) Indication of remote control pairs with remote device functionality
EP3975909B1 (en) Operating mode control systems and methods for a computer-assisted surgical system
WO2020179128A1 (en) Learning assist system, learning assist device, and program
JP2011110620A (en) Method of controlling action of robot, and robot system
US20180316911A1 (en) Information processing apparatus
JP6730363B2 (en) Operation training system
CN109300528B (en) Cognitive rehabilitation training system and method
WO2020170974A1 (en) Management device, management method, and management system
US10585476B2 (en) Apparatus operation device, apparatus operation method, and electronic apparatus system
JP2018045059A (en) Education support device
WO2010142455A2 (en) Method for determining the position of an object in an image, for determining an attitude of a persons face and method for controlling an input device based on the detection of attitude or eye gaze
CN115334992A (en) Virtual console for controlling surgical robot
WO2019087495A1 (en) Information processing device, information processing method, and program
US20200179077A1 (en) Control device, control method, and surgical system
WO2018198272A1 (en) Control device, information processing system, control method, and program
US20230092405A1 (en) Systems and methods for providing context-based data for an industrial automation system
US10747308B2 (en) Line-of-sight operation apparatus, method, and medical device
JP6625266B1 (en) Robot controller
JP2020019127A (en) Cooperative operation support device
KR20230037147A (en) Apparatus for recognizing user command using non-contact gaze based heading information and method using the same
US20210038335A1 (en) Robot Assisted Surgical System with Clutch Assistance
US20210386283A1 (en) Display apparatus, display method, and display program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20773492

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20773492

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP