WO2021019787A1

WO2021019787A1 - Task distribution device, task distribution system, method, and program

Info

Publication number: WO2021019787A1
Application number: PCT/JP2019/030362
Authority: WO
Inventors: 征久唐子; 真司川上; 隆宏徳; 多田　有為
Original assignee: オムロン株式会社
Priority date: 2019-08-01
Filing date: 2019-08-01
Publication date: 2021-02-04
Also published as: JPWO2021019787A1; CN114144806A; JP2022126658A; JP7331928B2

Abstract

This task distribution device comprises: an operation terminal connection unit that manages a connection state of an operation terminal owned by an operator; a state reception unit that receives state information from a robot; an information control unit that selects, from among available operators, an operator for coping with an exceptional event when it is determined on the basis of the state information that the exceptional event has occurred in the robot; and a transmission unit that transmits, to the operation terminal of the selected operator, a request for coping with the exceptional event of the robot together with the state information on the robot.

Description

Task distribution device, task distribution system, method, and program

The present invention relates to a task distribution technique for assigning a task for dealing with an exception event that occurs in a robot to an appropriate operator.

With the progress of information technology in recent years, robots can judge the surrounding situation and perform processing autonomously. However, since robots are not automated to handle all situations, humans must deal with this exceptional event in the event of an exceptional situation that the robot cannot handle.

Since the robot can be operated remotely, it is conceivable to ask any operator in a remote location to handle the exception event by remote control.

Patent Documents

1 and 2 can be mentioned as a technique for responding to an exception event by remote control.

Patent Document 1 discloses that the server discloses a task desired to be remotely controlled and determines a remote operator who requests the task in response to a bid from the remote operator. However, this method has a problem of lacking responsiveness because the task is not executed without a bid from the remote operator.

The autonomous traveling robot disclosed in Patent Document 2 is configured to output a signal prompting remote control when an obstacle is detected and avoid the obstacle by remote control. This autonomous traveling robot is configured to store operation information when an obstacle has been avoided in the past and to refer to it when it is necessary to newly avoid the obstacle. In Patent Document 2, it is assumed that the autonomous traveling robot and the remote operator are associated with each other in advance. If the robot and the remote operator have a one-to-one correspondence, there is a problem that a large number of remote operators are required. Further, when one remote operator is associated with a plurality of robots, there is a problem that it cannot be dealt with when an event requiring remote control occurs in a plurality of robots at the same time.

International Publication No. 2008/140011 Japanese Unexamined Patent Publication No. 2013-206237

An object of the present invention is to be able to quickly deal with an exceptional event that occurs in a robot.

In order to achieve the above object, the present invention adopts the following configuration.

Specifically, the first aspect of the present invention is an operation terminal connection unit that manages the connection state of the operation terminal owned by the operator, a state reception unit that receives state information from the robot, and the state reception unit based on the state information. When it is determined that an exception event has occurred in the robot, the information control unit that selects the operator for dealing with the exception event from the available operators, and the selected operator It is a task distribution device including a transmission unit that transmits a response request for an exception event of the robot together with the state information of the robot to an operation terminal.

In the present disclosure, the robot includes an arbitrary device capable of autonomously performing some movement. Typically, a robot is a device that includes a sensor and an actuator and is configured to autonomously control the actuator based on information obtained from the sensor. The robot may be a mobile robot capable of autonomous movement, or may be a fixed robot that cannot move autonomously.

The robot state information may be at least information that can grasp the current state of the robot. Examples of the robot state information include sensor information obtained from a sensor provided by the robot and analysis results analyzed by the robot based on the sensor information. It should be noted that it is not necessary that all the state information of the robot is transmitted to the task distribution device, and at least the state information that can determine that an exception event has occurred is transmitted to the robot. Further, receiving the state information from the robot includes both receiving the state information from the robot and receiving the state information via other devices.

An exceptional event is an event in which the robot cannot automatically process or is judged to be inappropriate. Alternatively, an exceptional event can be regarded as an event that requires human operation or confirmation. The exception event may be included in the status information, or may be determined by the task distribution device from the status information.

The management of the connection status of the operation terminal performed by the operation terminal connection unit may include establishing the online status of a plurality of operation terminals and monitoring the online status. The online state includes a state in which the operation terminal can immediately respond to the notification from the task distribution device.

When it is determined that an exception event has occurred in the robot, the information control unit selects an operator to deal with this exception event. Whether or not an exception event has occurred can be determined from the contents of the state information. However, in the embodiment in which the status information is notified to the task distribution device only when an exception event occurs, when the information control unit receives the status information, the exception event occurs without referring to the contents thereof. It can be determined that there is. For example, the information control unit may select an operator associated with the operating terminal that is online as an operator for dealing with the received exception event. The information control unit may further select an operator who satisfies the condition that the exception event of another robot is not being dealt with. However, this does not apply if there is no operator who can handle it.

Since this aspect has the above configuration, it is possible to select an operator who can quickly respond to an exception event. In other words, when an exception event occurs in the robot, it is possible to quickly respond to this exception event.

The task distribution device according to this embodiment may further have an operator information storage unit that stores the skill of the operator. In addition, the information control unit may select an operator having a skill that matches at least one of the robot type and the exceptional condition.

According to such a configuration, an appropriate operator can be selected for the exceptional event that has occurred.

The task distribution device according to this embodiment includes a receiving unit that receives an operation command for the robot from the operation terminal, an output unit that outputs the operation command to the robot, a state of the robot, and the operator. It may further have a corresponding storage unit that stores the operation contents in association with each other.

In the present embodiment, the receiving means receives the response result for the exception event of the robot from the operation terminal, and the response storage unit stores the response result in association with the exception event of the robot and the operator. You may. The content of the response result is not particularly limited, and may indicate, for example, whether the exception event has been resolved or not resolved, or may require confirmation on site by another operator. ..

In this embodiment, the information control unit may evaluate the skill of the operator with reference to the corresponding information storage unit and update the operator information storage unit.

In this embodiment, the operator information storage unit stores the usage cost of the operator, and the information control unit refers to the corresponding storage unit and pays a reward according to the correspondence history of the operator. May be determined.

In this embodiment, the information control unit may classify the exception events of the robot and select an operator according to the type of the exception event as an operator for dealing with the exception event. For example, a corresponding operator or a group of operators may be predetermined for each type of exception event. In this way, the same type of exception event will be dealt with by a particular operator, and these operators will become more proficient in that type of exception event.

In this embodiment, the transmission unit may transmit the state information of the robot to the operation terminal of the administrator at the site where the robot is installed in response to the request from the operator. In this way, when the operator determines that the robot needs to be processed at the work site, the site manager can confirm it.

In this embodiment, the state information may include an image taken by a camera mounted on the robot and an analysis result of sensor information obtained from a sensor provided on the robot. When the operator refers to this information, it becomes easy to deal with the exception event.

A second aspect of the present invention is a task distribution system including a robot, an operation terminal for remotely controlling the robot, and the above-mentioned task distribution device.

In this embodiment, when the operation terminal receives a request for coping with an exception event of the robot, the operation terminal is configured to display the state information and a graphical user interface (GUI) for operating the robot on a display unit. You may.

In this embodiment, the robot may be a fixed robot or a mobile robot. The mobile robot may be a mobile robot that has a mobile device, a sensor, and a camera and can move autonomously. It may be a robot in which an arm type robot is mounted on a mobile robot.

The present invention can be regarded as a task distribution device or task distribution system having at least a part of the above configuration or function. The present invention also includes a task distribution method including at least a part of the above processing, a program for causing a computer to execute the method, or a computer-readable recording medium in which such a program is recorded non-temporarily. It can also be regarded as. Each of the above configurations and processes can be combined with each other to construct the present invention as long as there is no technical contradiction.

According to the present invention, it is possible to quickly deal with an exceptional event that occurs in a robot.

FIG. 1 is a diagram showing a configuration example of a task distribution system according to an embodiment. FIG. 2 is a flowchart showing a process performed by the task distribution server in the embodiment. FIG. 3 is a flowchart showing a process performed by the task distribution server in the embodiment. FIG. 4 is a diagram illustrating a process performed by the information control unit in the embodiment. FIG. 5 is a diagram showing an example of the operator information DB in the first embodiment. FIG. 6 is a diagram showing an example of the correspondence information DB in the first embodiment. FIG. 7 is a diagram showing an example of an image taken by the robot in the first embodiment. FIG. 8 is a diagram showing an example of an image taken by the robot in the first embodiment. FIG. 9 is a diagram for explaining the result of dealing with the exception event in the first embodiment. FIG. 10 is a diagram showing a configuration example of the robot according to the second embodiment. 11A and 11B are diagrams showing an example of an image taken by the robot in the second embodiment. FIG. 12 is a diagram showing an example of the operation information DB in the second embodiment. FIG. 13 is a diagram showing an example of the correspondence information DB in the second embodiment. FIG. 14 is a diagram showing a graphical user interface for remote control of the robot displayed on the operator terminal in the second embodiment.

<Application example>
FIG. 1 is a block diagram showing a configuration example of a task distribution system to which the present invention is applied. The task distribution system includes a task distribution server (task distribution device) 1, an operation terminal 2, and robots such as a fixed robot 3a and a mobile robot 3b. Hereinafter, when it is not necessary to distinguish between the fixed robot 3a and the mobile robot 3b, it may be simply referred to as the robot 3. The task distribution server 1 and the operation terminal 2 are connected via a wide area network such as the Internet, and the operator is located in a remote place away from the work position of the robot 3.

In this system, the task distribution server 1 (hereinafter, also simply referred to as server 1) selects an operator for responding to an exception event when an exception event occurs in the robot 3, and the selected operator is selected. Deliver a task to deal with exceptions to. The server 1 holds the coping skill for each type of robot 3 or the type of exception event as information for each operator, and provides the robot 3 in which the exception event occurs or the operator having the skill to deal with the exception event. It may be configured to be selected.

The operator uses the operation terminal 2 to operate the robot 3 to deal with the exception event. The operation instruction to the robot 3 is transmitted to the robot 3 via the server 1, and the server 1 can store the history of the operation contents.

[Constitution]
The server 1 is a computer including a CPU 11, a memory 12, a storage 13, a network interface (not shown), and the like. The server 1 provides the following functions by loading the program stored in the storage 13 into the memory 12 and executing the program. That is, the server 1 has a state reception unit 101, an information control unit 102, a transmission unit 103, an operation terminal connection unit 104, a reception unit 105, an output unit 106, an operator information DB 107, and a correspondence information DB 108 as its functional units. DB means a database.

The state reception unit 101 receives state information from the fixed robot 3a and the mobile robot 3b. The state receiving unit 101 receives the state information from the fixed robot 3a via the robot controller 5a, and receives the state information from the mobile robot 3b via the wireless communication device 6. The state information may be transmitted from the robot 3 to the server 1 only when an exception event occurs in the robot 3, or may be transmitted from the robot 3 to the server 1 regardless of whether or not an exception event has occurred.

The information control unit 102 selects an operator to deal with the exception event when an exception event occurs in the robot 3, saves the operation content (correspondence history) from the operation terminal, evaluates the skill of the operator, and sends the operator to the operator. Performs processing such as reward determination. Details will be described below.

The transmission unit 103 transmits a response request for an exception event of the robot 3 and the state information of the robot 3 to the operation terminal 2.

The operation terminal connection unit 104 establishes a connection with the operation terminal 2 and monitors the online state of the operation terminal 2. Here, the online state means a state in which the operation terminal 2 can immediately communicate with the server 1 and send an operation command to the robot 3. Further, the online state not only means the connection state between the server 1 and the operation terminal 2, but may also mean that the operator can immediately respond to the request. Therefore, the operation terminal connection unit 104 may make an inquiry when, for example, the information representing the input by the operator is not received from the operation terminal 2 and it is unclear whether the operator can respond immediately. In addition, the online state may establish a connection by a push notification to the operation terminal 2. Further, the operation terminal connection unit 104 not only keeps the operation terminal 2 online, but also whether or not the operation terminal 2 is dealing with an exception state of any robot 3, in other words, a new exception event of the robot 3. It also manages whether or not it is possible to respond to. Information about the state of the operation terminal 2 is stored in, for example, the memory 12.

The receiving unit 105 receives the information input by the operator into the operation terminal 2 and transmitted to the server 1. The information transmitted from the operation terminal 2 includes an operation command for the robot 3 and a response result for an exception event. The operation command to the robot 3 is transmitted to the robot 3 via the output unit 106, and is stored in the correspondence information DB 108 as a correspondence history. In addition, the result of dealing with the exception event is also saved as a correspondence history in the correspondence information DB.

The operator information DB 107 stores information about the operator. The correspondence information DB 108 stores the contents of the correspondence history for the exception event. These databases will be described below with reference to FIGS. 5 and 6.

The output unit 106 transmits the operation command of the robot 3 input from the operation terminal 2 to the robot 3. An operation command to the fixed robot 3a is transmitted to the robot controller 5a, and the robot controller 5a controls the robot 3a according to the content of the operation command. The operation command to the mobile robot 3b is transmitted to the robot controller 5b in the mobile robot 3b via the wireless communication device 6, and the robot controller 5b controls the mobile robot 3b according to the content of the operation command. Hereinafter, when it is not necessary to distinguish between the robot controller 5a and the robot controller 5b, it may be simply referred to as the controller 5.

The operation terminal 2 is an arbitrary computer that can communicate with the server 1 and can remotely control the fixed robot 3a and the mobile robot 3b. Examples of the operation terminal 2 include a desktop PC, a laptop PC, a tablet terminal, and a smartphone terminal. The operation terminal 2 may be a dedicated teaching pendant for remotely controlling the robot. The operation terminal 2 provides the user with different types of operation GUIs depending on the type of the robot 3. This GUI may be generated by the operation terminal 2 itself, or may be provided by the server 1 or other devices. The operator can grasp the state information of the robot 3 via the GUI of the operation terminal 2. This state information includes sensor information obtained from the sensor of the robot 3, an analysis result of the sensor information, and an image taken by the camera of the robot 3. The operation terminal 2 receives information on a state requiring processing from the server 1, displays it on the screen, and prompts the operator to operate it. The operator can input and transmit an operation command for the robot 3 and the result of the operation for the robot 3 via the GUI of the operation terminal 2. The input information is transmitted from the operation terminal 2 to the receiving unit 105 of the server 1.

Robot 3 is a device that can autonomously execute some kind of operation. The fixed robot 3a has an actuator and a sensor, and executes a process by controlling the actuator according to the information obtained from the sensor. The control content of the fixed robot 3a is determined by the robot controller 5. Examples of the fixed robot 3a include, but are not limited to, FA robots, NC machine tools, molding machines, press machines, automatic feeding devices for breeding animals and plants, and automatic irrigation and fertilizing devices for cultivated plants. ..

The mobile robot 3b is a device that is equipped with a moving device and a robot controller 5b in addition to an actuator and a sensor, can move autonomously, and executes some operation at the moving destination. Examples of the mobile robot 3b include, but are not limited to, a transport robot, a cleaning robot, a waiter robot, a security robot, a robot combined with an arm type robot, and the like.

It is also preferable that both the fixed robot 3a and the mobile robot 3b have a camera (imaging means) as a sensor. The camera may be any camera such as a visible light camera, an infrared camera, and a three-dimensional camera.

The robot controller 5 controls the robot 3 according to a predefined program based on the information obtained from the sensor of the robot 3. The robot controller 5 may be configured so that the control content can be dynamically changed by online learning.

The robot controller 5 is configured to be able to determine whether an event has occurred in which the robot 3 cannot automatically perform processing or it is not appropriate to perform processing automatically. Such an event is referred to as an exceptional event in the present disclosure. As an example of the exceptional event, there is an event in which the operation instruction transmitted from the robot controller 5 to the robot 3 and the operation result returned from the robot 3 do not match, and the robot 3 does not operate according to the operation instruction. An example of an exceptional event is the occurrence of an event that requires human confirmation. Examples of such an event include an abnormal temperature rise of the robot 3, generation of abnormal noise, and abnormality of the operation target object.

[Database]
The operator information DB 107 stores information about the operator. For example, as shown in FIG. 5, the operator information DB 107 stores the operator ID 501, the operation terminal information 502, the skill information 503, the hourly wage 504, and the work time history 505. These are merely examples, and the operator information DB 107 may not store all of these information, or may store other information.

The operator ID 501 is an identifier for uniquely identifying the operator in the system. The operator logs in to the system using this operator ID and a password (not shown) to keep the system online. The operation terminal information 502 is information about the operation terminal 2 that keeps the online state, and includes information necessary for identifying and connecting the operation terminal 2 from the server 1, for example, an IP address and a MAC address. The operation terminal information 502 includes information indicating the characteristics of the operation terminal 2 itself such as a PC or a smartphone terminal, and information on a special user interface (UI) connected to the operation terminal, and these information are used for task matching. Be done. The special UI is, for example, a sensor worn on the hand to operate the finger portion of the end effector.

Skill information 503 includes skill possession information 503a and skill evaluation 503b. Skill 1, skill 2, ... Skill N shown in the figure are skills related to the robot 3 connected to the task distribution system, and these are collectively referred to as skill information 503. If the operator has a skill that can be operated for skill N, YES is registered in the skill possession information 503a, and NO is registered otherwise. If the operator has skill N, the evaluation is registered as skill evaluation 503b. The value of the skill evaluation 503b is determined by the number of experiences with skill N, the speed of response, the evaluation by the evaluator (administrator) in the system, and the like. For example, a value from 1 to 10 can be set, and 10 can be set as the highest rank. Further, the server 1 may automatically update the skill evaluation of the operator based on the result of responding to the exception event of the operator.

The hourly wage 504 is an hourly reward set for the skill of the operator. From the employer's point of view, the hourly wage 504 represents the operating cost of the operator. Here, the unit of hourly wage is used as a point, so that the reward can be calculated according to the minimum wage (or other wage level) of the operator's residential area. The working time history 505 records the cumulative total of working hours of the operator. The work time history 505 can total the hours worked by the operator on a daily basis, a weekly basis, a monthly basis, etc., and can be used for reward calculation.

Correspondence information DB 108 stores the contents of the correspondence history for the exception event. In the correspondence information DB 108, the state of the robot 3 and the operation content from the operator are stored in association with each other for each exception event. For example, as shown in FIG. 6, the correspondence information DB 108 stores the exception event occurrence time 601, the robot ID 602 in which the exception event occurred, the state information 603, the operation information 604, the operation result 605, the operator ID 606, and the correspondence time 607. Will be done. These are merely examples, and the operator information DB 107 may not store all of these information, or may store other information.

The exception event occurrence time 601 represents the time when the exception event occurs in the robot 3 and the server 1 (state reception unit 101) receives the status information of the exception event from the robot controller 5. The robot ID 602 is an identifier for uniquely identifying the robot 3 in which the exception event has occurred in the system.

The status information 603 is information such as an exceptional status and a status sent from the robot 3. The operation information 604 is information instructing an operation to be given to the robot 3 by the operator in order to return the exceptional state to the normal state based on the sent state information. The state information 603 and the operation information 604 will be described in detail when the examples are described.

The operation result 605 is information for recording the result of the robot 3 being operated by the operation information. The operator ID 606 is an identifier for uniquely identifying the operator who has dealt with the exception event. The response time 607 represents the time required to return the exceptional state to the normal state. If the normal state cannot be restored, it indicates the time until the operator determines that the problem cannot be solved by remote control alone.

[processing]
2 and 3 are flowcharts showing the processes performed by the server 1 in the task distribution system.

In step S101, when the receiving unit 105 of the server 1 receives the connection request from the operation terminal 2, the operation terminal connection unit 104 manages the connection of the operation terminal 2, that is, establishes the connection and monitors the online state in step S102. .. The operation terminal connection unit 104 also manages whether or not the operation terminal 2 is dealing with an exception event of any robot 3. The connection management by the operation terminal connection unit 104 continues even after step S102 until the operation terminal 2 disconnects.

In step S103, the state receiving unit 101 receives the state information from the robot 3 (fixed robot 3a or mobile robot 3b). Here, since it is assumed that the state information is transmitted from the robot 3 to the server 1 only when an exception event occurs, the exception event occurs in the robot 3 when the state information is received. Can be automatically determined. However, the robot 3 may be configured to transmit state information to the server 1 regardless of the presence or absence of an exception event. In this case, the information control unit 102 refers to the state information and an exception event occurs in the robot 3. If it is determined that an exception event has occurred, the process after step S104 may be executed.

In step S104, the information control unit 102 matches the task for dealing with the exception event with the operator, that is, selects the operator for dealing with the exception event. Specifically, the information control unit 102 selects an operator to which a coping task is assigned from among the available operators in consideration of the skill of the operator and the exception event. As shown in FIG. 4, the information control unit 102 inputs the type (type) and state (defect state) of the robot 3 included in the state information received by the state reception unit 101, and the information included in the operator information DB 107. Compare and extract the appropriate operator. The first condition for the operator to match (assign) a task is to be online. Another condition is that the coping skill for the type of robot in which the exception event occurs or the coping skill for the type of exception event is at the required level. It is also good to adopt the condition that other tasks are not being executed. However, if all the operators who can handle the exception event are executing other tasks, the operators who are executing the other tasks may be matched. Further, the matching operator may be determined in consideration of the reward for the operator. For example, the information control unit 102 may determine the operator who satisfies the condition that the skill is equal to or higher than the required level and has the highest score determined based on the skill and the reward as the matching operator. Here, it is assumed that the higher the required skill, the higher the score, and the lower the reward, the higher the score.

Note that the information control unit 102 may classify the exception events of the robot into several types and assign each exception event to a specific operator according to the type of the exception event. Here, the specific operator corresponding to the exception event is typically a plurality of operators, and it is preferable that the information control unit 102 assigns the exception event to the operators in the same group according to the exception type. However, it does not exclude that a certain kind of exception event is assigned to one specific operator.

In step S105, the information control unit 102 identifies the operation terminal 2 of the operator selected in step S104, and requests the specified operation terminal 2 to deal with exception handling and the state information of the robot 3. Send. As shown in FIG. 4, the information control unit 102 refers to the operator information DB 107 and acquires the operation terminal information of the selected operator. The operation terminal information is information for communicating with the operation terminal 2, and is, for example, an IP address or a MAC address. The operation terminal information may include terminal type information. The information control unit 102 passes the state information received from the robot 3 via the state reception unit 101 and the operation terminal information to the transmission unit 103 as transmission information. The transmission unit 103 transmits the state information and the task execution request to the operation terminal 2 specified by the operation terminal information.

When the operation terminal 2 receives the task of coping with the exception event and the state information of the robot 3, the GUI for operating the robot 3 is displayed on the display unit. In this GUI, it is possible to confirm the state information of the robot 3, input the operation information (operation command) to the robot 3, input the coping result, and the like. The operation information for the robot 3 may be a command for remotely controlling the robot 3 by real-time control, or may be a command for processing a series of procedures. The operation information to the robot 3 input to the operation terminal 2 is transmitted from the operation terminal 2 to the server 1 and received by the receiving unit 105.

In step S106, the receiving unit 105 receives information from the operation terminal 2. The reception information received by the reception unit 105 includes operation terminal information and operation information.

In step S107, the information control unit 102 extracts the operation information from the received information and transmits the operation information from the output unit 106 to the target robot 3. As a result, the robot 3 can be made to perform the operation instructed by the operator.

Although not shown in the flowcharts of FIGS. 2 and 3, when the state of the robot 3 is updated by an operation command or the like from the operation terminal 2, the state information of the robot 3 is operated directly from the robot 3 or via the server 1. It may be transmitted to the terminal 2. In this way, the operator can grasp the latest state of the robot 3.

After the robot 3 operates according to the operation command from the operator, the operator evaluates the result of the operation, inputs it to the operation terminal 2, and transmits it to the server 1. Examples of operation results include whether or not the handling of exception events has been completed, and whether or not it is necessary for the administrator to confirm on-site afterwards. It should be noted that the completion of the countermeasure includes the case where the normal state is restored and the case where it is determined that the countermeasure is unnecessary (not abnormal). It is also possible that the handling of the exception event is completed by the state receiving unit 101 receiving the state information that the robot 3 has returned to the normal operation according to the operation command from the operator. In this case, the task distribution server 1 transmits the operation result to the operation terminal 2.

In step S108, the receiving unit 105 receives the operation result from the operation terminal 2.

In step S109, the information control unit 102 associates the state information received from the robot 3 with the operation information and the operation result received from the operation terminal 2 and stores them in the correspondence information DB 108. As a result, it is possible to record as a history what kind of operation the operator performed when the robot 3 was in what state. Further, the information control unit 102 may store the time required from the start to the completion of the task for dealing with the exception event by the operator in the correspondence information DB 108. As a result, the working time of the operator can be grasped, and it can be used for the operator's reward calculation and the operator's skill evaluation. Information such as working hours and rewards may be stored in the operator information DB 107.

In step S110, the information control unit 102 determines whether or not the handling of the exception event is completed. This determination may be made based on the operation result information transmitted from the operation terminal 2. When the countermeasure is not completed (S110-NO), the process returns to step S105 and the robot 3 is operated based on the instruction from the operator. Although the same operator as the previous one is requested to take action here, the operator may return to step S104 to reselect the operator to take action. In this case, if necessary, the previous operator may be excluded from the candidates and another operator may be selected. If the countermeasure is completed (S110-YES), the process proceeds to step S111.

In step S111, the information control unit 102 evaluates the skill of the operator based on the correspondence history and the result stored in the correspondence information DB 108, and updates the skill stored in the operator information DB 107. Skill evaluation takes into account the results of dealing with exception events, the content of coping, and the time required for coping.

In step S112, the information control unit 102 determines the amount of remuneration to be paid to the operator based on the hourly remuneration of the operator stored in the operator information DB 107 and the time required to deal with the exception event. To do.

Here, skill evaluation (S111) and reward calculation (S112) are performed every time an exception event is dealt with, but these processes may be performed separately at an appropriate timing.

Further, in the above example, it has been described that the operator performs some operation on the robot 3 when an exception event occurs in the robot 3, but the operator does not actually perform the operation on the robot 3. May be good. For example, even when an exception event occurs in the robot 3, the operator may check the state of the robot 3 and confirm that no abnormality has actually occurred. In such a case, the operator may transmit from the operation terminal 2 to the server 1 that no action is required without transmitting the operation command to the robot 3.

[Advantageous effect]
According to the present embodiment, it is possible to have an operator at a remote location deal with an exceptional event generated by the robot 3. Here, the task distribution server 1 manages the connection with the operation terminal 2, and immediately selects an operator who can respond to the task distribution, so that a quick exception response is possible. In addition, considering the skill of the operator, an appropriate operator is selected according to the type of robot and the type of exception event and the task is distributed, so that appropriate exception handling is possible.

There is no need to make the operator stand by at the working position of the robot such as a factory, and the operator can operate the robot at multiple locations from a remote location. Therefore, if this system is introduced, it is not necessary to have the personnel corresponding to the exception handling of the robot on standby at the company's factory, so that the cost can be reduced.

In recent years, problems such as labor shortages and employment shortages in rural areas due to the declining birthrate and aging population have been pointed out, but this embodiment is a social infrastructure or platform that connects factories (robot work places) and remote labor force. Will contribute to solving these problems. With this system, the operator does not have to work in a specific place and can work from a remote place. Therefore, even people who have difficulty commuting long distances can provide a comfortable working environment, and even elderly people and local residents can work.

In addition, foreign labor and immigration to Japan are being discussed in order to solve the labor shortage in Japan. The operator in this system does not have to be located in Japan and may be located in a foreign country. If the interface of the operation terminal is in a foreign language, foreigners can operate it from abroad, and there is also the effect that labor can be imported without immigrating foreigners.

<Example 1>
Here, the task distribution system to which the present invention is applied will be described more specifically by taking as an example a case where the mobile robot 3b is a delivery robot that delivers a parcel or the like by automatic operation.

The delivery robot in this embodiment is a mobile robot that employs wheels as a moving mechanism. Here, an opposed two-wheel type delivery robot will be described as an example, but the moving mechanism of the delivery robot may be a wheel moving mechanism other than the opposed two-wheel type, or a multi-legged or endless track type moving mechanism. The delivery robot has a storage unit for stably housing the delivered product. In addition, the delivery robot has various sensors such as a position information acquisition sensor, a camera, a lidar, a millimeter wave radar, an ultrasonic sensor, and an acceleration sensor, and a built-in robot controller is based on the sensor information obtained from these sensors. 5 controls the movement and other movements of the delivery robot.

The delivery robot sends status information to the server 1 when an exception event occurs. In addition, it can operate based on an operation command transmitted from the operation terminal 2 via the server 1.

Here, it is assumed that an abnormal state (exception event) occurs in which the delivery robot is unmanned, that is, during delivery by automatic driving, the left wheel falls into a hole on the road and stops in a derailed state. Attempting to deal with the situation around the delivery robot without knowing it can lead to a more dangerous situation. For example, if the hole is larger than the delivery robot, there is a risk that the delivery robot will fall into the hole. Therefore, when the delivery robot determines from the information of the acceleration sensor that the aircraft is not leveled, it determines that an exception event has occurred and notifies the server 1 of the occurrence of the exception event together with the status information. It should be noted that the time at which the exception event is judged to have occurred depends on the level of the control technology of the mobile robot and the required safety level, and the above criteria are merely examples.

When the state reception unit 101 receives the state information from the mobile robot 3b (S103), this information is stored in the correspondence information DB 108. As shown in FIG. 6, the occurrence time 601 of the exception event, the robot ID 602 in which the exception event occurred, and the state information 603 are stored in the database.

The state information 603 here includes position information, vehicle body tilt, wheel state, cargo, and images taken by a camera mounted on the robot 3b. The position information is the position information obtained from the GPS device (position information acquisition sensor) mounted on the mobile robot 3b. The vehicle body tilt is information indicating the front / rear / left / right tilt of the vehicle body obtained from the tilt sensor (accelerometer) mounted on the mobile robot 3b. For example, when a tilt exceeding 4 degrees (absolute value) occurs, it can be determined to be abnormal. In this example, the left-right tilt is −5 degrees (minus indicates a downward slope to the left), and the absolute value of the tilt angle exceeds the threshold value, so that the robot 3b is determined to be in an abnormal state.

The state of the wheels is stopped. This is because the robot 3b stopped driving the wheels because it detected an abnormality. The cargo information is the information input by the delivery source. The operator can determine what kind of operation and how much should be performed based on the cargo information. For example, as shown in this example, if the cargo is a parcel, it can be handled roughly. On the other hand, if the cargo is food and drink, it needs to be handled carefully. Further, if the cargo is food and drink and the vehicle body inclination is a large value such as 45 degrees, it can be considered that it is better to contact the delivery source than to recover from the abnormal state. This is because even if the food or drink rolls over in the container of the mobile robot 3b and is delivered to the delivery destination, it is considered that the food or drink is in a meaningless state.

The image from the robot camera is an image from the camera attached to the vehicle body of the mobile robot 3b (see FIG. 7). The camera is mounted to capture at least the front in the direction of travel. A plurality of robot cameras may be attached around the vehicle body to capture an around-view image taken around the vehicle body. Alternatively, it may be an omnidirectional image (360 degree image) taken by an omnidirectional camera (360 degree camera).

The state control unit 102 of the server 1 can determine from the state information transmitted from the delivery robot (mobile robot) 3b that an exception event has occurred in the delivery robot 3b, and therefore, an operator for dealing with this exception event. The task matching process for selecting is performed (step S104). Since the exception event in this example is a malfunction of the delivery robot 3b, the information control unit 102 deals with this exception event from among the operators who are online and do not process other tasks. Select an operator who has the skills to do so.

Consider the case where the operator 1 and the operator 2 shown in FIG. 5 exist as operators who are online and do not process other tasks. Here, both the

operators

1 and 2 have the skills of "mobile robot, delivery, and troubleshooting". The information control unit 102 selects an operator in consideration of the evaluation of the skill of each operator and the hourly wage. For example, in this example, the operator 2 having a higher evaluation of skill 2 and a lower hourly wage is selected.

The information control unit 102 transmits a request for dealing with an abnormal event that has occurred in the delivery robot 3b and the state information of the delivery robot 3b to the operation terminal of the selected operator (step S105). The operation terminal 2 that has received such information displays a screen including the state information of the delivery robot 3b and the GUI for operating the delivery robot 3b. The operation terminal 2 is provided with a display unit for displaying such information and a user interface for inputting an operation instruction to the robot 3b. The input of the operation information may be performed via the keyboard or may be performed by touching the screen. As an example of another user interface, there is an interface for inputting simple instructions (program, command sequence) for moving the mobile robot 3b.

For example, the state information of the delivery robot (information shown in the state information 603 of FIG. 6) and the image (FIG. 7) taken by the camera mounted on the delivery robot 3b are displayed on the operation terminal 2. Based on this information, the operator can first determine whether or not there is an abnormal state that requires some action, and if it is an abnormal state, whether or not it is possible to deal with it by remote control. In some cases, determine what kind of operation should be performed.

In this example, the operator can read the following from the image shown in FIG. 7 and other sensor information.
-Since the entire landscape is downward-sloping, the mobile robot 3b is tilted to the left.
・ There is a high possibility that the left wheel has come off due to the angle of inclination.
-Since there is no gutter along the guardrail, there is a high possibility that the wheel has come off in an unexpected hole.
-Since there are guardrails on the left and right, it is a relatively safe place to operate and move the mobile robot 3b.
-When changing the direction of the mobile robot 3b and moving it back or straight, it is necessary to operate it so as not to touch the left and right guardrails.
・ Because the guardrail on the left side is closer, it is safe to back up to the right rear within 1 meter in order to recover from the derailed state.

The state information and images sent when an exceptional event occurs are diverse, and the above matters can be instantly grasped by humans, but it is difficult for computers (AI, etc.). Based on the image from the mobile robot 3b and other state information, the operator can conclude that the left wheel of the delivery robot has fallen into the hole. Further, the operator can determine from the surrounding conditions that there is no problem even if the mobile robot 3b moves backward about 1 meter to the right in the traveling direction, and that it is necessary to move the mobile robot 3b for recovery. Such situation determination and operation of the mobile robot 3b cannot be executed unless it is programmed in the mobile robot 3b. It is not realistic to perform programming that can handle all situations in advance, and it is still appropriate to let humans (operators) decide to properly deal with and operate various situations.

Based on such a judgment, the operator determines the operation command to be given to the mobile robot 3b as follows. First, as a first step, it is instructed to rotate the left wheel at −0.05 m / s while keeping the right wheel locked. Since the left wheel has fallen into the hole, the operator locks the right wheel outside the hole and instructs the operation to slowly retract the left wheel around the right wheel. By this operation, the left wheel hangs on the edge of the fallen hole, and the left wheel can be slowly pulled up.

As the second step, when the vehicle body tilt reaches 0 ± 2 degrees (normal state), the right wheel speed is instructed to be changed to -0.05 m / s. When the inclination of the car body is clearly within the normal range, it can be determined that the left wheel has been lifted from the hole. After that judgment, the right wheel is retracted at the same speed as the left wheel, and the mobile robot 3b is moved away from the hole. If the vehicle body tilt does not become normal even after the operation of the first step is continued for a certain period of time, it may be instructed to end abnormally.

As the third step, the operator stops the wheel operation after 10 seconds, takes an image with the front camera of the robot 3b, and instructs the execution of self-diagnosis whether or not the robot 3b has returned to the normal state. By continuing the backward operation of the second step for 10 seconds, it means that the vehicle has moved to a position 0.5 meters away from the position where it fell into the hole. This value should be set according to the surrounding situation, and it is appropriate to let a human (operator) determine the value. It is considered that the camera mounted on the robot 3b can take a picture of the hole at a distance of 0.5 meter from the hole.

The self-diagnosis is executed as the operation of the third step for reconfirmation. Not only is the vehicle body tilt within the normal range, but the mobile robot 3b can self-diagnose whether there is a functional problem, and if it returns to the normal state, the delivery work can be continued from there. Even if the mobile robot 3b determines that it is normal as a result of the self-diagnosis, the operator may determine that the abnormal state continues. In such a case, the determination by the operator may be prioritized.

The operator transmits the operation command consisting of the above first to third steps from the operation terminal 2 to the mobile robot 3b via the server 1 (steps S106 and 107). In this embodiment, it is assumed that the operator collectively transmits the instructions of the first to third steps to the mobile robot 3b. However, the instruction may be transmitted in the unit of each step, or the instruction may be transmitted in a finer unit. Further, when the state information from the mobile robot is transmitted in real time, the operator may remotely control the mobile robot in real time.

The mobile robot 3b operates in accordance with the above operation command, and transmits the information obtained as a result to the operation terminal 2 via the server 1 or directly. As a result of the first to third steps, the mobile robot 3b can recover from the derailed state, retreat from the hole, and acquire an image of the hole as shown in FIG.

In this example, the operator can read the following from the image shown in FIG.
-The mobile robot 3b is horizontal from the entire landscape.
・ As instructed in the operation information, it moved to the right rear from the stop position.
・ A hole could be recognized from the image, and the left wheel was removed from the hole, resulting in an abnormal condition.
-From the above, the mobile robot 3b has returned to the normal state.

The server 1 can record that there is a hole on the road together with the GPS position information. As a result, as shown in FIG. 9, the position 901 of the hole on the road can be grasped, and the mobile robot 3b traveling on this road will determine the traveling route 902 with reference to the position 901 of the hole in the future to reach the hole. You will be able to avoid the fall of.

In addition to the image shown in FIG. 8, information indicating that the vehicle body tilt is 0 degrees and the self-diagnosis result is normal is transmitted to the operation terminal 2 and displayed.

The operator refers to the state information transmitted from the mobile robot 3b, inputs the result of the operation, and transmits it to the server 1 (step S108). In this example, an operation result indicating that the normal state has been restored is input. The server 1 stores the transmitted operation result in the correspondence information DB 108 together with the operation content performed, the operator ID, and the time required for the correspondence (step S109). In this way, by storing the state information, the operation information, and the result information in association with each other, it is possible to accumulate know-how as to what kind of operation can be used to return to the normal state in what situation. The operation result information may be manually input to the operation terminal 2 by the operator, or may be caused by the mobile robot 3b to make a self-judgment.

In addition, it is assumed that trial and error may be repeated in order to return to the normal state without returning to the normal state with one operation. In this case, the operation command is input from the operator again. The server 1 may continuously request the same operator to take action, but may select a new operator and have another operator take action on the abnormal event.

<Example 2>
Next, the task distribution system to which the present invention is applied will be specifically described by taking as an example the case where the mobile robot 3b is an agricultural robot that manages the cultivation of tomatoes.

The mobile robot 3b in the present embodiment can at least move autonomously and patrol the field or the greenhouse, and has a function of taking a picture and transferring the robot. As shown in FIG. 10, the mobile robot 3b includes a moving mechanism 1011, a robot controller 1012, a position information acquisition device 1013, a camera 1014, a main body (running unit) including a power supply (not shown), a pesticide storage unit 1021, and a pesticide spraying unit. It includes an arm 1020 including a unit 1022, an illumination unit 1023, and a camera 1024. Here, one arm 1020 is provided with the functions of image capturing and pesticide spraying, but different arms may be provided with each function. In addition, the arm 1020 can harvest fruits and scrape and pick up leaves. Further, the mobile robot 3b may be provided with other centers such as a distance sensor and an acceleration sensor other than those shown in the figure.

The moving mechanism 1011 is a moving mechanism such as a wheel type or an endless track type. The robot controller 1012 is the same as that described above. The position information acquisition device 1013 is a device for identifying the position of the robot in a field or a vinyl house. The position information acquisition device 1013 may be a GPS device, or may be a device that reads a barcode or RFID attached to a field, a greenhouse, or a stock. The camera 1014 captures the surroundings of the robot in a relatively wide range.

The mobile robot 3b has a pesticide spraying function by the pesticide storage section 1021 and the pesticide spraying section 1022, and can automatically spray pesticides when the robot in need of pesticide spraying determines. The pesticide is sprayed, for example, from the arm. When there is a pest or there is a disease, the type and pathological condition of the pest can be identified from the image of the arm camera 1024, and the type of pesticide effective for the identified pest or pathological condition can be identified. For example, in the presence of Henosepilachna vigintioctopunctata, it can be determined that an aqueous solution of clothianidin is effective. The pesticide storage unit 1021 may be loaded with a plurality of types of pesticides so as to deal with a plurality of pests and medical conditions. It can be seen from the image obtained from the camera 1024 that an abnormality may have occurred, but the robot determines whether it is actually abnormal and what to do if it is abnormal. It can be difficult to determine automatically. For example, if the leaves are discolored, the causes are various, such as pests, molds, and viruses, and different pesticides should be used depending on the cause.

The robot arm 1020 can be moved back and forth and left and right, and can be inserted into thick leaves. A camera 1024 and an illumination unit 1023 are mounted on the tip of the robot arm 1020, and the direction of photography and illumination by the camera 1024 and the illumination unit 1023 can be changed by rotating the arm 1020. For example, by pointing the camera 1024 and the illumination unit 1023 upward, it is possible to take a picture of the back side of the leaf (see FIG. 11A).

FIG. 11A is a diagram showing an example of the image 1101 taken by the main body camera 1014 of the mobile robot 3b. Image 1101 shows the arm 1020 in an extended state. As shown, a discolored portion 1102 is observed on a part of the leaf surface of the tomato. It is assumed that the robot controller 1012 determines that the cause of the discoloration cannot be identified only from the image of the discolored portion 1102 on the leaf surface, and determines that it is necessary to operate the arm 1020 to take an image of the leaf back. The robot controller 1012 extends the arm to the position of the back of the leaf with the arm camera 1024 and the lighting unit 1023 facing upward, and takes a picture with the camera 1024. The above series of operations is merely an example, and the robot 3b may be controlled so as to inspect the entire number of strains even if no discolored portion is observed on the leaves. Further, if there is no abnormality, it is not necessary to send the image to the operation terminal.

FIG. 11B is a diagram showing an image 1103 of the back of a leaf taken by an arm camera 1024. By performing image processing on the image 1103, it can be recognized that there is also a discolored portion 1104 on the back of the leaf, and there are a plurality of small spots 1105 of the same size around the discolored portion 1104. This image processing may be performed by the robot controller 1012 or another processing device incorporated in the robot 3b, or may be performed by an external device to which the image is transmitted from the robot 3b. From the result of this image processing, it is highly possible that the robot controller 1012 has an abnormality, but it is unclear how to deal with it. Therefore, an event that requires the judgment of the operator (human). That is, it is determined that an exception event has occurred. Therefore, the robot controller 1012 transmits the state information to the server 1.

When the status reception unit 101 of the server 1 receives the status information from the mobile robot 3b (S103), this information is stored in the correspondence information DB 108. FIG. 12 shows an example of the correspondence information DB 108 in this embodiment. Since the table format itself is the same as that of the first embodiment (FIG. 6), detailed description thereof will be omitted. At this point, the date and time when the exception event occurred 1201, the robot ID 1202 where the exception event occurred, and the state information 1203 are stored in the database 108.

In this example, the state information 1203 includes position information, leaf surface state, leaf back state, leaf area, leaf area, fruit diameter, fruit color, and image. The position information includes latitude / longitude information, a vinyl house number, and a stock number acquired by the position information acquisition device 1013. Further, the position information includes height information, and this height information is the position imaged by the camera 1014 mounted on the robot main body 1010, or the arm camera 1024 when the image is taken by the arm camera 1024. Or the height of the arm 1020. The height of the arm camera 1024 or arm 1020 can be obtained from the encoder provided on the arm 1020. The height information represents the height of the fruit or leaf of interest.

The leaf table information indicates the result of determining the state of the leaf table by the image recognition technology. Examples of leaf surface information include "normal", "discolored part", "worm-eaten part", and "insect". Such image recognition can be realized by a machine learning method such as deep learning, and since it is known, detailed description thereof will be omitted.

The leaf back information indicates the result of determining the leaf back state by the image recognition technology. In this example, since the state of the leaf surface was "there is a discolored part", the mobile robot 3b takes an image of the leaf back and obtains an image recognition result. The information on the back of the leaf is the recognition result, and in this example, the recognition result that "there is a discolored part" and "there is an insect" is obtained. When the state of the leaf surface becomes "normal", the photography of the leaf back and the determination of the state may be omitted. Further, even if the state of the leaf surface is "normal", there may be an abnormality in the leaf back, so that both the leaf surface and the leaf back may be photographed to check the state.

The area of the leaf, the diameter of the fruit, and the color of the fruit are calculated based on the image taken by the main camera 1014 and the distance to the object (leaf or fruit) by the distance sensor. Since this information can be calculated using a known technique, detailed description thereof will be omitted.

The image from the camera 1014 mounted on the robot body and the image from the camera 1024 mounted on the robot arm are shown in FIGS. 11A and 11B, respectively.

The state control unit 102 of the server 1 can determine that an exception event has occurred from the state information transmitted from the mobile robot 3b, and therefore performs a task matching process for selecting an operator to deal with this exception event. (Step S104). Since the exception event in this example relates to the disease diagnosis of tomato, the information control unit 102 is for dealing with this exception event from among the operators who are online and do not process other tasks. Select an operator with skills.

FIG. 13 shows an example of the operator information DB 107 in this embodiment. Since the content of the operator information DB 107 is basically the same as that of the first embodiment, duplicate description will be omitted. Since this embodiment is an application to agricultural work, the skills related to agricultural work of each operator are stored. In the example shown in FIG. 10, "inspection of tomato leaf disease and pests" is set as one of the skill information. This skill means knowledge of the diseases and pests that occur in tomatoes and knowledge of the appropriate treatment for the diseases and pests found. Appropriate treatment is the ability to select or follow up effective pesticides against diseases and pests. As another skill, "harvesting tomatoes" is also set. This skill is a skill that allows you to look at the image sent from the robot, determine whether or not to harvest, and remotely control the arm to harvest.

Consider the case where the operator 1 and the operator 2 shown in FIG. 13 exist as operators who are online and do not process other tasks. Since the exception event in this example is related to the inspection of tomato leaf diseases and pests, the only operator who can handle it is operator 1. Therefore, the information control unit of the server 1 matches the exception event this time with the operator 1 (step S104).

The information control unit 102 transmits a request for dealing with an exception event and the state information of the mobile robot 3b to the operation terminal of the selected operator (step S105). The operation terminal 2 that has received such information displays a screen including the state information of the delivery robot and the GUI for operating the delivery robot.

FIG. 14 shows an example of the GUI 1400 displayed on the operation terminal 2. The GUI 1400 includes an image 1401 of the main camera 1014, an image 1402 of the arm camera 1024, and state information 1405 other than the image. The GUI 1400 also includes an input unit 1403 for operating (moving) the robot body 1010, an input unit 1404 for operating (moving) the arm 1020, an input unit 1406 for inputting a series of operation commands, and an operation result. The input unit 1407 for inputting the input contents and the transmission button 1408 for transmitting the input contents to the server 1 are included.

The operator inputs an operation command for the mobile robot 3b using the GUI 1400 and sends it to the server 1. For example, it is conceivable to input the following two-step operation command. The first step is a leaf pruning process. From the

images

1401, 1402 and the state information 1405, the operator confirms that the leaves have insects, and as a countermeasure, the leaves are pruned to prevent further spread of damage caused by the insects. Instruct to store in. The closed box is owned by the robot, and it is assumed that the robot can automatically perform the pruning process of the target leaf by the robot arm and the storage process in the closed box.

The second step is the spraying treatment of pesticides. The operator determines that the insect is likely to be an aphid, and inputs "YES" in the item for spraying pesticides, and then selects and inputs a pesticide acephate that is effective against aphids. Here, it is assumed that the robot is equipped with a plurality of types of pesticides, and that the pesticide spraying process can be performed automatically by the robot.

The operation command input by the operator is transmitted to the mobile robot 3b via the server 1 (steps S106 and S107). The mobile robot 3b operates in accordance with the above operation command, and transmits the information obtained as a result to the operation terminal 2 via the server 1 or directly. The operator can determine whether the leaf pruning treatment or the pesticide spraying treatment has been completed normally based on the image and the state information transmitted from the mobile robot 3b. If it is completed normally, enter "Processing completed" as the operation result. In addition, since the judgment by the operator is based on the image, the judgment may be wrong, and insects and diseases may have spread to places that cannot be found only by the robot image. The operator inputs post-confirmation as "necessary" as a matter of contact to those who can be confirmed directly on the farm. In addition, the operator may input "aphid" as the type of insect that is occurring as a judgment result. The information on these operation results is transmitted from the operation terminal 2 to the server 1 (step S108).

The server 1 stores the transmitted operation result in the correspondence information DB 108 together with the operation command content, the operator ID, and the time required for the correspondence (step S109). By this process, as shown in FIG. 12, the operation information 1204, the operation result 1205, the operator ID 1206, and the correspondence time 1207 of the correspondence information DB 108 are updated.

In this embodiment, since post-confirmation is required as the operation result, the information control unit 102 of the server 1 responds to the operation terminal 2 of the operator at the site (farm) with an exception event. It sends information to inform you of what happened and what you did, and to ask you to confirm after the fact. In response to this, the operator at the site examines the strain on which the insect was attached and confirms whether there are any insects remaining around, and examines the leaves pruned by the mobile robot 3b to confirm whether the insect is actually an aphid. To do. If the remote operator makes a mistake, the field operator takes another action or registers the correct result in the server 1.

In this embodiment, the mobile robot 3b may work on the farm during the night time. The mobile robot 3b in this embodiment is not intended to be used in a fully robotized or automated farm. Therefore, the farm worker and the mobile robot 3b work in the same place, but it is more efficient to work in different time zones than to work in a mixed manner with the farm worker and the robot. For this reason, the work by the mobile robot 3b may be performed at night. Exceptional events need to be dealt with at night, but remote operators can respond to them wherever they are connected via a network. For example, the task of dealing with exceptional events can be assigned to remote workers in foreign countries with a large time difference. Allocation is also preferred.

Also, if you let the farming robot take a picture during the night time, use the lighting mounted on the robot body or robot arm. By working at night when the lighting conditions are constant, rather than during the day when the lighting conditions change, it is possible to take images under the same conditions. Since remote workers can work under the same conditions, it can be expected that the skill proficiency speed will improve.

<Example 3>
A case where the mobile robot 3b is a robot that manages products in a store will be briefly described. The mobile robot 3b autonomously performs operations such as product replenishment, inspection, confirmation of missing items, and confirmation of price tags. Therefore, the mobile robot 3b is equipped with a manipulator and a camera.

Illustrate the exception event assumed in this embodiment. First, there is a case where object recognition cannot be performed with high accuracy from an image. Indeterminate package products with high reflectance, aluminum-laminated packages, and transparent package products with high transmittance may not be able to perform object recognition with high accuracy or at all. Therefore, in such a case, it is assumed that the remote worker is requested to recognize the product.

Next, as an example of an exceptional event, there is a case where the handling position when gripping an object cannot be determined. For products with irregular shapes or imbalanced weight balance, the mobile robot 3b may not be able to determine which part of the product should be grasped and grasped. In such a case, it is assumed that a remote operator is requested to teach which part of the product should be grasped for stable gripping.

Another example of an exceptional event is the case where a product is dropped during handling. If the robot 3b drops a product during the automatic operation of product delivery and the product rolls in an unexpected place, it is difficult to find out where the product went. Therefore, it is assumed that the remote operator is requested to search for and collect the dropped products. In addition, even if the product can be collected, the product may be damaged or dented, and the value as a product for sale may be impaired. Since it is difficult for the robot 3b to automatically make this determination, it is assumed that a remote operator is requested to perform the confirmation work.

Another example of an exceptional event is the case where the price tag cannot be read with high accuracy. At a store, a specific product may be sold at a special price on a limited sale date, and the special price and the regular price tag may be exchanged. When the price tag is replaced, it is assumed that an image is taken by the camera mounted on the robot 3b and the price tag is confirmed by image recognition. Depending on the state of the price tag, the robot may not be able to read the price tag with high accuracy. In such a case, it is assumed that a remote worker is requested to read the price tag. The remote operator may instruct not only to read the price tag from the sent image but also to retake the price tag from a direction in which the reflection of the lighting does not occur.

<Example 4>
A case where the mobile robot 3b is a cleaning robot will be briefly described. The mobile robot 3b has a moving mechanism, a floor cleaning function, a camera, and a wireless communication function, and performs self-propelling and floor cleaning based on map data of a work place. The work place is a large place such as a building, a station, or an airport. The mobile robot 3b also has a manipulator that pushes a button on the elevator. The mobile robot 3b is also equipped with a microphone and a speaker, and has a function of interacting with a human.

Such a mobile robot 3b self-propells at the place specified by the map data and cleans the floor. When an obstacle that does not exist in the map data is detected, the mobile robot 3b needs to automatically avoid it.

Illustrate the exception event assumed in this embodiment. First, an obstacle that is not in the map data is detected, but a safe workaround is unknown. In such a case, it is assumed that the image is sent to the operation terminal and the remote operator is requested to perform safe avoidance.

Another example of an exceptional event is when a dirt on the floor is detected, but it cannot be determined whether the robot 3b can clean it. In such a case, it is assumed that an image is sent to the operation terminal 2 and a remote operator is requested to input an instruction to the operation terminal to have the robot clean or dispatch a cleaner. For example, the robot 3b transmits the size, type, and the like of the dust falling on the floor surface to the operation terminal. It is also important whether or not the subsequent cleaning is automatically continued when the liquid is detected by the sensor. This is because in the case of liquid stains, the stains may be diffused by the cleaning brush or wheels of the robot 3b. Therefore, it may be desirable to have the robot 3b continue cleaning other parts without cleaning the liquid, and have the cleaning staff clean the liquid.

Another example of an exceptional event is the case where it cannot be moved. In the case of cleaning a building, it is assumed that the robot 3b moves between floors. The robot 3b uses an elevator to move between floors, and presses an operation button using a manipulator. However, the operation buttons of the elevator are diverse, and in some cases, the display on the floor may be faint and difficult to read. When the robot cannot automatically determine which operation button should be pressed in this way, it is assumed that the place (floor) to be moved and the image are sent to the operation terminal and the remote operator is requested to determine.

Another example of an exceptional event is when you should communicate with humans. There may be a person around the robot 3b, and when a person is complaining to the robot, it is preferable that the person (operator) speaks directly. Even when the robot 3b is automated to have a conversation with a person, it may be determined that an exception event has occurred when the robot 3b determines that the communication is not proceeding smoothly. In such a case, it is assumed that the voice received by the robot is transmitted to the operation terminal and the remote operator is requested to respond. The sound emitted by a person to the robot 3b is output from the speaker of the operation terminal via the microphone of the robot, and the sound emitted by the remote operator is output from the speaker of the robot via the microphone of the operation terminal. As a result, a conversation between the remote operator and the person in the field can be realized, and for example, a complaint can be dealt with.

<Example 5>
In the above embodiment, it is assumed that the remote operator handles the exception event that occurs in the mobile robot 3b, but the target robot may be the fixed robot 3a. An example of such a fixed robot 3a is a aquaculture cage management robot.

The cage management robot 3a manages aquaculture by sensing values such as water temperature, air temperature, and amount of solar radiation. One of the tasks that makes it difficult for the robot 3a to realize automatic processing is determining the amount of feed. It is necessary to feed an appropriate amount according to the condition of the fish on that day, and it is not enough to simply feed a large amount due to the problems of environmental pollution and cost.

The living cage management robot 3a of this embodiment is equipped with a feeding device, a camera, a wireless communication device, and various sensors (water temperature, air temperature, amount of solar radiation), and transmits data acquired by the sensors by wireless communication. In this embodiment, since the determination of the feeding amount requires a judgment by the operator (human), the feeding timing is always regarded as an exception event. Therefore, at the feeding timing, the image taken by the cage management robot 3a and the data acquired by the sensor are transmitted to the operation terminal, and the remote operator is requested to control the feeding device. The remote operator can control the feeding device while observing the image and adjust the amount of feeding while observing the state of the fish on the surface of the water, that is, the degree of biting into the food.

It should be noted that it is also effective when controlling the bait feeding device mounted on the ship equipped with the automatic driving device instead of the cage management robot 3a.

(Other)
Each of the above embodiments is merely an example of the present invention. The present invention is not limited to the above-mentioned specific form, and various modifications can be made within the scope of its technical idea.

(Appendix 1)
An operation terminal connection unit (104) that manages the connection status of the operation terminal (2) owned by the operator, and
The status reception unit (101) that receives status information from the robots (3a, 3b) and
When it is determined that an exception event has occurred in the robot based on the state information, the information control unit (102) selects an operator for dealing with the exception event from the available operators. )When,
A transmission unit (103) that transmits a response request for an exception event of the robot to the operation terminal (2) of the selected operator together with the state information of the robot.
A task distribution device (1).

(Appendix 2)
The method performed by the computer
A step (S102) for managing the connection status of the operation terminal (2) held by the operator, and
The step (S103) of receiving the status information from the robots (3a, 3b) and
When it is determined that an exception event has occurred in the robot based on the state information, an operator for dealing with the exception event is selected from the available operators (S104). ,
A step (S105) of transmitting a response request to the exception event of the robot to the operation terminal (2) of the selected operator together with the state information of the robot.
Including methods.

1: Task distribution server 2: Operation terminal 3a: Fixed robot 3b: Mobile robot 101: Status reception unit 102: Information control unit 103: Transmission unit 104: Operation terminal connection unit 105: Reception unit 106: Output unit 107 Operator information DB 108: Correspondence information DB

Claims

An operation terminal connection unit that manages the connection status of the operation terminal owned by the operator,
A status reception unit that receives status information from the robot,
When it is determined that an exception event has occurred in the robot based on the state information, an information control unit that selects an operator for dealing with the exception event from the available operators.
A transmission unit that transmits a response request to the exception event of the robot to the operation terminal of the selected operator together with the state information of the robot.
A task distribution device equipped with.
The operation terminal connection unit establishes an online state of a plurality of the operation terminals and monitors the online state.
The information control unit selects an operator associated with the operation terminal that is online as an operator for dealing with the exception event.
The task distribution device according to claim 1.
It also has an operator information storage unit that stores the skills of the operator.
The information control unit selects an operator having a skill that matches at least one of the robot type and the exception event.
The task distribution device according to claim 1.
A receiving unit that receives an operation command for the robot from the operation terminal,
An output unit that outputs the operation command to the robot and
A corresponding information storage unit that stores the state of the robot and the operation content from the operator in association with each other.
The task distribution device according to claim 3, further comprising.
The receiving unit receives the response result for the exception event of the robot from the operating terminal, and receives the response result.
The corresponding information storage unit stores the response result in association with the exception event of the robot and the operator.
The task distribution device according to claim 4.
The information control unit evaluates the skill of the operator with reference to the corresponding information storage unit, and updates the operator information storage unit.
The task distribution device according to claim 4.
The operator information storage unit stores the usage cost of the operator.
The information control unit refers to the corresponding information storage unit and determines a reward to be paid according to the correspondence history of the operator.
The task distribution device according to claim 4.
The information control unit classifies the exception events of the robot, and selects an operator according to the type of the exception event as an operator for dealing with the exception event.
The task distribution device according to claim 1.
In response to a request from the operator, the transmitter transmits state information of the robot to an operation terminal of an administrator at a site where the robot is installed.
The task distribution device according to claim 1.
The state information includes an image taken by a camera provided in the robot and an analysis result of sensor information obtained from a sensor provided in the robot.
The task distribution device according to claim 1.
With a robot
An operation terminal for remotely controlling the robot and
The task distribution device according to claim 1 and
Task delivery system with.
Upon receiving a request for coping with an exception event of the robot, the operation terminal is configured to display the state information and a graphical user interface for operating the robot on a display unit.
The task distribution system according to claim 11.
The robot is a mobile robot that has a mobile device, a sensor, and a camera and can move autonomously.
The task distribution system according to claim 11.
The method performed by the computer
Steps to manage the connection status of the operation terminal owned by the operator,
Steps to receive status information from the robot,
When it is determined that an exception event has occurred in the robot based on the state information, a step of selecting an operator for dealing with the exception event from the available operators, and
A step of transmitting a response request to the exception event of the robot to the operation terminal of the selected operator together with the state information of the robot, and
Including methods.
A program for making a computer function as each means of the task distribution device according to claim 1.
A program for causing a computer to execute each step of the method according to claim 14.