JP2022126658A - Apparatus, system, method, and program for cultivation management of crops - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a task distribution device, a task distribution system, a method, and a program for allocating a task to an appropriate operator to deal with an exceptional event occurring in a robot.

SOLUTION: In a task distribution system, a task distribution server 1 holds, as information, a type of a robot 3 or a coping skill for each type of exception event for each operator, and selects an operator for handling the exception event, and distributes the task of handling the exception event to the selected operator when the exception event requiring determination by the operator occurs in the robot 3.

SELECTED DRAWING: Figure 1

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a task distribution technique for allocating a task to an appropriate operator to deal with an exceptional event that has occurred in a robot.

With recent advances in information technology, robots have become able to determine their surroundings and perform autonomous processing. However, robots are not automated so that they can handle all situations, so when an exceptional situation that the robot cannot handle occurs, humans have to deal with this exceptional event.

Since the robot can be operated remotely, it is conceivable to ask any operator at a remote location to handle the exceptional event by remote operation. Japanese Patent Laid-Open Nos. 2003-100001 and 2003-200021 are examples of techniques for responding to such exceptional events by remote control.

Patent Literature 1 discloses that a server publishes a task desired to be remotely operated, and determines a remote operator to request the task in response to a bid from a remote operator. However, this approach suffers from a lack of responsiveness, as the task cannot be executed without a bid from the remote operator.

The autonomous mobile robot disclosed in Patent Document 2 is configured to output a signal prompting remote control when an obstacle is detected and avoid the obstacle by remote control. This autonomous mobile robot is configured to store operation information when avoiding obstacles in the past, and to be able to refer to it when it is necessary to avoid obstacles again. In Patent Document 2, it is assumed that the autonomous mobile robot and the remote operator are associated in advance. If a robot and a remote operator correspond one-to-one, there is a problem that a large number of remote operators are required. In addition, when one remote operator is associated with a plurality of robots, there is a problem that it is not possible to cope with events requiring remote operation occurring simultaneously in a plurality of robots.

WO2008/140011 JP 2013-206237 A

SUMMARY OF THE INVENTION An object of the present invention is to enable a robot to quickly deal with an exceptional event that has occurred.

In order to achieve the above objects, the present invention employs the following configurations.

Specifically, a first aspect of the present invention includes an operation terminal connection unit that manages the connection state of an operation terminal possessed by an operator, a state reception unit that receives state information from a robot, and the above-mentioned operation based on the state information. an information control unit that selects an operator from among available operators to deal with the exceptional event when it is determined that an exceptional event has occurred in the robot; and a transmission unit configured to transmit a handling request to an exception event of the robot together with state information of the robot to an operation terminal.

In the present disclosure, a robot includes any device capable of autonomously performing some action. Typically, a robot is a device that includes sensors and actuators and is configured to autonomously control the actuators based on information obtained from the sensors. The robot may be a mobile robot that can move autonomously, or a stationary robot that cannot move autonomously.

The state information of the robot may be at least information with which the current state of the robot can be grasped. Examples of robot status information include sensor information obtained from sensors provided in the robot and analysis results obtained by the robot based on the sensor information. It should be noted that it is not necessary to transmit all the state information of the robot to the task distribution device, and at least state information that can be used to determine that an exceptional event has occurred in the robot should be transmitted. Also, receiving state information from the robot includes both receiving state information from the robot and receiving it via another device.

An exceptional event is an event that the robot cannot automatically process or that is determined to be inappropriate. Alternatively, an exceptional event can be regarded as an event that requires human operation or confirmation. Exceptional events may be included in the state information or may be determined by the task distributor from the state information.

The management of the connection status of the operation terminals performed by the operation terminal connection unit may include establishing the online status of the plurality of operation terminals and monitoring the online status. Note that the online state includes a state in which the operating terminal can immediately respond to a notification from the task distribution device.

When it is determined that an exception has occurred in the robot, the information control unit selects an operator to deal with the exception. Whether or not an exception event has occurred can be determined from the content of the state information. However, in an embodiment in which the task distribution device is notified of the state information only when an exception event occurs, the information control unit receives the state information and does not refer to the content of the state information. It can be determined that The information control unit may select, for example, an operator associated with an operating terminal that is online as an operator to handle the received exception event. The information control unit may further select an operator who satisfies the condition that the exception event of another robot is not being dealt with. However, this is not the case when there is no operator who can handle it.

Since this aspect has the configuration as described above, it is possible to select an operator who can quickly respond to an exceptional event. In other words, when an exceptional event occurs in the robot, it is possible to quickly respond to this exceptional event.

The task distribution device according to this aspect may further include an operator information storage unit that stores operator skills. Also, the information control unit may select an operator who has a skill that matches at least one of the type of robot and the exceptional state.

According to such a configuration, it is possible to select an appropriate operator for the exceptional event that has occurred.

The task distribution device according to this aspect includes a receiving unit for receiving an operation command for the robot from the operation terminal, an output unit for outputting the operation command to the robot, a state of the robot and information from the operator. It may further include a correspondence storage unit that stores operation contents in association with each other.

In this aspect, the receiving means receives a result of coping with the exceptional event of the robot from the operating terminal, and the correspondence storage unit stores the result of coping in association with the exceptional event of the robot and the operator. may The content of the countermeasure result is not particularly limited, and for example, it may indicate whether the exceptional event has been resolved or not, or it may request confirmation on site by another operator. .

In this aspect, the information control section may refer to the corresponding information storage section to evaluate the skill of the operator and update the operator information storage section.

In this aspect, the operator information storage unit stores the operator's usage fee, and the information control unit refers to the correspondence storage unit to refer to the operator's correspondence history, and the reward to be paid according to the operator's correspondence history. may be determined.

In this aspect, the information control unit may classify the exceptional event of the robot and select an operator according to the type of the exceptional event as an operator to deal with the exceptional event. For example, a corresponding operator or operator group may be determined in advance for each type of exception event. In this way, the same types of exceptions will be handled by specific operators, and these operators will become more proficient with those types of exceptions.

In this aspect, the transmission unit may transmit the state information of the robot to an operation terminal of an administrator at the site where the robot is installed, in response to a request from the operator. In this way, when the operator determines that processing at the robot work site is necessary, the on-site manager can confirm.

In this aspect, the state information may include an image captured by a camera mounted on the robot and an analysis result of sensor information obtained from a sensor provided on the robot. By referring to this information, the operator can easily deal with exception events.

A second aspect of the present invention is a task distribution system comprising a robot, an operation terminal for remotely controlling the robot, and the task distribution device described above.

In this aspect, the operation terminal is configured to display the status information and a graphical user interface (GUI) for operating the robot on the display unit when receiving a request for handling an exceptional event of the robot. may

In this aspect, the robot may be a stationary robot or a mobile robot. The mobile robot may be a mobile robot that has a mobile device, a sensor, and a camera and is capable of autonomous movement. It may be a robot having an arm-type robot mounted on a mobile robot.

The present invention can be regarded as a task distribution device or task distribution system having at least part of the above configurations and functions. The present invention also provides a task delivery method including at least a part of the above processing, a program for causing a computer to execute the method, or a computer-readable recording medium in which such a program is non-temporarily recorded. can also be taken as Each of the above configurations and processes can be combined to form the present invention as long as there is no technical contradiction.

According to the present invention, it is possible to quickly deal with an exceptional event that has occurred in a robot.

FIG. 1 is a diagram illustrating a configuration example of a task distribution system according to an embodiment. FIG. 2 is a flowchart showing processing performed by the task distribution server in the embodiment. FIG. 3 is a flowchart showing processing performed by the task distribution server in the embodiment. FIG. 4 is a diagram illustrating processing performed by an information control unit in the embodiment. FIG. 5 is a diagram illustrating an example of an operator information DB according to the first embodiment; FIG. 6 is a diagram illustrating an example of a correspondence information DB according to the first embodiment; FIG. 7 is a diagram illustrating an example of an image captured by the robot according to the first embodiment; FIG. 8 is a diagram illustrating an example of an image captured by the robot according to the first embodiment; FIG. 9 is a diagram for explaining the results of dealing with exception events in the first embodiment. FIG. 10 is a diagram illustrating a configuration example of a robot according to the second embodiment; 11A and 11B are diagrams showing examples of images captured by the robot according to the second embodiment. FIG. 12 is a diagram illustrating an example of an operation information DB according to the second embodiment; FIG. 13 is a diagram illustrating an example of a correspondence information DB according to the second embodiment; FIG. 14 is a diagram showing a graphical user interface for remote control of the robot displayed on the operator terminal according to the second embodiment.

<Application example>
FIG. 1 is a block diagram showing a configuration example of a task distribution system to which the present invention is applied. The task distribution system includes a task distribution server (task distribution device) 1, an operation terminal 2, and a robot such as a stationary robot 3a or a mobile robot 3b. Hereinafter, when there is no need to distinguish between the fixed robot 3a and the mobile robot 3b, they may simply be referred to as the robot 3. The task distribution server 1 and the operation terminal 2 are connected via a wide area network such as the Internet, and the operator is located in a remote location away from the work position of the robot 3 .

In this system, a task distribution server 1 (hereinafter simply referred to as server 1) selects an operator to handle the exceptional event when an exceptional event occurs in the robot 3, and the selected operator Distributes tasks to deal with exception events to The server 1 stores, as information, the type of the robot 3 or the handling skill for each type of exception event for each operator, and identifies the robot 3 in which the exception event has occurred or the operator who has the skill to handle the exception event. may be configured to select.

The operator uses the operation terminal 2 to operate the robot 3 to deal with the exception event. Operation instructions for the robot 3 are transmitted to the robot 3 via the server 1, and the server 1 can store a history of operation details.

[Constitution]
The server 1 is a computer including a CPU 11, a memory 12, a storage 13, a network interface (not shown), and the like. The CPU 11 loads the program stored in the storage 13 into the memory 12 and executes it, so that the server 1 provides the following functions. That is, the server 1 has, as its functional units, a state reception unit 101, an information control unit 102, a transmission unit 103, an operation terminal connection unit 104, a reception unit 105, an output unit 106, an operator information DB 107, and a corresponding information DB . DB means database.

The state reception unit 101 receives state information from the fixed robot 3a and the mobile robot 3b. The state reception unit 101 receives state information from the fixed robot 3a via the robot controller 5a, and receives state information from the mobile robot 3b via the wireless communication device 6. FIG. The state information may be transmitted from the robot 3 to the server 1 only when an exception occurs in the robot 3, or may be transmitted from the robot 3 to the server 1 regardless of whether an exception has occurred.

The information control unit 102 selects an operator to deal with an exception when an exception occurs in the robot 3, saves operation details (response history) from the operation terminal, evaluates the skill of the operator, and provides information to the operator. Performs processing such as remuneration determination. Details are described below.

The transmission unit 103 transmits a request for handling the exceptional event of the robot 3 and status information of the robot 3 to the operation terminal 2 .

The operation terminal connection unit 104 establishes connection with the operation terminal 2 and monitors the online state of the operation terminal 2 . Here, the online state means a state in which the operation terminal 2 can immediately communicate with the server 1 and transmit an operation command to the robot 3 . Also, the online state means not only the state of connection between the server 1 and the operation terminal 2, but also the state that the operator can immediately respond to requests. Therefore, for example, the operation terminal connection unit 104 may make an inquiry when information representing an input by the operator is not received from the operation terminal 2 and it is unclear whether the operator can respond immediately. Note that the online state may be established by a push notification to the operation terminal 2 . In addition, the operation terminal connection unit 104 determines not only the online state of the operation terminal 2 but also whether the operation terminal 2 is dealing with an exception state of any of the robots 3 , in other words, whether the exception event of the robot 3 is being newly handled. It also manages whether or not it is possible to deal with Information about the state of the operation terminal 2 is stored in the memory 12, for example.

The receiving unit 105 receives information that the operator has input to the operation terminal 2 and transmitted to the server 1 . The information transmitted from the operation terminal 2 includes an operation command for the robot 3 and the result of coping with the exceptional event. An operation command to the robot 3 is transmitted to the robot 3 via the output unit 106 and stored in the correspondence information DB 108 as a correspondence history. In addition, the result of dealing with the exceptional event is also stored in the correspondence information DB as a correspondence history.

The operator information DB 107 stores information about operators. The response information DB 108 stores the content of the response history for exceptional events. These databases are described below with reference to FIGS.

The output unit 106 transmits an operation command for the robot 3 input from the operation terminal 2 to the robot 3 . An operation command for the fixed type robot 3a is transmitted to the robot controller 5a, and the robot controller 5a controls the robot 3a according to the contents of this operation command. An operation command to the mobile robot 3b is transmitted to the robot controller 5b in the mobile robot 3b via the wireless communication device 6, and the robot controller 5b controls the mobile robot 3b according to the contents of this operation command. Hereinafter, when there is no need to distinguish between the robot controllers 5a and 5b, the robot controllers 5a and 5b may simply be referred to as the controller 5.

The operation terminal 2 is any computer capable of communicating with the server 1 and remotely controlling the fixed robot 3a and the mobile robot 3b. A desktop PC, a laptop PC, a tablet terminal, and a smartphone terminal can be exemplified as the operation terminal 2 . The operation terminal 2 may be a dedicated teaching pendant for remotely controlling the robot. The operation terminal 2 provides the user with different types of operation GUIs according to the type of the robot 3 . This GUI may be generated by the operation terminal 2 itself, or may be generated by the server 1
or other device. The operator can grasp the state information of the robot 3 via the GUI of the operation terminal 2 . This state information includes sensor information obtained from the sensors of the robot 3 , analysis results of the sensor information, and images captured by the camera of the robot 3 . The operation terminal 2 receives the information on the state requiring processing from the server 1, displays it on the screen, and prompts the operator to operate. The operator can input/transmit an operation command for the robot 3 and an operation result for the robot 3 via the GUI of the operation terminal 2 . Input information is transmitted from the operation terminal 2 to the receiving unit 105 of the server 1 .

The robot 3 is a device capable of autonomously performing some action. The fixed robot 3a has an actuator and a sensor, and executes processing by controlling the actuator according to information obtained from the sensor. The contents of control of the fixed robot 3a are determined by the robot controller 5. FIG. Examples of the fixed robot 3a include, but are not limited to, FA robots, NC machine tools, molding machines, press machines, automatic feeding devices for reared animals and plants, and automatic watering and fertilizing devices for cultivated plants. .

The mobile robot 3b is equipped with a moving device and a robot controller 5b in addition to actuators and sensors, and is capable of autonomous movement, and is a device that performs some action at the destination. Examples of the mobile robot 3b include, but are not limited to, a transport robot, a cleaning robot, a serving robot, a security robot, and a robot combined with an arm-type robot.

Both the fixed robot 3a and the mobile robot 3b preferably have a camera (imaging means) as a sensor. The camera may be any camera, such as a visible light camera, an infrared camera, a three-dimensional camera, or the like.

The robot controller 5 controls the robot 3 according to a predefined program based on information obtained from sensors of the robot 3 . Note that the robot controller 5 may be configured to be able to dynamically change the content of control through online learning.

The robot controller 5 is configured to be able to determine whether an event has occurred in which the robot 3 cannot automatically process or is inappropriate to automatically process. Such events are referred to as exceptional events in this disclosure. As an example of an exceptional event, there is a mismatch between the operation instruction sent from the robot controller 5 to the robot 3 and the operation result returned from the robot 3, and the robot 3 does not operate according to the operation instruction. An example of an exceptional event is the occurrence of an event that requires human confirmation. Such events include an abnormal temperature rise of the robot 3, generation of abnormal noise, and abnormality of the operation target.

[Database]
The operator information DB 107 stores information about operators. For example, as shown in FIG. 5, the operator information DB 107 stores operator ID 501, operating terminal information 502, skill information 503, hourly wage 504, and work time history 505. FIG. These are merely examples, and the operator information DB 107 does not have to store all of these information, and may store other information.

The operator ID 501 is an identifier for uniquely identifying the operator within the system. The operator uses this operator ID and password (not shown) to log in to the system and maintain the online state. The operating terminal information 502 is information about the operating terminal 2 that is kept online, and includes information necessary for identifying and connecting the operating terminal 2 from the server 1, such as an IP address and a MAC address. The operating terminal information 502 includes information indicating the characteristics of the operating terminal 2 itself, such as a PC or smartphone terminal, and information on a special user interface (UI) connected to the operating terminal, and this information is used for task matching. be done. The special UI is, for example, a sensor worn on the hand to operate the finger portion of the end effector.

The skill information 503 includes skill possession information 503a and skill evaluation 503b. Skill 1, skill 2, . . . skill N shown in FIG. If the operator has a skill capable of operating skill N, YES is registered in the skill possession information 503a, otherwise NO is registered. Also, when the operator has the skill N, the evaluation thereof is registered as the skill evaluation 503b. The value of the skill evaluation 503b is determined based on the number of experiences with skill N, speed of response, evaluation by an evaluator (administrator) in the system, and the like. For example, values from 1 to 10 can be set, with 10 being the highest rank. Further, the server 1 may automatically update the operator's skill evaluation based on the operator's response to the exceptional event.

The hourly wage 504 is a reward per hour set for the skill or the like of the operator. From the employer's point of view, hourly rate 504 represents the operator's utilization costs. Here, the hourly wage unit is used as a point so that the compensation can be calculated according to the minimum wage (or other wage level) in the operator's area of residence. The work time history 505 records the total time worked by the operator. The work time history 505 can aggregate the hours worked by the operator for each day, week, month, etc., and can be used for remuneration calculation.

The response information DB 108 stores the content of the response history for exceptional events. In the corresponding information DB 108, the status of the robot 3 and the details of the operation performed by the operator are stored in association with each exception event. For example, as shown in FIG. 6, the correspondence information DB 108 stores an exception occurrence time 601, an exception robot ID 602, state information 603, operation information 604, an operation result 605, an operator ID 606, and a correspondence time 607. be done. These are merely examples, and the operator information DB 107 does not have to store all of these information, and may store other information.

The exceptional event occurrence time 601 represents the time when an exceptional event occurred in the robot 3 and the server 1 (state receiving unit 101) received state information of the exceptional event from the robot controller 5 . The robot ID 602 is an identifier for uniquely specifying within the system the robot 3 in which the exception event has occurred.

The state information 603 is information such as exception states and statuses sent from the robot 3 . The operation information 604 is information that instructs the operator to give an operation to the robot 3 in order to restore the exceptional state to the normal state based on the sent state information. The state information 603 and the operation information 604 will be described in detail when describing the embodiments.

The operation result 605 is information for recording the result of operating the robot 3 according to the operation information. The operator ID 606 is an identifier for uniquely identifying the operator who handled the exception event. Response time 607 represents the time required to restore the exceptional state to the normal state. It represents the time until the operator determines that the problem cannot be resolved by remote control alone if the normal state cannot be restored.

[process]
2 and 3 are flowcharts showing the processing performed by the server 1 in the task distribution system.

In step S101, when the receiving unit 105 of the server 1 receives a connection request from the operation terminal 2, in step S102, the operation terminal connection unit 104 manages the connection of the operation terminal 2, that is, establishes the connection and monitors the online state. . The operating terminal connection unit 104 also manages whether or not the operating terminal 2 is dealing with an exceptional event of any of the robots 3 . The connection management by the operation terminal connection unit 104 continues after step S102 until the operation terminal 2 disconnects.

In step S103, the status reception unit 101 receives status information from the robot 3 (fixed robot 3a or mobile robot 3b). Here, it is assumed that the status information is transmitted from the robot 3 to the server 1 only when an exception event occurs. Therefore, when the status information is received, the robot 3 has an exception event. can be determined automatically. However, the robot 3 may be configured to transmit state information to the server 1 regardless of the presence or absence of an exception event. If it is determined that an exceptional event has occurred, the processing from step S104 onward may be executed.

In step S104, the information control unit 102 performs matching between the exceptional event handling task and the operator, that is, selects the operator to handle the exceptional event. Specifically, the information control unit 102 selects an operator to be assigned a coping task from available operators in consideration of the skill of the operator and the exceptional event. As shown in FIG. 4, the information control unit 102 receives the type and state (failure state) of the robot 3 included in the state information received by the state reception unit 101, and the information included in the operator information DB 107. A suitable operator is extracted by comparison. As a condition for an operator to be matched (assigned) with a task, first, the operator must be online. Another condition is that the level of coping skill for the type of robot in which the exceptional event has occurred or the coping skill for the type of exceptional event is at a required level. Also, it is preferable to adopt the condition that no other task is being executed. However, if all the operators who can handle the exception event are executing other tasks, the operators executing other tasks may be matched. Furthermore, operators to be matched may be determined in consideration of rewards for the operators. For example, the information control unit 102 may determine, as a matching operator, an operator who satisfies the condition that the skill is equal to or higher than the required level and has the highest score determined based on the skill and reward. Here, it is assumed that the higher the required skill, the higher the score, and the lower the reward, the higher the score.

The information control unit 102 may classify robot exceptions into several types and assign each exception to a specific operator according to the type of the exception. Here, the specific operator corresponding to the exception event is typically a plurality of operators, and the information control unit 102 preferably assigns the exception event to the operators within the same group according to the exception type. However, this does not preclude the assignment of certain types of exceptions to one particular operator.

In step S105, the information control unit 102 identifies the operating terminal 2 of the operator selected in step S104, and sends the identified operating terminal 2 a handling request for exception handling and status information of the robot 3. Send. As shown in FIG. 4, the information control unit 102 refers to the operator information DB 107 and acquires the operating terminal information of the selected operator. The operating terminal information is information for communicating with the operating terminal 2, and is, for example, an IP address or a MAC address. The operating terminal information may include terminal type information. The information control unit 102 passes the state information received from the robot 3 via the state reception unit 101 and the operating terminal information to the transmission unit 103 as transmission information. The transmitting unit 103 transmits the state information and the task execution request to the operating terminal 2 specified by the operating terminal information.

When the operation terminal 2 receives the exceptional event handling task and the status information of the robot 3, the operation terminal 2 displays a GUI for operating the robot 3 on the display unit. With this GUI, confirmation of status information of the robot 3, input of operation information (operation command) to the robot 3, input of coping results, and the like can be performed. The operation information for the robot 3 may be a command for remotely operating the robot 3 by real-time control, or may be a command for performing a series of procedures. Operation information for the robot 3 input to the operation terminal 2 is transmitted from the operation terminal 2 to the server 1 and received by the receiving unit 105 .

In step S<b>106 , the reception unit 105 receives information from the operation terminal 2 . The reception information received by the reception unit 105 includes operating terminal information and operation information.

In step S<b>107 , the information control unit 102 extracts operation information from the received information, and transmits the operation information from the output unit 106 to the target robot 3 . As a result, the robot 3 can be made to perform the action instructed by the operator.

Although not shown in the flowcharts of FIGS. 2 and 3, when the state of the robot 3 is updated by an operation command or the like from the operation terminal 2, the state information of the robot 3 is updated directly from the robot 3 or via the server 1. It may be transmitted to the terminal 2. By doing so, the operator can grasp the latest state of the robot 3 .

After the robot 3 operates according to the operation command from the operator, the operator evaluates the result of the operation, inputs it to the operation terminal 2 and transmits it to the server 1 . Examples of the operation result include whether or not the handling of the exceptional event has been completed, and whether or not the manager needs to check after the fact on site. It should be noted that the completion of handling includes the case where the normal state is restored and the case where it is determined that no handling is necessary (there is no abnormality). It should be noted that handling of the exceptional event may be completed when the state reception unit 101 receives state information indicating that the robot 3 has returned to normal operation in accordance with an operation command from the operator. In this case, the task distribution server 1 transmits the operation result to the operation terminal 2 .

In step S<b>108 , the reception unit 105 receives the operation result from the operation terminal 2 .

In step S<b>109 , the information control unit 102 associates the state information received from the robot 3 with the operation information and operation result received from the operation terminal 2 and stores them in the correspondence information DB 108 . As a result, it is possible to record what kind of operation the operator performed when the robot 3 was in what kind of state. Further, the information control unit 102 may store in the corresponding information DB 108 the time required from the start to the completion of the exceptional event handling task by the operator. As a result, it becomes possible to grasp the working hours of the operator, which can be used to calculate the operator's remuneration and evaluate the operator's skill. Information such as work hours and rewards may be stored in the operator information DB 107 .

In step S110, the information control unit 102 determines whether or not handling of the exceptional event has been completed. This determination may be made based on the operation result information transmitted from the operation terminal 2 . If the countermeasure is not completed (S110-NO), the process returns to step S105, and the robot 3 is operated based on the instruction from the operator. Here, the same operator as the previous operator is requested to take action, but the process may return to step S104 to reselect the operator to take action. In this case, if necessary, the previous operator may be excluded from the candidates and another operator may be selected. If the countermeasure has been completed (S110-YES), the process proceeds to step S111.

In step S<b>111 , the information control unit 102 evaluates the skill of the operator based on the response history and results stored in the response information DB 108 and updates the skill stored in the operator information DB 107 . The evaluation of the skill takes into consideration the result of handling the exceptional event, the content of the handling, the time required for handling, and the like.

In step S112, the information control unit 102 determines the amount of remuneration to be paid to the operator based on the operator's hourly remuneration stored in the operator information DB 107 and the time required to deal with the exception event. do.

Here, skill evaluation (S111) and remuneration calculation (S112) are performed each time an exceptional event is dealt with, but these processes may be performed separately at appropriate timing.

Further, in the above example, it was explained that the operator would perform some operation on the robot 3 when an exception occurred in the robot 3, but in reality the operator does not have to perform any operation on the robot 3. good too. For example, even if an exceptional event occurs in the robot 3, the operator may be able to check the state of the robot 3 and confirm that no abnormality has actually occurred. In such a case, the operator may transmit from the operation terminal 2 to the server 1 that no action is required without transmitting an operation command to the robot 3 .

[advantageous effect]
According to this embodiment, an exception occurring in the robot 3 can be dealt with by a remote operator. Here, the task distribution server 1 manages the connection with the operation terminal 2, selects an operator who can respond immediately, and distributes the task, so it is possible to quickly handle exceptions. In addition, considering the operator's skill, an appropriate operator is selected according to the type of robot and the type of exception event, and tasks are distributed, so appropriate exception handling is possible.

There is no need for the operator to stand by at the robot's working position in a factory or the like, and the operator can operate the robots at multiple locations from a remote location. Therefore, if this system is introduced, there is no need to have personnel who can handle robot exceptions waiting at the company's own factory, so costs can be reduced.

In recent years, problems such as labor shortages and employment shortages in rural areas due to the declining birthrate and aging population have been pointed out. contributes to solving these problems. With this system, the operator does not need to work at a specific location and can work from a remote location. Therefore, even people who have difficulty commuting long distances can be provided with an environment that is easy to work in, so even elderly people and people living in rural areas can work.

In addition, foreign labor and immigration to Japan are being discussed in order to solve Japan's labor shortage. The operator in this system does not need to be located in Japan, and may be located in a foreign country. If the interface of the operation terminal is written in a foreign language, it will be possible for foreigners to operate it from abroad, and it will also have the effect of making it possible to import labor without immigration of foreigners.

<Example 1>
Here, the task delivery system to which the present invention is applied will be described more specifically, taking as an example the case where the mobile robot 3b is a delivery robot that automatically delivers parcels and the like.

The delivery robot in this embodiment is a mobile robot that employs wheels as its locomotion mechanism. Here, an opposed two-wheel type delivery robot will be described as an example, but the movement mechanism of the delivery robot may be a wheel movement mechanism other than the opposed two-wheel type, or a multi-legged or endless track type movement mechanism. The delivery robot has a storage section for stably storing the delivery items. In addition, the delivery robot has various sensors such as a position information acquisition sensor, camera, LIDAR, millimeter wave radar, ultrasonic sensor, acceleration sensor, etc. Based on the sensor information obtained from these sensors, the built-in robot controller 5 controls the locomotion and other movements of the delivery robot.

The delivery robot sends status information to the server 1 when an exception occurs. Also, it can operate based on an operation command transmitted from the operation terminal 2 via the server 1 .

Here, it is assumed that an abnormal state (exceptional event) occurs in which the left wheel of the delivery robot falls into a hole in the road and stops in a derailed state during unmanned, that is, automatic delivery. Trying to deal with delivery robots without knowing what's going on around them can lead to more dangerous situations. For example, if the hole is larger than the delivery robot, there is a danger that the delivery robot will fall into the hole. Therefore, when the delivery robot determines that the body is not kept horizontal from the information of the acceleration sensor, it determines that an exceptional event has occurred, and notifies the server 1 of the occurrence of the exceptional event together with the state information. It should be noted that the point at which it is determined that an exceptional event has occurred varies according to the level of control technology of the mobile robot and the required safety level, and the above criteria are only examples.

When the status reception unit 101 receives status information from the mobile robot 3b (S103), this information is stored in the correspondence information DB . As shown in FIG. 6, the exceptional event occurrence time 601, the robot ID 602 where the exceptional event occurred, and the state information 603 are stored in the database.

The state information 603 here includes position information, the tilt of the vehicle body, the state of the wheels, cargo, and an image captured by the camera mounted on the robot 3b. The position information is position information obtained from a GPS device (position information acquisition sensor) mounted on the mobile robot 3b. The vehicle body tilt is information representing the front, rear, left, and right tilts of the vehicle body obtained from a tilt sensor (acceleration sensor) mounted on the mobile robot 3b. For example, an abnormality can be determined when an inclination exceeding 4 degrees (absolute value) occurs. In this example, the left-right tilt is -5 degrees (minus indicates left-down), and the absolute value of the tilt angle exceeds the threshold, so it is determined that the robot 3b is in an abnormal state.

The wheel state is stationary. This is because the robot 3b detected an abnormality and stopped driving the wheels. Cargo information is information entered at the origin of delivery. Based on the cargo information, the operator can determine what kind of operation and how much. For example, if the cargo is a parcel as shown in this example, it is considered that there is no problem even if it is handled somewhat roughly. On the other hand, it can be seen that if the cargo is food and drink, it needs to be handled carefully. Also, if the cargo is food and drink and the vehicle body tilt is a large value such as 45 degrees, it can be considered that it is better to contact the delivery source rather than recover from the abnormal state. This is because, even if the food and drink rolls over in the container of the mobile robot 3b and is delivered to the delivery destination, it is considered to be in a meaningless state.

The image from the robot camera is an image from a camera attached to the body of the mobile robot 3b (see FIG. 7). The camera is attached so as to photograph at least forward in the traveling direction. A plurality of robot cameras may be attached around the vehicle body to capture around-view images of the surroundings of the vehicle body. Alternatively, it may be an omnidirectional image (360-degree image) captured by an omnidirectional camera (360-degree camera).

The state control unit 102 of the server 1 can determine from the state information transmitted from the delivery robot (mobile robot) 3b that an exception has occurred in the delivery robot 3b. (step S104). Since the exceptional event in this example is a problem with the delivery robot 3b, the information control unit 102 handles this exceptional event among the operators who are online and are not processing other tasks. Select an operator who has the skills to

Consider the case where operator 1 and operator 2 shown in FIG. 5 exist as operators who are online and are not processing other tasks. Here, operators 1 and 2 both have the skills of "mobile robot, delivery, trouble shooting". The information control unit 102 selects an operator in consideration of the skill evaluation and hourly wage of each operator. For example, in this example, the operator 2 with a higher skill 2 evaluation and a lower hourly wage is selected.

The information control unit 102 transmits a request to deal with the abnormal event occurring in the delivery robot 3b and the status information of the delivery robot 3b to the operation terminal of the selected operator (step S105). The operation terminal 2 that has received these information displays a screen including status information of the delivery robot 3b and a GUI for operating the delivery robot 3b. The operation terminal 2 is provided with a display unit for displaying such information and a user interface for inputting operation instructions to the robot 3b. Operation information may be input via a keyboard or by touching the screen. An example of another user interface is an interface for inputting simple instructions (programs, command strings) for moving the mobile robot 3b.

The operation terminal 2 displays, for example, the status information of the delivery robot (information indicated by the status information 603 in FIG. 6) and the image (FIG. 7) captured by the camera mounted on the delivery robot 3b. Based on this information, the operator first determines whether or not there is an abnormal state that requires some kind of countermeasure, and if the abnormal state exists, whether or not it is possible to respond by remote control. In some cases, it determines what kind of operation should be performed.

In this example, the operator can read the following from the image shown in FIG. 7 and other sensor information.
- Since the entire landscape slopes downward to the right, the mobile robot 3b is tilted to the left.
・From the angle of inclination, it is highly likely that the left wheel has derailed.
・Since there is no side ditch along the guardrail, there is a high possibility that the train has derailed into an unexpected hole.
・Because there are guardrails on the left and right, it is a relatively safe place to operate and move the mobile robot 3b.
- When changing the direction of the mobile robot 3b to make it go backwards or go straight, it is necessary to operate so as not to contact the left and right guardrails.
・Because the guardrail on the left side is closer, it is safe to back up to the right rear within 1 meter in order to recover from the derailed state.

State information and images sent when an exceptional event occurs are infinitely different, and while the above matters can be grasped instantly by humans, it is difficult for computers (such as AI). Based on the image from the mobile robot 3b and other state information, the operator can conclude that the left wheel of the delivery robot has fallen into the hole. Furthermore, the operator can determine from the surrounding circumstances that there is no problem even if the mobile robot 3b retreats about 1 meter to the right rear in the traveling direction, and that it is necessary to move the mobile robot 3b for recovery. Such situation determination and operation of the mobile robot 3b cannot be executed unless the mobile robot 3b is programmed. It is not realistic to perform programming that can handle all situations in advance, and it is still appropriate to let humans (operators) make judgments on how to appropriately handle and operate in a wide variety of situations.

Based on such judgment, the operator determines the operation command to be given to the mobile robot 3b as follows. First, as the first step, it is instructed to rotate the left wheel at -0.05 m/s while keeping the right wheel locked. Since the left wheel is in a hole, the operator should
Lock the right wheel outside the hole and instruct the left wheel to slowly move backward around it. By this action, the left wheel is caught on the edge of the hole where it fell, and the left wheel can be pulled up slowly.

As a second step, when the vehicle body tilt becomes 0±2 degrees (normal state), an instruction is given to change the right wheel speed to -0.05 m/s. When the tilt of the vehicle clearly falls within the normal range, it can be determined that the left wheel has been lifted out of the hole. After that determination, the right wheels are retracted at the same speed as the left wheels to move the mobile robot 3b away from the hole. If the tilt of the vehicle body does not return to the normal state even after the operation of the first step is continued for a certain period of time, an instruction may be given to terminate the process abnormally.

As a third step, the operator stops the wheel movement after 10 seconds, takes an image with the front camera of the robot 3b, and instructs the execution of self-diagnosis to check whether the normal state has been restored. By continuing the retreating motion of the second step for 10 seconds, the robot moves to a position 0.5 meters away from the position where it fell into the hole. This value should be set according to the surrounding conditions, and it is appropriate to allow a human (operator) to determine the value. It is considered that the camera mounted on the robot 3b can photograph the hole by moving 0.5 meters away from the hole.

Self-diagnosis is executed as the operation of the third step for reconfirmation. Not only is the tilt of the vehicle body within the normal range, but the mobile robot 3b self-diagnoses whether or not there is a functional problem. Even if the mobile robot 3b determines that it is normal as a result of the self-diagnosis, the operator may determine that the abnormal state continues. In such a case, priority may be given to the operator's determination.

The operator transmits the operation command consisting of the first to third steps from the operation terminal 2 to the mobile robot 3b via the server 1 (steps S106, 107). In this embodiment, it is assumed that the operator collectively transmits the instructions of the first to third steps to the mobile robot 3b. However, the instruction may be transmitted in units of each step, or may be transmitted in finer units. Moreover, when the status information from the mobile robot is transmitted in real time, the operator may remotely operate the mobile robot in real time.

The mobile robot 3b operates in accordance with the above operation commands and transmits the information obtained as a result to the operation terminal 2 via the server 1 or directly. As a result of the first to third steps, the mobile robot 3b recovers from the derailed state and retreats from the hole, and can acquire an image of the hole as shown in FIG.

In this example, the operator can read the following from the image shown in FIG.
- The mobile robot 3b is in a horizontal state from the entire landscape.
・As instructed by the operation information, the aircraft moved to the right rear from the stop position.
・A hole was recognizable from the image, and the left wheel was derailed there, causing an abnormality.
・From the above, the mobile robot 3b has returned to the normal state.

The server 1 can record that there is a hole on the road together with GPS position information. As a result, the position 901 of the hole on the road can be grasped as shown in FIG. will be able to more reliably avoid falling.

In addition to the image shown in FIG. 8, information indicating that the vehicle body tilt is 0 degrees and the result of the self-diagnosis is normal is transmitted to and displayed on the operation terminal 2 .

The operator refers to the state information transmitted from the mobile robot 3b, inputs the result of the operation, and transmits it to the server 1 (step S108). In this example, an operation result indicating that the normal state has been restored is input. The server 1 stores the transmitted operation result in the response information DB 108 together with the details of the operation performed, the operator ID, and the time required for response (step S109). By correlating and storing the state information, the operation information, and the result information in this manner, it is possible to accumulate know-how about what kind of operation can restore the normal state in what kind of situation. The information of the operation result may be manually input to the operation terminal 2 by the operator, or may be determined by the mobile robot 3b.

Note that there may be a case where trial and error are repeated in order to return to the normal state without returning to the normal state by one operation. In this case, the operator inputs the operation command again. The server 1 may continue to request the same operator to take action, or may select a new operator and have another operator take action against the abnormal event.

<Example 2>
Next, a task distribution system to which the present invention is applied will be described in detail, taking as an example a case where the mobile robot 3b is an agricultural robot that manages the cultivation of tomatoes.

The mobile robot 3b in this embodiment can at least move autonomously to patrol fields and greenhouses, and has a function of taking and transferring photographs. As shown in FIG. 10, the mobile robot 3b has a main body (running section) including a moving mechanism 1011, a robot controller 1012, a position information acquiring device 1013, a camera 1014, and a power supply (not shown), an agricultural chemical storage section 1021, and an agricultural chemical spraying unit. An arm 1020 including a section 1022 , an illumination section 1023 and a camera 1024 is provided. Here, one arm 1020 is provided with the functions of image capturing and agricultural chemical spraying, but different arms may be provided with the respective functions. The arm 1020 can also harvest fruit and push and pick leaves. Moreover, the mobile robot 3b may be provided with other centers such as a distance sensor and an acceleration sensor other than those illustrated.

The moving mechanism 1011 is a moving mechanism such as a wheel type or an endless track type. Robot controller 1012 is similar to that described above. The position information acquisition device 1013 is a device for specifying the position of the robot in the field or inside the greenhouse. The position information acquisition device 1013 may be a GPS device, or may be a device that reads barcodes or RFIDs attached to fields, vinyl greenhouses, or stocks. Camera 1014 captures a relatively wide range around the robot.

The mobile robot 3b has an agricultural chemical spraying function with an agricultural chemical storage unit 1021 and an agricultural chemical spraying unit 1022, and can automatically spray agricultural chemicals when the robot determines that it is necessary to spray agricultural chemicals. Pesticides are sprayed, for example, from an arm. If there are pests or diseases, the type of pests and disease conditions can be determined from the image of the arm camera 1024, and the type of agricultural chemicals effective against the specified pests and disease conditions can be specified. For example, it can be determined that an aqueous solution of clothianidin is effective when there is Nijuu no hoshita beetle. The pesticide storage section 1021 may be loaded with a plurality of types of pesticides so as to deal with a plurality of pests and disease states. Although it is known from the image obtained from the camera 1024 that there is a possibility that an abnormality has occurred, the robot cannot determine whether an abnormality has actually occurred and how to deal with the abnormality. It can be difficult to determine automatically. For example, when leaves are discolored, there are various causes such as pests, fungi, and viruses, and different pesticides should be used depending on the cause.

The robotic arm 1020 can move back and forth, left and right, and can be inserted into dense foliage. A camera 1024 and an illumination unit 1023 are mounted at the tip of the robot arm 1020. By rotating the arm 1020, the direction of photographing and illumination by the camera 1024 and the illumination unit 1023 can be changed. For example, by directing the camera 1024 and the lighting unit 1023 upward, it is possible to photograph the underside of the leaf (see FIG. 11A).

FIG. 11A is a diagram showing an example of an image 1101 captured by the body camera 1014 of the mobile robot 3b. An image 1101 shows the arm 1020 in an extended state. As shown in the figure, a discolored portion 1102 is recognized on a portion of the tomato leaf surface. It is assumed that the robot controller 1012 determines that the cause of the discoloration cannot be identified from only the image of the discolored portion 1102 of the leaf surface, and that it is necessary to operate the arm 1020 to take an image of the underside of the leaf. With the arm camera 1024 and the lighting unit 1023 facing upward, the robot controller 1012 extends the arm to the position of the underside of the leaf and performs photographing with the camera 1024 . The series of operations described above is merely an example, and the robot 3b may be controlled so as to inspect all the plants even if no discolored portions are observed on the leaves. Also, if there is no abnormality, the image may not be sent to the operation terminal.

FIG. 11B is a diagram showing an image 1103 of the underside of a leaf captured by the arm camera 1024. FIG. By performing image processing on the image 1103, it can be recognized that there is also a discolored portion 1104 on the underside of the leaf, and that there are a plurality of small spots 1105 of the same size around it. This image processing may be performed by the robot controller 1012 or another processing device incorporated in the robot 3b, or may be performed by an external device to which the image is transmitted from the robot 3b. From the result of this image processing, the robot controller 1012 has a high possibility that an abnormality has occurred, but it is unclear how to deal with it. That is, it is determined that an exception event has occurred. Accordingly, the robot controller 1012 sends state information to the server 1 .

When the status reception unit 101 of the server 1 receives status information from the mobile robot 3b (S103), this information is stored in the correspondence information DB . FIG. 12 shows an example of the correspondence information DB 108 in this embodiment. Since the table format itself is the same as that of the first embodiment (FIG. 6), detailed description will be omitted. At this point, the date and time 1201 of the exceptional event, the robot ID 1202 where the exceptional event occurred, and the state information 1203 are stored in the database 108 .

In this example, the state information 1203 includes position information, leaf surface state, leaf underside state, leaf area, leaf area, fruit diameter, fruit color, and image. The location information includes latitude and longitude information, greenhouse number, and stock number acquired by the location information acquisition device 1013 . In addition, the position information includes height information, and this height information is the position captured by the camera 1014 mounted on the robot body 1010, or the arm camera 1024 if the image is captured by the arm camera 1024. Or the height of arm 1020 . The height of arm camera 1024 or arm 1020 can be obtained from an encoder provided on arm 1020 . The height information represents the height of the target fruit or leaf.

Leaf surface information indicates the result of determining the state of the leaf surface by image recognition technology. Examples of leaf surface information include "normal", "discolored", "worm-eaten", and "insects". Such image recognition can be realized by a machine learning method such as deep learning, and since it is publicly known, detailed description thereof will be omitted.

The information on the underside of the leaf indicates the result of determining the state of the underside of the leaf by the image recognition technology. In this example, the state of the leaf surface is "discolored", so the mobile robot 3b captures an image of the leaf underside and obtains the image recognition result. The information on the underside of the leaf is the result of this recognition.
A recognition result of "there are insects" is obtained. Note that when the state of the leaf surface is "normal", the photographing of the underside of the leaf and the determination of the state may be omitted. Further, even if the state of the leaf surface is "normal", the underside of the leaf may be abnormal.

The area of the leaf, the diameter of the fruit, and the color of the fruit are obtained by calculation based on the image captured by the main body camera 1014 and the distance to the target object (leaf or fruit) by the distance sensor. Since these pieces of information can be calculated using known techniques, detailed description is omitted.

The images from camera 1014 mounted on the robot body and from camera 1024 mounted on the robot arm are shown in FIGS. 11A and 11B, respectively.

The state control unit 102 of the server 1 can determine from the state information transmitted from the mobile robot 3b that an exceptional event has occurred, and therefore performs task matching processing for selecting an operator to deal with this exceptional event. (step S104). Since the exception event in this example relates to disease diagnosis of tomatoes, the information control unit 102 selects an operator who is online and is not processing other tasks to handle this exception event. Select operators with skills.

FIG. 13 shows an example of the operator information DB 107 in this embodiment. Since the content of the operator information DB 107 is basically the same as that of the first embodiment, redundant description will be omitted. Since this embodiment is applied to agricultural work, each operator's skill related to agricultural work is stored. In the example shown in FIG. 10, "inspection of tomato leaf diseases and pests" is set as one of the skill information. This skill implies knowledge of diseases and pests that occur in tomatoes and knowledge of appropriate treatments for the diseases and pests that are found. Appropriate treatment is the judicious ability to select or monitor effective pesticides against diseases and pests. As another skill, "Tomato Harvest" is also set. This skill is a skill for viewing images sent from the robot, determining whether or not harvesting is permitted, and harvesting by remotely manipulating the arm.

Consider the case where operator 1 and operator 2 shown in FIG. 13 exist as operators who are online and are not processing other tasks. Since the exceptional event in this example relates to the inspection of tomato leaf diseases and pests, only the operator 1 can handle it. Therefore, the information control unit of the server 1 matches the exception event of this time with the operator 1 (step S104).

The information control unit 102 transmits a request for dealing with the exceptional event and the state information of the mobile robot 3b to the operation terminal of the selected operator (step S105). The operation terminal 2 that has received these information displays a screen containing the status information of the delivery robot and a GUI for operating the delivery robot.

FIG. 14 shows an example of a GUI 1400 displayed on the operation terminal 2. As shown in FIG. The GUI 1400 includes an image 1401 of the body camera 1014, an image 1402 of the arm camera 1024, and state information 1405 other than the image. The GUI 1400 also includes an input unit 1403 for operating (moving) the robot body 1010, an input unit 1404 for operating (moving) the arm 1020, an input unit 1406 for inputting a series of operation commands, and an operation result. and a send button 1408 for sending the input contents to the server 1 .

The operator uses the GUI 1400 to input an operation command for the mobile robot 3 b and transmit it to the server 1 . For example, it is conceivable to input the following two-step operation command. The first step is leaf pruning. From the images 1401 and 1402 and the state information 1405, the operator confirms that the leaves are infested with insects. instruct to store in It is assumed that the robot has the closed box, and that the robot can automatically perform the pruning process of the target leaf by the robot arm and the storage process in the closed box.

The second step is the application of agricultural chemicals. The operator determines that the insect is likely to be an aphid, enters "YES" in the field of pesticide application, and then selects and enters an agricultural chemical acephate that is effective against aphids. Here, it is assumed that the robot is equipped with a plurality of types of pesticides, and that the robot can automatically spray the pesticides.

The operation command input by the operator is transmitted to the mobile robot 3b via the server 1 (steps S106, S107). The mobile robot 3b operates according to the above-described operation commands, and transmits information obtained as a result to the operation terminal 2 via the server 1 or directly. Based on the image and status information transmitted from the mobile robot 3b, the operator can determine whether the leaf pruning process and the agricultural chemical spraying process have been completed normally. If the process has been completed normally, enter "process completed" as the operation result. In addition, since the judgment by the operator is based on the image, there is a possibility that the judgment is wrong, and there is a possibility that insects and diseases have spread to places that cannot be detected by the robot's image alone. The operator enters "required" for post-confirmation as a message to the person who can check directly on the farm. Further, the operator may input "aphids" as the type of insect that has occurred as a determination result. Information on these operation results is transmitted from the operation terminal 2 to the server 1 (step S108).

The server 1 stores the transmitted operation result in the correspondence information DB 108 together with the contents of the operation command, the operator ID, and the time required for the correspondence (step S109). By this processing, the operation information 1204, the operation result 1205, the operator ID 1206, and the response time 1207 of the response information DB 108 are updated as shown in FIG.

In this embodiment, since post-confirmation is required as an operation result, the information control unit 102 of the server 1 notifies the operation terminal 2 of the operator at the site (farm) how to deal with the exceptional event. Send information to inform what kind of event occurred, what action was taken, and notify you to perform post-event confirmation. In response to this, the on-site operator examines the plant on which the insect was attached to see if there are any insects remaining around it, and examines the leaves pruned by the mobile robot 3b to see if the insect is actually an aphid. do. If the judgment by the remote operator is wrong, the on-site operator takes another action or registers the correct result in the server 1 .

In this embodiment, the mobile robot 3b may work in the farm at night time. The mobile robot 3b in this embodiment is not intended for use on a fully robotized or automated farm. Therefore, the farm worker and the mobile robot 3b work in the same place, but it is more efficient to divide the work hours than to work together with the farm worker and the robot. For this reason, the work by the mobile robot 3b may be performed at night. Exceptional events need to be handled at night, but remote operators can handle them from anywhere as long as they are connected to the network. Assignment is also preferred.

In addition, when the farming robot is caused to take a photograph at night, the lighting mounted on the robot body or the robot arm is used. By performing the work at night when the lighting conditions are constant rather than during the daytime when the lighting conditions change, it is possible to take images under the same conditions. Since remote workers can work under the same conditions, it is expected that the skill acquisition speed will improve.

<Example 3>
A case in which the mobile robot 3b is a robot that manages merchandise in a store will be briefly described. The mobile robot 3b autonomously performs operations such as product replenishment, product inspection, confirmation of missing products, price tag confirmation, and the like. Therefore, the mobile robot 3b is equipped with a manipulator and a camera.

Exceptional events assumed in this embodiment are exemplified. First, there is a case where object recognition from an image cannot be performed with high accuracy. Object recognition may not be possible with high accuracy or not at all for products with irregular packages with high reflectance, packages laminated with aluminum, and products with transparent packages with high transmittance. Therefore, in such a case, it is assumed that the remote worker is requested to recognize the product.

Next, as an example of an exceptional event, there is a case where a handling position cannot be determined when grasping an object. For products with irregular shapes or uneven weight balance, the mobile robot 3b may not be able to determine which part of the product should be gripped. In such a case, it is conceivable to ask the remote worker to teach which part of the product to grasp in order to stably grasp it.

Another example of an exceptional event is a case where a product is dropped during handling. When the robot 3b drops an item during the automatic work of picking out items, if the item rolls to an unexpected place, it is difficult to find where the item has gone. Therefore, it is assumed that the remote operator is requested to search for and recover the dropped product. Moreover, even if the product can be collected, there are cases where the product is damaged or dented and loses its value as an item for sale. Since it is difficult for the robot 3b to automatically make this determination, it is conceivable to ask a remote worker to perform the confirmation work.

Yet another example of an exception is when a price tag cannot be read with high accuracy. A store may sell a specific product at a special price on a limited sale date, and may replace the special price and regular price tags. When the price tag is replaced, it is assumed that the camera mounted on the robot 3b takes an image and confirms the price tag by image recognition. Depending on the state of the price tag, the robot may not be able to read the price tag with high accuracy. In such a case, it is assumed that the remote operator is requested to read the price tag. In addition to reading the price tag from the sent image, the remote operator may instruct to re-photograph the price tag from a direction that does not cause reflection of the illumination.

<Example 4>
A brief description will be given of the case where the mobile robot 3b is a cleaning robot. The mobile robot 3b has a moving mechanism, a floor cleaning function, a camera, and a wireless communication function, and performs self-propulsion and floor cleaning based on the map data of the work place. The work place is a large place such as a building, a station, an airport, or the like. The mobile robot 3b also has a manipulator to press the elevator buttons. The mobile robot 3b also has a microphone and a speaker, and has a function of interacting with humans.

Such a mobile robot 3b runs on its own in a location designated by map data and performs floor cleaning work. When detecting an obstacle that does not exist in the map data, the mobile robot 3b must automatically avoid it.

Exceptional events assumed in this embodiment are exemplified. First, there is a case where an obstacle that is not in the map data is detected, but a safe avoidance method is unknown. In such a case, it is conceivable to send an image to the operation terminal and request the remote operator to safely avoid.

Another example of an exceptional event is a case where dirt on the floor is detected, but it cannot be determined whether the robot 3b can clean it. In such a case, it is conceivable to send an image to the operation terminal 2 and request the remote operator to input an instruction to the operation terminal to instruct the robot to clean or dispatch a cleaner. For example, the robot 3b transmits the size, type, etc. of the garbage that has fallen on the floor to the operation terminal. It is also important whether or not cleaning should be automatically continued after detection of liquid by the sensor. This is because, in the case of liquid dirt, the dirt may be spread by the cleaning brushes or wheels of the robot 3b. Therefore, it may be desirable to have the robot 3b continue cleaning other locations without cleaning the liquid, and have the cleaner clean the liquid.

Another example of an exception event is when movement is not possible. In the case of cleaning a building, it is assumed that the robot 3b moves between floors. The robot 3b uses an elevator to move between floors and presses an operation button using a manipulator. However, there are many different types of elevator operation buttons, and in some cases the floor display may be faint and difficult to read. In this way, if the robot cannot automatically determine which operation button to press, it is conceivable to send the location (floor) to which the robot wishes to move and an image to the operation terminal and request the remote operator to make the determination.

Yet another example of an exceptional event is when communication with humans is to be made. There is a possibility that there are people around the robot 3b, and when the people complain about the robot, it is preferable for the human (operator) to directly talk to them. Even if the robot 3b is automated to have a conversation with a person, it may be determined that an exception event has occurred when the robot 3b determines that the communication is not progressing smoothly. In such a case, it is assumed that the voice received by the robot is transmitted to the operation terminal to request the remote operator to respond. The voice uttered by the person to the robot 3b is output from the speaker of the operating terminal via the microphone of the robot, and the voice uttered by the remote operator is output from the speaker of the robot via the microphone of the operating terminal. As a result, a conversation can be realized between the remote operator and the person on site, and for example, complaints can be dealt with.

<Example 5>
In the above embodiment, it is assumed that the remote operator handles the exceptional event that occurs in the mobile robot 3b, but the target robot may be the fixed robot 3a. An example of such a fixed robot 3a is a management robot for aquaculture cages.

The cage management robot 3a senses values such as water temperature, air temperature, amount of solar radiation, and manages aquaculture. As a work that is difficult to realize automatic processing by the robot 3a, there is determination of the feed amount. As for the amount of feed, it is necessary to give an appropriate amount according to the condition of the fish on that day, and from the problems of environmental pollution and cost, simply feeding a large amount is not enough.

The cage management robot 3a of this embodiment is equipped with a bait feeding device, a camera, a wireless communication device, and various sensors (water temperature, air temperature, amount of solar radiation), and transmits data acquired by the sensors by wireless communication. In the present embodiment, it is assumed that an exceptional event always occurs in the timing of feeding, since determination of the feeding amount requires judgment by the operator (human). Therefore, at the feeding timing, the image captured by the cage management robot 3a and the data acquired by the sensor are transmitted to the operation terminal, and the remote operator is requested to control the feed feeding device. A remote operator can control the feeding device while watching the image, and can adjust the feeding amount while observing the state of the fish on the surface of the water, that is, how the fish bites into the feeding.

It is also effective to control a bait feeding device mounted on a ship equipped with an automatic operation device instead of the cage management robot 3a.

(others)
Each embodiment described above is merely an illustration of the present invention. The present invention is not limited to the specific forms described above, and various modifications are possible within the technical scope of the present invention.

(Appendix 1)
an operation terminal connection unit (104) that manages the connection state of the operation terminal (2) owned by the operator;
a state reception unit (101) for receiving state information from the robots (3a, 3b);
An information control unit (102) for selecting an operator to deal with the exceptional event from available operators when it is determined that an exceptional event has occurred in the robot based on the state information. )When,
a transmission unit (103) for transmitting a request to handle the exceptional event of the robot together with state information of the robot to the operation terminal (2) of the selected operator;
A task delivery device (1) comprising:

(Appendix 2)
A computer-implemented method comprising:
a step of managing the connection state of the operation terminal (2) owned by the operator (S102);
a step of receiving state information from the robots (3a, 3b) (S103);
a step of selecting an operator to deal with the exceptional event from available operators when it is determined that an exceptional event has occurred in the robot based on the state information (S104); ,
a step (S105) of transmitting a request for coping with the exceptional event of the robot together with state information of the robot to the operation terminal (2) of the selected operator;
A method, including

1: task distribution server 2: operation terminal 3a: fixed robot 3b: mobile robot 101: status reception unit 102: information control unit 103: transmission unit 104: operation terminal connection unit 105: reception unit 106: output unit 107 operator information DB 108: correspondence information DB

TECHNICAL FIELD The present invention relates to a technique for allocating to an operator an event that has occurred in a robot and should be judged by the operator regarding the cultivation of crops .

SUMMARY OF THE INVENTION An object of the present invention is to make it possible to cope with an event that has occurred in a robot and requires an operator's judgment regarding the cultivation of crops .

Advantageous Effects of Invention According to the present invention, it is possible to deal with an event that has occurred in a robot and requires an operator's judgment regarding cultivation management of crops .

Claims (16)

an operation terminal connection unit that manages the connection state of an operation terminal owned by an operator;
a state reception unit that receives state information from the robot;
an information control unit that, when it is determined that an exception has occurred in the robot based on the state information, selects an operator from available operators to handle the exception;
a transmitting unit configured to transmit a request for handling an exceptional event of the robot together with status information of the robot to the operation terminal of the selected operator;
A task delivery device. The operation terminal connection unit establishes an online state of the plurality of operation terminals and monitors the online state,
The information control unit selects an operator associated with the operating terminal that is online as an operator to handle the exception event.
The task distribution device according to claim 1. further having an operator information storage unit that stores the operator's skill,
The information control unit selects an operator who has a skill that matches at least one of the robot type and the exception event.
The task distribution device according to claim 1. a receiving unit that receives an operation command for the robot from the operation terminal;
an output unit that outputs the operation command to the robot;
a correspondence information storage unit that associates and stores the state of the robot and the operation content of the operator;
4. The task distribution device according to claim 3, further comprising: The receiving unit receives, from the operating terminal, a result of coping with the exceptional event of the robot,
The correspondence information storage unit stores the countermeasure result in association with the exceptional event of the robot and the operator.
The task distribution device according to claim 4. The information control unit refers to the correspondence information storage unit, evaluates the operator's skill, and updates the operator information storage unit.
The task distribution device according to claim 4. The operator information storage unit stores usage fees of the operator,
The information control unit refers to the correspondence information storage unit and determines a reward to be paid according to the operator's correspondence history.
The task distribution device according to claim 4. The information control unit classifies the exceptional event of the robot, and selects an operator according to the type of the exceptional event as an operator to deal with the exceptional event.
The task distribution device according to claim 1. The transmission unit, in response to a request from the operator, transmits the state information of the robot to an operation terminal of an administrator at the site where the robot is installed.
The task distribution device according to claim 1. The state information includes an image captured by a camera provided on the robot and an analysis result of sensor information obtained from a sensor provided on the robot.
The task distribution device according to claim 1. robot and
a control terminal for remotely controlling the robot;
a task delivery device according to claim 1;
task delivery system. The operation terminal is configured to display the state information and a graphical user interface for operating the robot on a display unit upon receiving a request for handling an exceptional event of the robot.
The task distribution system according to claim 11. The robot is a mobile robot that has a mobile device, a sensor, and a camera and is capable of autonomous movement.
The task distribution system according to claim 11. A computer-implemented method comprising:
a step of managing a connection state of an operation terminal owned by an operator;
receiving status information from the robot;
when it is determined that an exception has occurred in the robot based on the state information, a step of selecting an operator to deal with the exception from available operators;
a step of transmitting a request to handle the exceptional event of the robot together with state information of the robot to the operation terminal of the selected operator;
A method, including A program for causing a computer to function as each means of the task distribution device according to claim 1. A program for causing a computer to perform the steps of the method of claim 14.

Images