JP2022132166A - Robot supporting system - Google Patents

Abstract

To provide a robot supporting system which can support robots in a short time while reducing the burden on an operator, especially without requiring skill of the operator when supporting a large number of various robots.SOLUTION: A robot supporting system 1 includes robots 51, 52, 53 connectable over a network 3, a support management server 2, user interface (UI) terminals 41, 42, and a storage server 6. The robot provides operation status information including task information on each task, or the operation status information and robot information, to the support management server. The support management server selects work assistance information on the basis of the operation status information and/or the robot information provided from the robot, using a work assistance information database, and provides the selected work assistance information to the user interface terminals (UI). The UI terminal provides a user with the work assistance information by means of output means.SELECTED DRAWING: Figure 2

Description

The present invention relates to a robot support system, in particular, comprising a robot that can operate autonomously, a support management server, and a user interface terminal, the support management server being connectable to the robot and the user interface terminal via a network, The present invention relates to a robot assistance system that allows an operator to assist the operation of the robot from a user interface terminal when the robot needs assistance.

In recent years, automation by robots that work autonomously in various fields is progressing, but when a trouble occurs in a robot, the robot itself is provided with a means to automatically solve the trouble, and an operator etc. supports the robot. are being carried out. Also, depending on the type of robot, there is a limit to automation, and an operator may need to support the robot.

For example, there has been known an automatic return when the current position of the guided vehicle cannot be identified based on the principle of triangulation (Patent Document 1). Patent Document 1 discloses a transport system that uses a laser scanner, a reflector, and a transport vehicle to identify the current position based on the principle of triangulation, and uses the identified current position to determine a target work position along a designated route. It is disclosed to run the carrier to. If the current position of the transport vehicle cannot be identified based on the principle of triangulation during this travel, the transport system will compare the information from the measurement sensors that measure the environment within the facility with the environment map within the facility. estimating the current position of the transport vehicle using the estimated current position, and using the estimated current position to drive the transport vehicle to the return location, so that the current position of the transport vehicle can be identified again based on the principle of triangulation; A transport system is disclosed in which the specified current position is used to cause the transport vehicle to run again.

Furthermore, conventionally, there has been known a guiding means for driving the moving means based on a user's operation to guide the own machine (Patent Document 2). Patent document 2 discloses an object information acquisition unit that acquires detection information of an object existing around the self-machine, and mask detection information in a predetermined area among the detection information of the object acquired by the object information acquisition unit. local map creating means for creating a local map around the aircraft from the masking means, detection information masked by the masking means, moving means for moving the own aircraft, and local map creating means self-position estimation means for estimating self-position based on the local map and movement amount of the movement means; guidance means for driving the movement means based on user's operation to guide the self-machine; and during guidance by the guidance means, An autonomous mobile device is disclosed that includes environment map creation means for creating an environment map of a movement area from the local map and the self-position estimated by the self-position estimation means.

Further, as a system requiring robot support by an operator, a remote support system for remotely supporting a surgical support robot has been known (Patent Document 3). In Patent Document 3, at least one piece of operation information relating to the operation of a surgical assistance robot is received by a server device for performing remote assistance work of the surgical assistance robot, and when a predetermined event is detected, the server Disclosed is a remote assistance method for a surgical assistance robot that delivers at least one of voice, image, and text from a device to at least one of the surgical assistance robot and a terminal device.

Japanese Patent No. 6687313 Japanese Patent No. 6136435 Japanese Patent Application Laid-Open No. 2020-60880

In Patent Document 1, as a method of recovering the self-position from the loss of the self-position, the final position of the guided vehicle specified by the position specifying unit is set as the initial position, and the closest recovery location (self-position The vehicle is driven toward a preset location where the Also, when the transport vehicle moves between the last position and the position estimation guidance, the current position of the transport vehicle is calculated from the information from sensors such as encoders that detect the rotation of the running wheels and the last position, Using this as the initial position, the vehicle is caused to travel to the return location. However, the position and orientation of the transport vehicle may change unexpectedly due to slippage or vibration of the running wheels, and the actual position and orientation of the transport vehicle may differ from the initial position of the position identification unit. is assumed. In such a case, there is a possibility that the return location cannot be reached because the current position is estimated incorrectly. In addition, since the transport vehicle automatically returns to the return position, if a new article is placed or a temporary obstacle occurs before the return position, the moving body It is not possible to return to running.

Further, Patent Document 2 discloses guidance means for driving a moving means based on the operation of a user (operator) to guide the own aircraft. The user guides the moving body, and no recovery method is disclosed when the self-location is lost. Furthermore, in Japanese Patent Application Laid-Open No. 2002-200001, a predetermined area in which the user is located is masked in order to prevent the user from being erroneously registered as an obstacle when creating an environment map. In this way, the guiding means of Patent Document 2 is assumed to be operated while the user visually recognizes it near the autonomous mobile device, and in order to cope with the error of self-location loss that occurs unexpectedly, Each time an error occurred, it was necessary to go to the vicinity of the autonomous mobile device, which was troublesome.

In Patent Document 3, operation information regarding the operation of the surgical assistance robot is received by a server device for performing remote assistance work for the surgical assistance robot, and when a predetermined event is detected, the surgical assistance robot and the terminal are sent from the server device. Disclosed is a remote support system that performs surgery or the like using a surgical support robot by delivering at least one of voice, image, and text to at least one of devices. When an event occurs in the surgical assistance robot, operation information such as images taken by the surgical assistance robot and images of the operating room are sent to the terminal device via the server device. discloses a remote support system for distributing information corresponding to an event from a terminal device to a surgical support robot via a server device. Advanced skills and knowledge are required.

In recent years, there has been an increase in the number of environments in which multiple robots of different types that perform different tasks operate. For example, in a store, a reception robot that autonomously accepts requirements (orders), an arm robot (manipulator) that grasps and loads the target (product) that is the requirement, and the target (product) along a predetermined route. A system composed of a plurality of robots such as a carrier robot that autonomously moves in space along a path has been proposed. The reception robot has a voice recognition function and accepts customer requests. The arm robot has an image recognition function and joints, selects objects, grasps the objects, and delivers them to the transport robot. The transport robot scans the surrounding space using an external sensor such as a laser distance sensor, matches the scanning result with a map prepared in advance, and estimates its own current position and posture while reaching the target position. Moving. However, if the reception robot cannot understand the customer's instructions, or if the arm robot cannot determine the object to be grasped, the transfer robot fails to match the scanning result with a map prepared in advance, and the transfer robot fails to operate. When the robot loses its own position and cannot move to the destination, the operator who manages the robot support system needs support. Thus, conventionally, there has been no suitable system for supporting a plurality of different types of robots performing different tasks. In addition, the operator needs knowledge to support all the operations of all robots, and it takes time and money to train such operators. Furthermore, even if you have knowledge about a certain robot, the operation method may be different for robots from different manufacturers, and new functions may be added or the operation method may be changed due to version upgrades of robots. I had to study again and again.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-described problems. To provide a robot support system capable of supporting a robot in a short period of time.

In order to solve the above problems, a robot support system of the present invention includes a robot, a support management server, and a user interface terminal, wherein the robot and the support management server are connected via a network. The support management server and the user interface terminal can be connected via a network, and the robot includes control means capable of autonomously executing a plurality of tasks and the support management server. a support request means capable of executing a support request; and a storage means storing robot information; can be provided to a management server, the support management server includes a work assistance information database in which work assistance information associated with status information and/or robot information is stored; Work assistance information is selected based on the provided operation status information and/or the robot information, and the selected work assistance information can be provided to the user interface terminal, and the user interface terminal instructs or operates the robot. Possible input means and output means are included, and the work assistance information provided from the support management server can be provided to the user by the output means.

Furthermore, in the above-described robot support system, the work assistance information may include work content of the robot.

Further, in the above robot support system, the work assistance information includes a moving image, still image, or sound recording normal operation of the work corresponding to the task information associated with the work assistance information, and the robot associated with the work assistance information. A moving image, still image, or sound that records the normal operation of the robot corresponding to the identification information, and/or a moving image, still image, or sound that indicates the expected situation during normal operation may be included.

Further, in the above-mentioned robot support system, the support management server associates videos recording normal operations of a plurality of tasks of the robot in time series with task information of the robot, classifies them for each task, and classifies them as the work assistance information. It may be possible to provide the user interface terminal with a moving image of a category corresponding to task information stored in a database and included in the operation status information provided from the robot as the work assistance information.

Further, in the robot support system, the robot includes error detection means, and can provide error information detected by the error detection means to the support management server, and the work assistance information database stores the work assistance information. The support management server can search the work assistance information database based on the error information provided from the robot and provide the retrieved work assistance information to the user interface terminal. may be

Further, in the above robot support system, the user interface terminal may be capable of setting whether or not to individually transmit task assistance information for a plurality of tasks of the robot with respect to the support management server.

Further, in the robot support system, the work assistance information database stores the work assistance information in association with language information related to language, and the user interface terminal transmits the language information used by the user to the support management server. The support management server is capable of searching the work assistance information database based on the language information provided from the user interface terminal, and providing the retrieved work assistance information to the user interface terminal. good too.

Further, in the robot support system, the robot information includes at least one of robot identification information that uniquely identifies each robot, robot type information that identifies the type of robot, and operating location information that specifies the operating location of the robot. The support management server includes a robot information database that stores robot type information and/or operating location information about the robot in association with the robot identification information, and the robot identification information provided from the robot The robot type information and/or the operating location information may be searched based on the robot type information and/or the operating location information, and the work assistance information may be selected based on the retrieved robot type information and/or the operating location information.

Further, in the above robot support system, the support request means includes environmental information around the robot, image information around the robot, audio information around the robot, position information of the robot on a map, and At least one of the relative arrangement information of the components of is obtained, and included in the operation status information and provided to the support management server.

Further, in the above robot support system, the robots include a plurality of robots of different types that perform different tasks, each robot is arranged at a plurality of operating locations, and the user interface terminals are installed at a plurality of bases. may be placed.

According to the robot support system of the present invention, when an autonomously operating robot requests operator support, the support management server receives task information and/or robot identification information included in the operation status information provided by the robot. It is possible to search the work assistance information database based on the above, and provide appropriate work assistance information to the user interface terminal, and the operator can easily and quickly provide appropriate support based on the work assistance information. . Further, this work assistance information may include the details of the work performed by the robot, may include moving images or still images recording the normal operation of the robot, or may include error information from the error detection means of the robot. may also contain the language information you want to use. By providing such work assistance information, the operator can quickly assist the robot with less burden, and even an inexperienced operator can assist the robot. Furthermore, it is also possible for a single operator to support a wide variety of different types of robots that perform different tasks.

An example of a flow chart in the robot support system of the present invention A diagram showing an example of hardware that realizes the robot support system of the present invention. Diagram showing each component and data structure of the present invention A diagram for explaining a series of operations in the robot group of the present invention. A diagram for explaining a series of operations in the robot group of the present invention. A diagram for explaining a series of operations in the robot group of the present invention. A diagram for explaining a series of operations in the robot group of the present invention. A diagram for explaining a series of operations in the robot group of the present invention. Diagram showing an example of a robot information database A diagram showing an example of a work assistance information database A diagram showing another example of the work assistance information database A diagram showing another example of the work assistance information database A diagram showing an example of a display screen of a UI terminal A diagram showing an example of a display screen of a UI terminal (A) shows map information indicating the range that the transfer robot transfers, and (B) shows the display screen of the UI terminal in which the transfer robot is moving normally. (A) and (B) are diagrams for explaining a state in which the transport robot has lost its own position; (A) is a diagram explaining support work using image information acquired by the camera of the transport robot, and (B) is a diagram explaining the support work using environmental information acquired by Lidar of the transport robot. (A) shows a state in which the tentative destination has been reached, and (B) shows map information reflecting the sum of surrounding environmental information. display). A diagram showing a display screen of a UI terminal in an abnormal state A diagram showing a display screen of a UI terminal in an abnormal state A diagram showing a display screen of a UI terminal in an abnormal state The figure which shows an example of the notification setting in a UI terminal.

Ideally, robots should be able to automatically handle all tasks and troubles, but it is technically and economically difficult to prepare perfect countermeasures for all tasks and troubles. Therefore, we propose a new realistic idea of designing a robot on the premise that an operator will support some tasks or troubles. For example, in normal times, work is autonomously processed, but when autonomous processing is not possible (for example, when there is an abnormality or when processing requires human judgment), the operator's support is requested, and the operator's support (for example, instructions or operations) Design robots to work. Such a design can be applied to a wide variety of multiple robots located at multiple work locations and performing different tasks. This design concept presupposes the intervention of the operator, and by allowing the operator to assist the robot through the network, multiple robots working at multiple different work locations can perform the same type of work. It is also possible to provide a system that supports not only a plurality of robots that perform various tasks, but also a wide variety of robots that perform various tasks. In addition, user interface terminals (UI terminals) are arranged, and operators carry out support work (including the operator's location when the operator moves while carrying a UI terminal). A mode in which an operator works, a mode in which a plurality of sites where a large number of operators work is provided, and a mode in which individual operators work at various sites may be used. In that case, dedicated operators may be assigned to a wide variety of robots or to a plurality of different tasks. The operator's support work volume can be made uniform, and rapid support can be realized. In other words, although the frequency with which assistance is required varies depending on the robot and/or the type of work, if a dedicated operator is assigned, an operator in charge of less frequent assistance can assist with more frequent assistance. As a result, it becomes difficult to provide frequent assistance at the appropriate timing. In this regard, if the operator can support a plurality of different tasks of a wide variety of robots, the support tasks can be shared and processed.

In such a robot support system, when an operator performs support work (for example, instructions or operations) using a UI terminal, the present invention provides work support information for assisting the operator's support work according to the robot type and work content. provided to the operator accordingly. Therefore, in the present invention, it becomes easy for the operator to perform support work for a plurality of different work performed by a wide variety of robots.

FIG. 1 is an example of a flow chart from a robot operator support request to handling by operator support in the robot support system of the present invention. The robot of the present invention normally operates autonomously by control means capable of autonomously executing a plurality of tasks. When the robot cannot operate autonomously, the support management server acquires the operation status from the robot, and transmits work assistance information selected according to the operation status to the user interface terminal (UI terminal). Support work is performed based on the work assistance information presented on the UI terminal. In S1 of FIG. 1, when the robot detects a situation in which it cannot move autonomously, it requests the support management server for operator support. Robot identification information that identifies the robot may be provided to the support management server when making the request. In S2, the support management server assigns an operator in charge of processing the support request, and in S3, the support management server acquires operation status information from the robot that has requested support. The operation status information includes at least task information related to each work, and may further include environment information such as error information, surrounding still image information, and moving image information. In S4, the support management server selects work assistance information expected to be required by the operator based on the task status of the robot's operation status information and/or the robot identification information. In S5, the support management server transmits the selected work assistance information to the UI terminal. In S6, the UI terminal presents the work assistance information sent from the support management server to the operator, and in S7, the operator assists the robot based on the work assistance information presented to the UI terminal. In addition, in S1, the robot may provide not only the robot identification information but also the operation status information to the support management server (in this case, S3 is unnecessary). Also, the assignment of the operator in charge in S2 may be performed after acquiring the operation status information or after acquiring the work assistance information.

FIG. 2 is an example of hardware for realizing the robot assistance system 1 of the present invention, and FIG. 3 is an example of a block diagram of the components and data structure necessary for the robot assistance system 1 for realizing the present invention. be. In the robot support system 1 according to the present embodiment, the support management server 2 is connected via the network 3 to a plurality of user interface terminals 41 and 42 and a plurality of different types of robots 51, 52 and 52 that perform different tasks. ing. The support system 1 may include a storage server 6 connectable via the network 3 as required. A plurality of robots 51, 52, and 52 include, for example, a receiving robot 51 that receives instructions, an arm robot 52 that can select and grasp an instructed object, and an autonomous robot that moves to a destination while estimating its own position. It includes an autonomous transfer robot 53 that can operate. The robot support system 1 of this embodiment can be used when the reception robot 51 cannot select an instruction target, when the arm robot 52 cannot determine a picking target, when the transport robot 53 loses its own position and cannot return, and so on. , the UI terminals 41 and 42 are connected via the network 3, and work is carried out with the support of the operator. For example, as support work, the operator instructs the reception robot 51 to select an object and talks with customers, instructs and operates the picking object to the arm robot 52, instructs the transfer robot 53 to a temporary destination, and performs movement operations. conduct. The reception robot 51, the arm robot 52, and the transport robot 53 request support from the support management server 2 and provide operation status information when they cannot perform autonomous processing (for example, when there is an abnormality or when processing requires human judgment). . The support management server 2 selects work assistance information by analyzing the operation status information from the reception robot 51, the arm robot 52, and the transport robot 53, and displays it on the UI terminals 41, 42, and the like. Since the operator can grasp the content of the support work from the work assistance information even without skillful knowledge, the operator can give appropriate instructions to the reception robot 51, the arm robot 52, and the transport robot 53 and operate them. In FIG. 2, lines connecting elements mean that they are connected so as to be able to provide information to each other or to one side. It also includes the case where it is installed in a device and connected by a wired or wireless network.

The support management server 2 includes a robot information database 21 and a work assistance information database 22, as shown in FIG. , and the work assistance information database 22 includes work assistance information 221 , motion status information 222 , robot information 210 , and language information 223 . These pieces of information may be stored in the storage means of the support management server 2, or may be stored in the storage server 6, and the support management server 2 may be connected via the network 3 to obtain the information as necessary. can be configured to Note that the storage server 6 may be provided in each of the UI terminals 41 and 42 . The support management server 2 selects an operator in charge of performing the support work in response to an operator support request from the robot, and selects and provides work assistance information useful for the support work to the operator in charge. The support management server 2 includes control means and communication means for communicating with a network (not shown), and may further include an operator database.

The robot information database 21 is a database of information about robots (robot information), and records robot type information 212, operating place information 213, etc. about the robot in association with the robot identification information 211. It is possible to search from The robot identification information 211 is information for uniquely identifying a robot corresponding to each robot on a one-to-one basis, such as a manufacturing number or reference number. The robot type information 212 is information for specifying the type of robot. For example, the functions, performance, and equipment provided by the robot may be used. For ready-made robots, the manufacturer, model name, model number, etc. may be used. If there is, it may be the manufacturer, place of operation, equipment used, and the like. The operating location information 213 is information about the location, facility, etc., where the robot is operating. The robot information, that is, the robot identification information 211, the robot type information 212, the operating location information 213, etc., identifies the robot that requested support, and preferably identifies the work to be performed by the robot. may be used when selecting work assistance information useful for the support work. The robot identification information 211 provided from the robot may include the robot type information 212 and the operating location information 213, in which case the robot information database 21 may not be provided.

The work assistance information database 22 is a database for selecting work assistance information 221 that is useful for robot assistance work by the operator. The work assistance information 221 can be retrieved from the operation status information 222 and/or the robot information 210, and the language information 223 may be associated therewith. The work assistance information 221 is information that is useful for the operator to assist the robot, and may be text information for giving instructions to the operator, moving images, still images, voices, etc., including the details of the work of the robot. For example, a video, still image, or sound recording the normal operation of the task corresponding to the task information, a video, still image, or sound recording the normal operation of the robot corresponding to the robot identification information, and/or a situation expected in normal operation including animations, still images, or voices that indicate , instructions for support work or operation contents, etc. in the input means of the UI terminal. Moving images and still images include those shot from the actual real world, those drawn by computer graphics (CG), and those that are a combination thereof. For example, a camera mounted on each robot as an environmental information acquisition means (a camera that acquires two-dimensional RGB images, an infrared camera that acquires thermal information, an infrared camera that can also acquire distance information to the subject, a stereo camera, a pattern projection camera, etc.) ), a video or still image from a camera installed at the robot's operating location, or a video or still image in which CG (including text) is superimposed on these videos or still images Alternatively, it may be a 2D or 3D CG moving image or CG still image entirely rendered by a computer. The CG moving image or CG still image may be prepared in advance, or may reflect the current situation based on information from a sensor mounted on the robot or a sensor installed at the working place. Also, the voice may be actual sound, computer-synthesized sound, or a combination thereof. The operation status information 222 corresponds to the information obtained from the robot, and includes at least task information related to each task, and further includes environmental information such as still image information, moving image information, audio information, etc. at the time of the support request or at the present time, and support request. It may also include positional information of the robot at the time or the present time, error information regarding an error that has occurred in the robot, and the like. The robot information 210 includes robot identification information 211 , robot type information 212 , operating place information 213 , etc., and can be acquired based on the robot information 561 . The language information 223 is information related to the language included in the work assistance information 221 . For example, when the description and voice of the moving image are in Japanese, the language information of Japanese is associated with the work assistance information 221 . When multiple languages are included by subtitles, dubbing, etc., multiple language information may be associated. A work assistance information ID that uniquely identifies each piece of work assistance information may be assigned.

The network 3 is not particularly limited, and includes the Internet. Community Antenna TeleVision) network, optical fiber network, wireless LAN (Local Area Network), CS (Communication Satellite) broadcasting, mobile communication system network (including 3G, 4G, 5G lines, etc.), etc. can be used. Multiple robots 51 , 52 , 53 and multiple UI terminals 41 , 42 may be connected to the network 3 . Via the network 3, the support management server 2 can be connected to a plurality of robots 51, 52, 53, the support management server 2 can be connected to a plurality of UI terminals 41, 42, and a plurality of UI terminals can be connected. The terminals 41, 42 and the plurality of robots 51, 52, 53 may be mutually connectable.

The user interface terminals 41 and 42 include output means 43, input means 44 and storage means 45, and also control means and communication means for communicating with a network (not shown). UI terminals include, for example, computers, laptop computers, personal digital assistants (including smartphones and tablet terminals), and the like. A UI terminal may be realized by distributing the means such as using a speaker as an output means, a keyboard/mouse as an input means, a storage server on a network as a storage means, and processing by a computer.

The output means 43 is means for outputting various information, and outputs at least the work assistance information 220 provided by the support management server 2 . The output means 43 includes, for example, a liquid crystal display (LCD), an organic EL display, a projector, a speaker, a printer, and the like. If a touch panel type display is used as the output means 43, it can also be used as the input means 44. FIG. Also, a plurality of output means 43 may be provided corresponding to the plurality of robots 51 to 53, or the information of the plurality of robots 51 to 53 may be displayed simultaneously or sequentially on one output means 43. . The output means 43 displays, for example, moving images and still images including the work content of the robot as work assistance information, and outputs voice from the speaker. Status (environment information, location information, error information, etc.) may be output. In this way, by outputting both the status during normal times and the status at the time of the support request or at the present time, it is easier to find abnormalities compared to normal times, and to understand the instructions or operations required to return to normal times. Become.

The input means 44 is means for executing support work requested by the robot, and for example, can give instructions to the robot and operate the robot. The input means 44 includes, for example, a keyboard, pointing device (mouse, touch pad, etc.), touch panel (including touch screen), controller, microphone, and the like. The input means 44 can also operate a UI terminal. For example, the work assistance information displayed on the output means 43 by the input means 44 can be operated (playback, stop, fast forward, reverse playback, enlargement, reduction, movement, etc.). Alternatively, the desired information may be obtained by manipulating environmental information at the time of the support request or at the present time.

The storage unit 45 stores instruction operation information 451, work assistance information 452 provided from the support server 2, notification settings 453, operator information 454, and the like. The storage means 45 includes, for example, a USB memory, a semiconductor memory, a CD, a DVD, an HDD, etc., and may have a plurality of storage means. Instruction operation information 451 is information on instructions or operation details for the robot input by the input means 44 and is provided to the robots 51 to 53 via the network 3 . The work assistance information 452 is provided from the support server 2 and provided to the operator via the output means 43 . The notification setting 453 is a setting of necessary information and unnecessary information among the information provided from the support management server 2, and it is preferable that each operator can set it individually. Furthermore, it is preferable to be able to set for each robot type or work content. For example, since the operator is familiar with the support work for the type of robot and work that he is good at, it is not necessary to provide work assistance information. The operator information 454 is information about the operator using the UI terminal, and includes, for example, the language used by the operator, the type of robot or work that the operator is good at, the work experience of the operator, the ability of the operator (for example, each average processing time for support work, etc.). The UI terminals 41 and 42 can provide notification settings 453 and operator information 454 to the support management server 2, and the support management server 2 determines information to be provided to the UI terminals based on the provided notification settings 453. Alternatively, the operator in charge may be selected based on the provided operator information 454, or appropriate work assistance information may be selected based on the provided operator information 454, considering the support content and/or language information. You may

The robots 51 to 53 are a plurality of various robots that perform various tasks. For example, a reception robot 51 that receives requests from customers, etc., and an arm robot 52 that recognizes an object, holds it, and delivers it to a transport robot. , and a transport robot 53 that transports the object to the destination. The robots 51 to 53 include at least autonomous control means 54, support request means 55, and storage means 56, and may further include error detection means 57, environment information acquisition means 58, position information detection means 59, and the like. Although it does not, it includes means for realizing the functions of each robot and communication means for communicating with the network. Furthermore, the storage means 56 includes robot information 561 and operation status information 562, and the operation status information 562 includes at least task information 563, and may further include error information 564, environment information 565, position information 566, and the like. .

The autonomous control means 54 controls the means for realizing the functions of each robot to autonomously perform a plurality of tasks. For example, in the case of the reception robot 51, there are multiple tasks, such as waiting until there is a customer request, moving to obtain the customer request, patrol, receiving and recognizing the customer request, and transferring the customer request to another robot. etc. In the case of the arm robot 52, multiple tasks include waiting for a request, picking up an object, grasping the object using joint functions, releasing the grip, and the like. In the case of the transport robot 53, a plurality of tasks such as waiting for a request, moving to a loading location, waiting, autonomous transport, standby for loading and unloading, and return movement are executed. Further, the autonomous control means 54 causes the support request means 55 to request support from the support management server 2 when the robot cannot operate autonomously.

The support requesting means 55 can execute a support request to the support management server 2, and provides the support management server 2 with operation status information 562 including at least task information 563 when requesting support. Further, the support request means 55 may provide the robot information 561 to the support management server 2 at the time of requesting support, but may provide the robot information 561 at another timing. The operating status information 562 may further include error information 564, environment information 565, location information 566, and the like.

The storage means 56 stores various information, and the storage means 56 includes, for example, USB memory, semiconductor memory, CD, DVD, HDD, etc., and may have a plurality of storage means. The robot information 561 stored in the robot includes at least its own robot identification information, and may also include robot type information, operating location information, etc., and is provided to the support management server 2 . The support management server 2 searches the robot information database 21 for robot type information 212, operating place information 213, etc. from the provided robot identification information as necessary. When the operation status information 562 is provided to the support management server 2 at the time of requesting support, it includes at least task information 563 associated with the work requiring support, error information 564, environment information 565, location information 566, and the like. may be included. Note that the task information 563 does not have to be included when the operation status information 562 is provided at a time other than when support is requested (for example, provided by an operator's instruction after a support request). Task information 563 is associated with one or more of a plurality of tasks performed by the robot.

The error detection means 57 detects a failure or anomaly occurring in the robot, and acquires error information 564 by classifying the failure or anomaly into predetermined types of failure or anomaly. The error information 564 may be information specifying the type of failure or abnormality (including an error code), or may be information including the content of the failure or abnormality (for example, self-position lost).

The environment information acquisition means 58 acquires environment information 565 around the robot, and is a means for detecting the presence and position (relative or absolute position) of obstacles around the robot, and a means for detecting the surrounding image. or a means for acquiring voice. As the environment information acquisition means 58, for example, LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) using laser light, a range sensor, a stereo camera using multiple cameras, a pattern projection camera, radio waves (millimeter waves) A radar using , a magnetic sensor, a camera (CCD camera, CMOS sensor, infrared camera) for acquiring a two-dimensional image, an audio acquisition means for acquiring audio, and the like can be used. In addition, at least an image (still image or moving image) in front of the movement direction of the robots 51 to 53 can be acquired, and if a high-definition 3D LiDAR or the like is used, the surrounding obstacles can be judged or confirmed by the pattern of the measurement points. is also possible.

The position information detection means 59 is means for estimating the positions of the robots 51 to 53 in the map information and detecting the positions of the joints, arms, etc. of the robot, and acquires position information 566 thereof. When estimating the self-position, for example, it is realized by executing a position estimation program, for example, by a CPU (Central Processing Unit) and working memory, or by an integrated circuit. As the position information detection means 59, it is sufficient if the position of the mobile object in the map information can be specified. The position of the robot may be calculated based on the matching result by matching with prepared map information. Other position information estimation means 59 converts coordinates specified by GPS into coordinates of map information, converts coordinates specified by communication antennas or base stations into coordinates of map information, and converts coordinates specified by communication antennas or base stations into coordinates of map information. A symbol indicating a position may be written on the floor, image recognition of the symbol may be performed, and the symbol may be identified by associating the symbol with map information. When detecting the position of a joint or an arm, for example, the position may be detected from the amount of displacement by an actuator that drives the joint or arm. The position may be detected by detecting the state of the arm after driving with a camera or sensor. Note that the position information detection means 59 may be provided in the support management server 2 instead of the robots 51-53.

[Normal operation]
4 to 8 illustrate a series of operations in a system composed of a reception robot 51, an arm robot 52, and a transport robot 53, recognizing a customer's request and fulfilling the request. In this example, the operation of delivering the beverage requested by the customer to the customer's seat is performed.

FIG. 4 shows a state in which the reception robot 51, the arm robot 52, and the transport robot 53 are on standby. The reception robot may be roaming or stationary. FIG. 5 shows a state in which the request of the customer 10 is accepted by the robot 51 and the customer's instruction is recognized. The reception robot 51 may move toward the customer 10 when it recognizes that the customer 10 has entered the store. The reception robot 51 responds to the customer 10 and allows the customer to input a request by voice or a touch panel. When the customer 10 instructs a beverage, the reception robot 51 receives the beverage as the request of the customer 10 by voice recognition or screen touch, and transmits the recognized request of the customer 10 to the arm robot 52 and the transport robot 53 through the network.

FIG. 6(A) shows a state in which the arm robot 52 recognizes the customer's request transmitted from the reception robot and identifies the object, and (B) shows the arm robot 52 as the object. (C) shows a state in which the arm robot 52 moves its own joints and lifts the beverage 11 by recognizing the beverage 11 on the desk and moving the arm to grasp the beverage 11 with the hand at the tip. showing.

7A shows a state in which the transport robot 53 approaches the vicinity of the arm robot 52 and waits at the loading standby position, and FIG. 7B shows a state where the arm robot 52 places the beverage 11 on the transport robot 53. (C) shows a state in which the arm robot 52 places the beverage 11 on the carrier robot 53, releases the grip by the hand, and finishes its role, and the carrier robot 53 moves the beverage 11. 11 shows a state in which transportation of No. 11 is started.

8A shows a state in which the transport robot 53 is in the process of transporting the beverage 11 to the seat of the customer 10, and FIG. 8B shows the transport robot 53 transporting the beverage 11 to the seat of the customer 10. FIG. , shows the state of waiting at the unloading waiting position, and (C) shows the state where the customer 10 has received the beverage 11, the transport robot 53 has finished its role, and has returned to the waiting place.

[Database example]
FIG. 9 is an example of the robot information database 21. As shown in FIG. The robot information database 21 stores robot type information 212 , robot operating location information 213 , and various functions 214 to 217 in association with robot identification information (ID) 211 . In this embodiment, the robot ID 211 consists of 7 digits of letters and numbers, where the first digit is information about the operating location, the 2nd to 4th digits are information about the robot type, and the 5th to 7th digits are the numbering for each robot type. , which itself includes robot type information 212 and operating location information 213, but it is also possible to simply number each robot. In the robot type information 212, different types are assigned not only to robots manufactured by manufacturers but also to robots manufactured by the same company α that have different models or different functions. As for the operating place information 213, even if it is the same company, it is distinguished by factory or warehouse, and furthermore, the operating place is specifically classified such as reception desk in the factory. As the various functions 214 to 217, voice recognition 214, image recognition 215, moving means 216, and grasping means 217 are exemplified. not something. Work assistance information can be associated with the robot information 210 .

FIG. 10 is an example of the work assistance information database 22a. The work assistance information database 22 a includes work assistance information ID 220 , robot information 210 on robot type and robot operating location, work assistance information type 221 a , work assistance information 221 , language information 223 and error information 224 . In the work assistance information database 22a, work assistance information 221 can be selected from robot information 210, language information 223, and error information 225. FIG. However, when selecting from work assistance information of IDs 1 to 3, work assistance information can be selected only from robot information 210, and when selecting from work assistance information of IDs 1 to 6, robot information 210 and It can be selected from language information 223 . Thus, the selection of work assistance information does not require all of the robot information 210, language information 223 and error information 224. FIG. The work assistance information for IDs 1 to 6 is animation, IDs 1 to 3 are for Japanese, and IDs 4 to 6 are animations for English. The work assistance information of ID1 is, for example, a moving image of the normal operation of the reception robot manufactured by X company in the XX factory of company A explained in Japanese. The work assistance information of ID7 is a still image, for example, an image of the XX warehouse of company B at normal times. The work assistance information with IDs 8 to 11 is voice, and corresponding work assistance information is selected according to the language information 223 and the error information 224 . For example, if there is no object to be transported, the operator is notified by voice that "The object has arrived at the destination, but the object to be transported cannot be found. Please indicate the object to be transported." In addition, in the case of duplicate deliveries, the operator is notified by voice that "The object to be transported has already been transported. Please specify another object to be transported." The work assistance information type 221a indicates the data format of the work assistance information, such as moving image, still image, voice, and text.

FIG. 11 is an example of another work assistance information database 22b. The work assistance information database 22b includes work assistance information ID 220 (only the ID is shown in the figure), robot information 210 of the robot type and robot operating place, language information 223, task information 225, and work assistance information 221 as the start position of the normal operation animation. and video length information. In the work assistance information database 22b, work assistance information 221 can be selected from robot information 210, language information 223, and task information 225. FIG. The task information 225 categorizes the work content of the robot. The manipulator manufactured by Company Y is divided into "Waiting for Request", "Gripping", "Transporting" and "Releasing Grip". waiting", "waiting for loading", "moving", "waiting for unloading" and "moving back". The start position and the length of the animation during normal operation are specified for each, and the animation of the designated start position and animation length among the series of animations during normal operation is provided to the UI terminal as the auxiliary work information 221. can. In this way, by using the work assistance information database 22b, the corresponding work assistance information 221 can be selected from the task information 563 included in the operation status information 562 at the time of assistance request provided from the robot. It should be noted that when work assistance information with IDs 1 to 6 is selected in the work assistance information database 22a shown in FIG. 10, more specific work assistance information may be selected using the work assistance information database 22b shown in FIG. .

FIG. 12(A) is a layout diagram of each transport object A01 to E01 in the warehouse, (B) is an example of another work assistance information database 22c, and (C) is an example of a still image. . The work assistance information database 22c includes a work assistance information ID 220 (only the ID is shown in the figure), a still image file name as work assistance information 221, and information on the coordinates of the upper left vertex and the lower right vertex as the position range 226a. contains. In the work assistance information database 22c, the work assistance information 221 can be selected from the position information provided by the robot. For example, a still image included in the position range 226a of the position information provided by the robot can be selected and provided to the UI terminal. As the still image of the work auxiliary information 221, for example, as shown in FIG. 12(C), images of the transportation objects A01 to E01 during normal operation may be used, but images other than the transportation objects in the warehouse may also be used. Each position range 226a is a fixed range from each of the transport objects A01 to E01 in this embodiment, but it is not limited to this aspect. For example, the inside of the warehouse may be divided into a matrix, each division may be set as a position range, and an image that can be confirmed in normal times from that position may be used as the auxiliary work information 221 . When the work assistance information of ID7 is selected in the work assistance information database 22a shown in FIG. 10, more specific work assistance information may be selected using the work assistance information database 22c shown in FIG.

[Behavior when requesting support]
First, an example in which the reception robot 51 asks the operator for assistance when it cannot recognize the content of the request of the customer 10 will be described. When the receiving robot 51 cannot recognize the content of the request of the customer 10 during the work of "customer request reception", the error detection means 57 recognizes "customer request cannot be recognized". The reception robot 51 prepares the video and audio (environmental information 565) of the correspondence with the customer 10 recorded by the image acquisition means of the environment information acquisition means 58, and furthermore, the position acquired by the position information detection means 59. Prepare information 566 . Then, the reception robot 51 transmits robot information 561 and operation status information 562 to the support management server 2 via the network 3 by the support request means 55 . The operation status information 562 includes task information 561 "customer request reception", error information 564 "customer request unrecognizable", video and audio (environmental information 565) corresponding to the customer 10, and position information 566. You can stay.

From the received robot information 561, the management support server 2 uses the robot information database 21 as necessary to identify the robot type and operating place of the reception robot 51, and if necessary, the received operation status information 562 and At the same time, the work assistance information 221 to be provided to the user interface terminal is selected by matching with the work assistance information database 22 . Here, the management support server 2 may select a UI terminal with an appropriate operator in consideration of language information, time, degree of operator congestion, etc. according to the country and region where the robot is operating. For example, the management support server 2, using the work assistance information database 22b, selects the animation of ID1-3 from the robot information 561 and the task information 561 "customer request reception", and the language information of the selected UI terminal, Send to terminal. Furthermore, the management support server 2 may transmit the operation status information 562 received from the robot, for example, video and audio (environmental information 565) of correspondence with the customer 10 to the UI terminal. The operation status information 562 may be directly transmitted from the reception robot 51 to the UI terminal.

FIG. 13 shows an example of a display screen 411 of the UI terminal 41, in which a work assistance information area 412 is displayed on the right side and an instruction/operation area 413 is displayed on the left side. In the work auxiliary information area 412, an area 412a for reproducing a moving image of normal operation provided as work auxiliary information and a play button 412b are displayed. An upper area 413a of the instruction/operation area 413 shows the situation at the time of the request for support from the robot or the current situation, and in FIG. A region 413b in which beverages are arranged is a region for inputting a request of the customer 10, and a lower circle mark is an enter button 413c. While confirming the moving image of the normal operation reproduced in the area 412a, the operator judges the customer's request from the moving image and voice (environmental information 565) of the correspondence with the customer 10, A desired item is selected, and an instruction is transmitted to the reception robot 51 by the decision button 413c. Instead of specifying the desired beverage 11, the UI terminal 41 may be connected to the reception robot 51, and the operator may respond to the customer 10 by voice, video, or text to specify the beverage 11 desired.

Next, an example will be described in which the arm robot 52 takes over the information on the desired beverage 11 from the reception robot 51, but fails to grip the beverage 11 from the beverage pool. When the arm robot 52 cannot determine the picking object during the work of "holding", the error detection means 57 recognizes "unidentifiable". Furthermore, the arm robot 52 records the beverage pool by the image acquisition means of the environment information acquisition means 58, and transmits robot information 561 and operation status information 562 to the support management server 2 via the network 3 by the support request means 55. . The operation status information 562 includes the task information 561 "being grasped", the error information 564 "unidentifiable", the image of the recorded beverage pool (environmental information 565), and the requested beverage 11, which is the object to be picked. May contain information.

As in the case of the reception robot 51 , the management support server 2 selects a UI terminal with an appropriate operator, matches it with the work assistance information database 22 , and selects work assistance information 221 . For example, the management support server 2 selects the moving image of ID5-2 from the robot information 561 and the task information 561 "holding", and the language information of the selected UI terminal, using the work auxiliary information database 22b, Send to Furthermore, the management support server 2 may transmit the operation status information 562 received from the robot, for example, the image of the recorded beverage pool (environmental information 565), the information of the desired beverage 11 that is the object to be picked, etc. to the UI terminal. good. The operation status information 562 may be directly transmitted from the arm robot 52 to the UI terminal.

FIG. 14 shows an example of a display screen 411 of the UI terminal 41, in which a work assistance information area 412 is displayed on the right side and an instruction/operation area 413 is displayed on the left side. In the work auxiliary information area 412, an area 412a for reproducing a moving image of normal operation provided as work auxiliary information and a play button 412b are displayed. An instruction/operation area 413 shows an image of an actual beverage pool and an unmarked beverage 11 that is an object to be picked. In the area 413, the beverage 11a indicated by the dotted line is the position that the arm robot 52 normally grips, and the beverage 11b indicated by the solid line is the actual position. Since the arm robot 52 did not have a beverage at the gripping position under normal conditions, an error of "unidentifiable" occurred. The operator selects and instructs a picking object with a dashed line 414 while confirming the moving image of normal operation reproduced in the area 412a. The UI terminal 41 may be connected to the arm robot 52 and the operator may operate the hand of the arm robot 52 to pick up the desired beverage 11b instead of indicating the picking object.

Further, an example will be described in which the transport robot 53, which moves while estimating its own position by the position estimating means 59, asks the operator for assistance when it loses its own position. FIG. 15A shows map information 70 indicating the floor plan of the movement range of the transport robot 53, which may be created in advance by importing an existing map such as a floor plan of a building or an architectural drawing. Simultaneous Localization and Mapping (SLAM) using the environment information acquisition means 58 by moving the transport robot 53 within the movement range of the transport robot 53 may simultaneously perform self-position estimation and environment map creation and acquire it. Body or other computer generated map information may be used.

FIG. 15B shows an example of the display screen 411 of the UI terminal 41, in which a work assistance information area 412 is displayed on the right side and an instruction/operation area 413 is displayed on the left side. The current floor plan 71 is shown in the instruction/operation area 413, and an object 71a not included in the map information 70 exists. The transport robot 53 is located in the lower left corner of the room in this figure. The transport robot 53 has four wheels as moving means, a LiDAR and a camera as environment information acquiring means 58 , and a position estimating means 59 . LiDAR sequentially irradiates micropulse laser light at each predetermined step angle, generates reflected light or scattered light from obstacles such as surrounding walls, and detects the returned reflected light or scattered light with the LiDAR detector. Then, by measuring the time from irradiation to return, the distance and direction to each measurement point are measured, and the polar coordinates (distance, angle) of each obstacle centered on LiDAR as environmental information can be obtained. Note that the polar coordinates of each measurement point may be converted into rectangular coordinates. Then, the position estimation means 59 matches the environment information acquired by LiDAR with the map information 70 prepared in advance to estimate the self position.

16A and 16B are diagrams for explaining a state in which the transport robot 53 has lost its own position due to an object 71a not on the map. FIG. 16A shows the actual position of the transport robot 53 in the current floor plan 71, and FIG. Each acquired measurement point indicated by a black circle does not match the map information 70, indicating a state in which the self-position is erroneously recognized as being to the left of the actual position. The arrow in FIG. 16B indicates that the measurement point is shifted on the map information by the object 71a. Furthermore, in FIG. 16B, an operation method that can be used for the recovery procedure is displayed in the work auxiliary information area 412 on the right side. In the upper area 412a, the operation method of the transport robot 53 is explained by animation, and the operator can operate the movement of the transport robot 53 and the direction for acquiring environmental information while referring to the area 412a. . Image information acquired from the camera of the transport robot 53 is displayed in the middle area 412b, and environment information of LiDAR acquired from the transport robot 53 is displayed in the lower area 412c.

FIG. 17A shows surrounding image information 72 acquired by the camera of the environment information acquiring means 58 of the transport robot 53 displayed in the instruction/operation area 413 . The image information 72 is an image captured at a position where the transport robot 53 has lost its own position. The operator operates the input means 44 of the UI terminal 41 to select one point on the image information with the cursor 73 , sets the provisional destination 73 a , and transmits the provisional destination 73 a to the transport robot 53 . FIG. 17B is a diagram showing surrounding environment information 74 acquired by the LiDAR of the environment information acquiring means 58 of the transport robot 53 displayed in the instruction/operation area 413 . The operator may operate the input means 44 of the UI terminal 41 to select one point on the environment information with the cursor 73 to set the provisional destination 73a.

The provisional destination can be set by the input means 47, for example, by inputting coordinates with a keyboard, or by selecting a point on environmental information or image information with a pointing device or a touch panel, and setting the selected coordinates to the provisional destination. You may choose to Also, for example, by drawing a travel route on environmental information or image information, the end point of the route may be set as the provisional destination. The input means 47 may be capable of enlarging or reducing environmental information and/or image information. Furthermore, the input means 47 may be capable of designating the current position of the transport robot 53 in the map information. For example, when the operator can estimate the current position of the transport robot 53 in the map information from the environmental information and/or the image information, when the input means 47 specifies the current position of the mobile object in the map information, the transport robot 53 is moved to the specified location. Autonomous movement can be resumed with the current position as the self-position.

FIG. 18A is a diagram showing a state in which the temporary destination 73a designated by the operator has been reached, and shows environmental information (display of only neighboring SLAMs) acquired at each position during movement. The transport robot 53 continuously acquires the environment information while moving, and repeats movement toward the provisional destination 73a set in the environment information with the acquisition position of the environment information as the current position. FIG. 18(A) shows the total sum of ambient environment information 74 newly acquired while the transport robot 53 is moving from the point where the self-position is lost to the temporary destination 73a. The transport robot 53 that has arrived at the temporary destination 73a determines whether the position information detecting means can estimate its own position at that position. If the current self-position estimation succeeds, it can return to autonomous movement. If it fails, it again requests the UI terminal 41 to set a temporary destination, or waits for an operation based on the work assistance information of the UI terminal 41 . If the current position is successfully estimated, it is preferable to reflect the environmental information acquired during the movement to the temporary destination 73a in the map information. FIG. 18B shows map information 75 in which the current position of the transport robot 53 has been successfully estimated at the provisional destination 73a, and the sum total of newly acquired surrounding environmental information is reflected.

FIG. 19 shows an example of the display screen 411 of the UI terminal 41 in a situation where the object 71a that exists in the map information does not actually exist. , an instruction/operation area 413 is displayed on the left side. When the transport robot 53 moves to a position close to the object 71a, the object 71a does not exist, and an abnormality is detected and an emergency stop is made. The upper right area 412a displays an RGB image expected to be normal, and the right center area 412b displays a thermal imaging camera image or a CG image based on the thermal imaging camera image, An image of an actual thermal imaging camera is displayed in the lower right area 412c. By using the thermal image, it is possible to provide the operator with work assistance information even in environments such as dark places that are not suitable for RGB cameras. The operator compares the normal images 412a and 412b with the actual image 412c, and on the premise that the object 71a does not exist, operates or instructs the robot to return to normal operation.

FIG. 20 shows an example of the display screen 411 of the UI terminal 41 in a situation where the object 71a that exists in the map information does not actually exist and an unidentified object exists. 412a and 412b are displayed, and an instruction/operation area 413 is displayed on the left side. When the transport robot 53 moves to a position close to the object 71a, the object 71a does not exist, and an unidentified object exists, so that the transport robot 53 loses its own position. In the upper right area 412a, an image is displayed in which the determination result of image recognition is overlaid on the expected image in the case of normality. In the lower right area 412b, an image is displayed in which the determination result of image recognition is overlaid on the actual image. The operator compares the normal image 412a and the actual image 412b, and finds that the object 71a should be placed in front in the normal state, but the object 71a actually does not exist and the part A falls. recognizing that the robot is moving, the robot is operated or instructed to avoid the part A to return to normal operation. The number of the determination result is the recognition accuracy, which indicates the accuracy of the image recognition, and is preferably displayed as reference information for the operator, but the recognition accuracy does not have to be displayed.

FIG. 21 shows an example of the display screen 411 of the UI terminal 41 in a situation where the object 71a that exists in the map information does not actually exist. , an instruction/operation area 413 is displayed. When the transfer robot 53 moved to a position close to the object 71a, the intended short-range object 71a did not exist, and the LiDAR detected an unidentified obstacle in the long range. be. An upper right area 412a displays an RGB image in the normal case. The lower right area 412b displays an actual LiDAR point group or a 3DCG image based on the LiDAR point group. In the lower right region 412b, an obstacle based on the pillar 71b was detected at a position 415b away from the self-position 415a where the robot's LiDAR was placed. The operator compares the map information, the normal image 412a, and the actual image 412b, and on the premise that the object 71a does not exist and the pillar 71b is detected as an obstacle, operates or instructs the robot to operate normally. return to

FIG. 22 is an example of notification settings 453 in the UI terminal. The notification setting 453 is a setting of necessary information and unnecessary information among the information provided from the support management server 2, and can be individually set by each operator. The notification setting 453 includes a work assistance information ID, an operator ID, and necessity of notification. The work assistance information ID is associated with the ID 220 of the work assistance information database 22a of FIG. The operator ID is information that uniquely identifies an operator. Notification necessity can be set to those requiring notification (ON) and those not requiring notification (OFF). FIG. 22A shows the notification settings 453 of an operator with an ID of ABC, but since he is unfamiliar with the work, notification of all work assistance information is turned ON. FIG. 22B shows the notification settings 453 of the operator with the ID XYZ, but because he is skilled in the work, notification of all work assistance information is turned off. FIG. 22C shows the notification setting 453 of the operator with the ID Iroha. Since the operator is accustomed to work related to the warehouse, the notification is turned off, but the other operators turn the notification on. In this way, by setting the necessity of notification for each operator, efficient work is possible.

1 robot support system 2 support management server 3 network 6 storage server 41, 42 user interface terminal (UI terminal)
51 reception robot 52 arm robot 53 transport robot

Claims (10)

A robot support system including a robot, a support management server, and a user interface terminal,
The robot and the support management server are connectable via a network, the support management server and the user interface terminal are connectable via a network,
The robot includes control means capable of autonomously executing a plurality of tasks, support request means capable of requesting support from the support management server, and storage means storing robot information, wherein the support request means , the operation status information including task information about each work or the operation status information and the robot information can be provided to the support management server;
The support management server includes a work assistance information database in which work assistance information associated with status information and/or robot information is stored. Alternatively, work assistance information can be selected based on the robot information, and the selected work assistance information can be provided to the user interface terminal,
The robot support system, wherein the user interface terminal includes an input means capable of instructing or operating the robot and an output means, and the output means can provide the user with work assistance information provided from the support management server. . 2. The robot support system according to claim 1, wherein said work assistance information includes work content of said robot. The work assistance information may be a moving image, still image, or sound recording normal operation of the work corresponding to the task information associated with the work assistance information, or a normal operation of the robot corresponding to the robot identification information associated with the work assistance information. 3. The robot assistance system according to claim 1 or 2, comprising moving images, still images or sounds recording actions and/or moving images, still images or sounds showing expected situations in normal times. The support management server stores, in the work assistance information database, moving images recorded in time series of normal operations of a plurality of tasks of the robot in association with task information of the robot, classified for each task, and 4. The robot support according to any one of claims 1 to 3, wherein the user interface terminal can be provided with moving images of categories corresponding to task information included in the operation status information provided from the robot as work assistance information. system. The robot includes error detection means, and can provide error information detected by the error detection means to the support management server;
The work assistance information database stores the work assistance information in association with error information,
5. The support management server according to any one of claims 1 to 4, wherein the support management server searches the work assistance information database based on the error information provided from the robot, and can provide the retrieved work assistance information to the user interface terminal. 1. The robot support system according to claim 1. 6. The user interface terminal according to any one of claims 1 to 5, wherein the user interface terminal can individually set whether or not to transmit work assistance information for a plurality of tasks of the robot to the support management server. Robotic assistance system. The work assistance information database stores the work assistance information in association with linguistic information,
The user interface terminal is capable of notifying the support management server of language information used by the user,
6. The support management server is capable of searching the work assistance information database based on language information provided from the user interface terminal and providing the retrieved work assistance information to the user interface terminal. The robot assistance system according to any one of Claims 1 to 1. The robot information includes at least one of robot identification information that uniquely identifies each robot, robot type information that identifies the type of robot, and operating location information that identifies the operating location of the robot,
The support management server includes a robot information database that stores robot type information and/or operating location information about the robot in association with the robot identification information, and based on the robot identification information provided from the robot: Robot support according to any one of claims 1 to 7, wherein the robot type information and/or the operating location information is searched, and work assistance information is selected based on the searched robot type information and/or the operating location information. system. The support request means includes environmental information around the robot, image information around the robot, audio information around the robot, position information of the robot on a map, and relative arrangement of components of the robot. 9. The robot support system according to any one of claims 1 to 8, wherein at least one piece of information is obtained and included in said operation status information and provided to said support management server. The robots include a plurality of different types of robots that perform different tasks, each robot being placed at a plurality of operating locations;
10. The robot support system according to any one of claims 1 to 9, wherein said user interface terminals are arranged at a plurality of bases.

Images