CN117255978A - System and method for management of a robot team - Google Patents
System and method for management of a robot team Download PDFInfo
- Publication number
- CN117255978A CN117255978A CN202280029914.3A CN202280029914A CN117255978A CN 117255978 A CN117255978 A CN 117255978A CN 202280029914 A CN202280029914 A CN 202280029914A CN 117255978 A CN117255978 A CN 117255978A
- Authority
- CN
- China
- Prior art keywords
- robot
- responsibility
- responsibilities
- data
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title abstract description 93
- 230000007613 environmental effect Effects 0.000 claims abstract description 52
- 238000007726 management method Methods 0.000 claims description 179
- 239000000126 substance Substances 0.000 claims description 22
- 230000036541 health Effects 0.000 claims description 10
- 230000033001 locomotion Effects 0.000 claims description 10
- 230000003213 activating effect Effects 0.000 claims description 8
- 238000003825 pressing Methods 0.000 claims description 8
- 238000013501 data transformation Methods 0.000 claims description 2
- 238000004458 analytical method Methods 0.000 description 43
- 238000004891 communication Methods 0.000 description 34
- 238000012423 maintenance Methods 0.000 description 30
- 230000000694 effects Effects 0.000 description 29
- 230000015654 memory Effects 0.000 description 28
- 230000008859 change Effects 0.000 description 27
- 230000008569 process Effects 0.000 description 22
- 230000006870 function Effects 0.000 description 20
- 238000010801 machine learning Methods 0.000 description 18
- 238000004886 process control Methods 0.000 description 17
- 238000012545 processing Methods 0.000 description 17
- 238000013473 artificial intelligence Methods 0.000 description 13
- 238000004519 manufacturing process Methods 0.000 description 13
- 238000012544 monitoring process Methods 0.000 description 13
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 12
- 230000002776 aggregation Effects 0.000 description 11
- 238000004220 aggregation Methods 0.000 description 11
- 238000005259 measurement Methods 0.000 description 10
- 238000007689 inspection Methods 0.000 description 9
- 238000013480 data collection Methods 0.000 description 8
- 239000007789 gas Substances 0.000 description 8
- 238000003860 storage Methods 0.000 description 8
- 230000004048 modification Effects 0.000 description 7
- 238000012986 modification Methods 0.000 description 7
- 230000005540 biological transmission Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 6
- 238000013439 planning Methods 0.000 description 6
- 239000000047 product Substances 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006378 damage Effects 0.000 description 5
- 238000013523 data management Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 238000009529 body temperature measurement Methods 0.000 description 4
- 238000007405 data analysis Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004140 cleaning Methods 0.000 description 3
- 230000000977 initiatory effect Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000033228 biological regulation Effects 0.000 description 2
- 238000013481 data capture Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000012530 fluid Substances 0.000 description 2
- 239000000446 fuel Substances 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000007257 malfunction Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000002689 soil Substances 0.000 description 2
- 238000010200 validation analysis Methods 0.000 description 2
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 239000003570 air Substances 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 235000013361 beverage Nutrition 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 210000000078 claw Anatomy 0.000 description 1
- 230000009194 climbing Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000009833 condensation Methods 0.000 description 1
- 230000005494 condensation Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000003862 health status Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000007654 immersion Methods 0.000 description 1
- 230000001976 improved effect Effects 0.000 description 1
- 238000010921 in-depth analysis Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 229920001690 polydopamine Polymers 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 238000013442 quality metrics Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000008093 supporting effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000026676 system process Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 238000004065 wastewater treatment Methods 0.000 description 1
Landscapes
- Manipulator (AREA)
Abstract
A system or method comprising: defining responsibilities based on factors associated with the responsibilities or environmental data associated with the system; assigning the responsibilities to a robotic fleet based on capabilities of the robot; generating a schedule of the responsibilities and the robot; and managing the robotic team using feedback.
Description
Technical Field
Various embodiments of the present disclosure relate generally to management of robotic services in, for example, industrial facilities, and more particularly to receiving, defining, distributing, and managing responsibilities performed by a robotic team.
Background
Industrial facilities such as chemical plants, refineries, power plants, etc. can be very large and can involve a large number of simultaneous processes throughout the facility. Some aspects of these processes may be managed remotely by operators in a central control room, but other aspects require tasks to be completed at various locations in the facility. Such tasks may include monitoring and adjusting equipment, sensing environmental or other conditions within a facility, and the like. Some of these tasks may require 24 hours of attention per day, and others may require exposure to potentially dangerous conditions or work in confined or difficult to reach spaces. Increasingly, facility operators are turning to the use of robots to accomplish industrial tasks, for example, to reduce costs, extend facility availability time, or to protect workers from potentially dangerous conditions. However, the scope of tasks required in complex facilities may require multiple robots of different types and capabilities provided by different manufacturers. These robots may not operate according to a common protocol or command and may not transmit data by a similar method. Thus, as the number of robots and tasks allocated by the robots increases, it may become more and more complex to integrate robot fleets into industrial facilities. This complexity can be a significant obstacle to the adoption of robots in industrial facilities.
The present disclosure is directed to overcoming one or more of the challenges set forth above as well as other challenges.
Disclosure of Invention
Additional objects and advantages of the disclosed embodiments will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the disclosed embodiments. The objects and advantages of the disclosed embodiments will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims. As will be apparent from the following embodiments, the disclosed systems and methods have the advantage that tasks may be assigned to robots and other operators in an efficient manner, which may maximize the use of robots and other operators available to perform tasks within a facility. For example, the disclosed systems and methods discussed below may allow responsibilities to be designed, assigned, and monitored based on all available robot or operator types, capabilities of the robot or other operator, and feedback.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments as claimed.
According to one aspect, a system for management of a robotic team includes: a plurality of robots including a first robot of a first robot type and a second robot of a second robot type; and a robot service platform, the robot service platform comprising: a first data adapter corresponding to a first robot type and configured to convert responsibility feedback collected by the first robot into a generic data format compatible with the robot services platform; a second data adapter corresponding to a second robot type and configured to convert responsibility feedback collected by the second robot into a generic data format; a first control adapter corresponding to a first robot type and configured to convert the generic role control information into first robot role control information compatible with the first robot; a second control adapter corresponding to a second robot type and configured to convert the generic role control information into second robot role control information compatible with the second robot; and a responsibility manager. The responsibility manager is configured to: selecting a first robot from the plurality of robots to perform a selected robot role based on the capabilities of the first robot and the characteristics of the selected robot role in the plurality of robot roles; transmitting first robot role control information to the first robot via the first control adapter, the first control adapter having converted the first generic robot role control information into the transmitted first robot role control information; and receiving, from the first data adapter, responsibility feedback of the first data adapter from data transformations obtained by the first robot.
The responsibility manager may be further configured to: selecting a second robot from the plurality of robots to perform the selected robot responsibilities based on the capabilities of the second robot and the characteristics of the selected robot responsibilities; transmitting second robot responsibility control information to the second robot via the second control adapter, the second control adapter having converted the first generic robot responsibility control information into the transmitted second robot responsibility control information; and receiving, via the second data adapter, responsibility feedback collected by the second robot.
The responsibility manager can consider the first robot and the second robot to perform at least a portion of the selected robot responsibilities and select the second robot to perform at least a portion of the selected robot responsibilities in place of the first robot.
The responsibility manager can consider the first robot and the second robot to perform at least a portion of the selected robot responsibilities and select the second robot to perform at least a portion of the selected robot responsibilities in cooperation with the first robot.
The second data adapter may be different from the first data adapter and the second control adapter may be different from the first control adapter.
The received responsibility feedback may include one or more of the following information: information that the selected robot responsibilities are fully completed, information that the selected robot responsibilities are partially completed, information that the selected robot responsibilities are not initiated, health information about the first robot, status information about the first robot, capability information about the first robot, photographs, videos, environmental data, sensor readings, movement data about the first robot, and position data about the first robot.
The first robot responsibility control information transmitted to the first robot may include one or more of a multi-step responsibility instruction or a separate operation instruction including one or more of: navigating to a destination, taking a photo, recording video, recording sound, measuring environment, measuring temperature of a substance, measuring temperature of air, measuring humidity, determining instrument readings, measuring presence or concentration of a gas or chemical, emitting light of a specific wavelength, intensity and/or emission pattern, emitting sound of a specific tone, intensity and/or emission pattern, emitting a radio frequency homing beacon, transmitting stored data via a radio frequency or wireless data network, connecting to a power source, connecting to a radio frequency or wireless data network, connecting to a data network port, modifying settings of a valve or other control, adjusting the position of a valve, blade, locally controlled pump or actuator, taking a product sample, pressing a button, changing a switch position, moving an object, stopping all operations, returning to an original position or activating a sensor.
According to another aspect, a robotic service platform for management of a robotic team is configured to: selecting a first robot from the plurality of robots to perform robot responsibilities; selecting a first data adapter of a first robot type corresponding to the first robot from a plurality of data adapters, the first data adapter configured to convert responsibility feedback collected by the first robot into a generic data format; selecting a first control adapter corresponding to a first robot type from a plurality of control adapters, the first control adapter configured to convert generic role control information into first robot role control information compatible with the first robot; transmitting first robot role control information to the first robot via a first control adapter, the first control adapter having converted the first generic robot role control information into the transmitted first robot role control information; and receiving, via the first data adapter, responsibility feedback collected by the first robot.
The robotic service platform may be further configured to: selecting a second robot from the plurality of robots to perform robot responsibilities; selecting a second data adapter corresponding to a second machine type from the plurality of data adapters, the second data adapter configured to convert responsibility feedback collected by the second robot into a generic data format, wherein the second data adapter is different from the first data adapter; selecting a second control adapter corresponding to a second robot type from the plurality of control adapters, the second control adapter configured to convert the generic role control information into second robot role control information compatible with the second robot, wherein the second control adapter is different from the first control adapter; transmitting second robot role control information to the second robot via a second control adapter, the second control adapter having converted the first generic robot role control information into the transmitted second robot role control information; and receiving, via the second data adapter, responsibility feedback collected by the second robot.
The robot service platform may consider the first robot and the second robot to perform at least part of the selected robot responsibilities and select the second robot to perform at least part of the robot responsibilities in place of the first robot.
The robot service platform may consider the first robot and the second robot to perform at least a portion of the selected robot responsibilities and select the second robot to perform at least a portion of the robot responsibilities in cooperation with the first robot.
The robot service platform may select the first robot to perform the robot responsibilities based on the capabilities of the first robot and the characteristics of the robot responsibilities.
The robot service platform may select the second robot to perform the robot responsibilities based on the capabilities of the second robot and the characteristics of the robot responsibilities.
The received responsibility feedback collected by the first robot may include one or more of the following information: information that the selected robot responsibilities are fully completed, information that the selected robot responsibilities are partially completed, information that the selected robot responsibilities are not initiated, health information about the first robot, status information about the first robot, capability information about the first robot, photographs, videos, environmental data, sensor readings, movement data about the first robot, and position data about the first robot.
The first robot responsibility control information transmitted to the first robot may include one or more of a multi-step responsibility instruction or a separate operation instruction including one or more of: navigating to a destination, taking a photo, recording video, recording sound, making an environmental measurement, making a temperature measurement of a substance, making a temperature measurement of air, making a humidity measurement, determining an instrument reading, measuring the presence or concentration of a gas or chemical, emitting light of a specific wavelength, intensity and/or emission pattern, emitting sound of a specific tone, intensity and/or emission pattern, emitting a radio frequency homing beacon, transmitting stored data via a radio frequency or wireless data network, connecting to a power source, connecting to a radio frequency or wireless data network, connecting to a data network port, modifying settings of a valve or other control, adjusting the position of a valve, blade, locally controlled pump or actuator, taking a product sample, pressing a button, changing a switch position, moving an object, stopping all operations, returning to home or activating a sensor.
According to another aspect, an adapter for managing a robot fleet comprises: a data adapter corresponding to the first robot type, the data adapter configured to convert data collected by the first robot into a generic data format compatible with the robot service platform; and a control adapter corresponding to the first robot type, the control adapter configured to convert the generic role control information into robot role control information compatible with the first robot.
The responsibility feedback collected by the first robot may include one or more of the following information: information that robot responsibilities are fully completed, information that robot responsibilities are partially completed, information that robot responsibilities are not initiated, health information about the first robot, status information about the first robot, capability information about the first robot, photographs, videos, environmental data, sensor readings, movement data about the robot, and position data about the robot.
The robot responsibility control information may include one or more of a multi-step responsibility instruction or a separate operation instruction including one or more of: navigating to a destination, taking a photograph, recording video, recording sound, measuring environment, measuring temperature of a substance, measuring temperature of air, measuring humidity, determining instrument readings, measuring presence or concentration of a gas or chemical, emitting light of a specific wavelength, intensity and/or emission pattern, emitting sound of a specific tone, intensity and/or emission pattern, emitting a radio frequency homing beacon, transmitting stored data via a radio frequency or wireless data network, connecting to a power source, connecting to a radio frequency or wireless data network, connecting to a data network port, modifying settings of a valve or other control, adjusting the position of a valve, blade, locally controlled pump or actuator, taking a product sample, pressing a button, changing a switch position, moving an object, stopping all operations, returning to an original position or activating a sensor.
The adapter may be configured to communicate with a particular type of robot or a robot produced by a single manufacturer.
The adapter may be configured to communicate with multiple types of robots or robots produced by multiple manufacturers.
According to one aspect, a computer-implemented method of managing a fleet of robots includes: acquiring characteristics of a plurality of robot responsibilities; obtaining the capabilities of a plurality of robots of a robot team; selecting a robot from the plurality of robots to perform a selected robot role of the plurality of robot roles based on the obtained capabilities of the selected robot and the obtained characteristics of the selected robot role; transmitting robot role control information of the selected robot role to the selected robot; receiving responsibility feedback from the selected robot for the selected robot responsibility; storing the received responsibility feedback to a database; transmitting the received responsibility feedback to an information analysis module; receiving an analysis result from the information analysis module; and transmitting the analysis result to the operation management system.
The robot responsibility control information for the selected robot responsibility may be sent to the selected robot via a control adapter configured to convert the generic responsibility control information into robot responsibility control information.
Responsibility feedback for the selected robot responsibility and analysis results of the responsibility feedback may be received from the selected robot via a data adapter configured to convert the responsibility feedback collected by the selected robot into a generic data format.
The information analysis module may perform data aggregation and analysis of the responsibility feedback.
The data aggregation and analysis of the responsibility feedback may include one or more of processing the photo information, determining instrument readings from the photo information, comparing sensor readings to previous or expected sensor readings, and performing statistical analysis on the responsibility feedback.
The data aggregation and analysis of the responsibility feedback may be performed using machine learning or artificial intelligence analysis of the responsibility feedback from the previous robot responsibility or analysis results of the responsibility feedback from the previous robot responsibility.
The method may further comprise: selecting a second robot from the plurality of robots to perform the selected robot responsibilities in cooperation with the robot; and receiving second responsibilities feedback for the selected robot responsibilities and second analysis results of the responsibilities feedback from the second robot, wherein the second responsibilities feedback and second analysis results of the responsibilities feedback for the selected robot responsibilities are receivable from the second robot via a second data adapter configured to convert second responsibilities feedback collected by the second robot into a generic data format.
According to one aspect, a computer-implemented method for management of a robotic team includes: acquiring characteristics of a plurality of robot responsibilities; obtaining the capability of a plurality of robots; selecting a robot from the plurality of robots to perform a selected robot role of the plurality of robot roles based on the obtained capabilities of the selected robot and the obtained characteristics of the selected robot role; transmitting robot role control information of the selected robot role to the selected robot; receiving responsibility feedback for the selected robot responsibility from the selected robot and analyzing the responsibility feedback; and transmitting the analysis result to the operation management system.
The analysis of the responsibility feedback may be data aggregation and analysis of the responsibility feedback performed by the robot.
The data aggregation and analysis of the responsibility feedback may be performed using machine learning or artificial intelligence analysis of the responsibility feedback from the previous robot responsibility or analysis results of the responsibility feedback from the previous robot responsibility.
The robot responsibility control information for the selected robot responsibility may be sent to the selected robot via a computer network, which is one of a global network, a virtual private network, or a local network.
The robot role control information for the selected robot role may be transmitted to the selected robot via the server.
The method may further include processing the robot responsibility control information for the selected robot responsibility through a first robot interface corresponding to the first robot type before transmitting the robot responsibility control information for the selected robot responsibility to the selected robot.
The method may further include processing, by the first robot control adapter corresponding to the first robot type configured to convert the generic robot responsibility control information into robot responsibility control information, the robot responsibility control information of the selected robot responsibility prior to processing the robot responsibility control information of the selected robot responsibility by the first robot interface.
According to another aspect, a system for management of a robotic team includes: a plurality of robots; and a robot service platform, the robot service platform comprising: a data storage device storing instructions for managing a robot fleet in an electronic storage medium; and a processor configured to execute the instructions to perform a method comprising: acquiring characteristics of a plurality of robot responsibilities; obtaining the capability of a plurality of robots; selecting a robot from the plurality of robots to perform a selected robot role of the plurality of robot roles based on the obtained capabilities of the selected robot and the obtained characteristics of the selected robot role; transmitting responsibility control information of the selected robot responsibility to the selected robot; receiving responsibility feedback from the selected robot for the selected robot responsibility; storing the received responsibility feedback to a database; transmitting the received responsibility feedback to an information analysis module; receiving an analysis result from the information analysis module; and transmitting the analysis result to the operation management system.
Storing the received responsibility feedback to the database may include one or more of: providing the received responsibility feedback to a Machine Learning (ML) and Artificial Intelligence (AI) module for further analyzing the responsibility feedback from the previous robot responsibility or the analysis of the responsibility feedback from the previous robot responsibility; providing the received responsibility feedback to the report generator; or store a record of responsibilities and received responsibilities feedback to a responsibilities log.
The responsibility feedback may be received from the selected robot via a data adapter configured to convert responsibility feedback collected by the selected robot into a generic data format compatible with the robot services platform.
The system may also include processing of the responsibility feedback by the robot interface, and thereafter processing by the data adapter.
The responsibility feedback for the selected robot responsibility may include one or more of the following information: information that the selected robot responsibilities are fully completed, information that the selected robot responsibilities are partially completed, information that the selected robot responsibilities are not initiated, health information about the selected robot, status information about the selected robot, capability information about the selected robot, photographs, videos, environmental data, sensor readings, movement data about the selected robot, and location data about the selected robot.
Responsibility feedback from the information analysis module and the analysis results may be provided to an end-user application, which is one of an operations management application, a maintenance application, a field check application, or a workflow management application.
According to another aspect, a computer-implemented method of managing a fleet of robots includes: receiving one or more responsibilities, each responsibilities from the one or more responsibilities including a responsibilities summary; determining a robot associated with the system; determining at least one capability of each robot associated with the system; assigning one or more responsibilities to one or more of the robots associated with the system based on the at least one capability; and generating a responsibility schedule based on the one or more responsibilities assigned to the one or more robots.
The at least one capability may be stored in a memory of the robot, wherein determining the at least one capability of the robot may include transmitting the at least one capability from the robot to a memory or processor of the system, and wherein the at least one capability may include a capability of the robot to perform a task, an availability of the robot, a battery level of the robot, one or more tools associated with the robot, or one or more materials of the robot.
Each responsibility summary may include at least one factor related to information for performing the corresponding responsibility, and wherein if at least one capability of the robot is capable of satisfying at least a portion of the at least one factor, the processor of the system may assign a robot from the one or more robots to a responsibility from the one or more responsibilities such that the robot may perform at least a portion of the corresponding responsibility.
The method may further include determining, by a processor of the system, at least one environmental data of the facility, wherein one or more responsibilities may also be assigned to one or more robots based on the at least one environmental data, wherein determining the at least one environmental data includes the processor of the system receiving data from at least one or more sensors or from a third party connected to the system via a wired or wireless connection.
The at least one environmental data may include at least one of pressure, temperature, humidity, weather conditions, noise level, noise type, vibration, presence of a substance, or level of a substance.
The one or more responsibilities may include a plurality of responsibilities that can be assigned to a plurality of robots from among the one or more robots, wherein each responsibilities profile may include one or more priorities, each priority having a desired level of importance to the responsibilities, and wherein assigning the one or more responsibilities may further include assigning the one or more responsibilities based on the one or more priorities.
The one or more priorities may include one or more of a quality of execution of each of the responsibilities in the responsibilities summary, a cost of each of the responsibilities in the responsibilities summary, and energy to be used in each of the responsibilities in the responsibilities summary, where the one or more priorities are stored in a memory of the system and transmitted to a processor of the system to determine a priority responsibilities.
The method may further comprise: determining, by a processor of the system, a plurality of responsibility schedules, wherein the plurality of responsibilities are assigned by the processor of the system to different ones of the plurality of robots in each of the plurality of responsibility schedules; determining, by a processor of the system, a priority responsibility schedule from the plurality of responsibility schedules based on the one or more priorities; and initiating, by a processor of the system or by a processor of each of the plurality of robots, a plurality of responsibilities of the priority responsibilities schedule.
The method may further comprise: receiving, by one or more of a processor of the system or a processor of the robot, feedback from a robot of the one or more robots, wherein the feedback is transmitted via a wired or wireless connection; and modifying, by a processor of the system, a role summary of the first role from the one or more roles based on the feedback, wherein the feedback is received by the processor of the first robot scheduled to perform the first role, and wherein modifying the role summary prevents the first robot from completing the first role.
According to another aspect, a computer-implemented method of managing a fleet of robots includes: receiving a first responsibility; assigning a first responsibility to the first robot; determining a role schedule based on assigning the first role to the first robot; instructing the first robot to perform the first role based on the role schedule; receiving feedback from the first robot; and assigning a second responsibility to a second robot different from the first robot based on the feedback.
The first responsibility may be defined by a first responsibility summary stored in one or more of a memory of the first robot or a memory of the second robot, and wherein the assigning step may further comprise: updating, by a processor of the system or a processor of the first robot, data in the first responsibility summary based on the feedback; and generating, by the processor of the system or the processor of the first robot, a second responsibility summary based on the updated first responsibility summary, wherein the second responsibility may be defined by the second responsibility summary, wherein the second robot may be capable of performing at least a portion of the second responsibility, and wherein the first robot may not be capable of performing at least a portion of the second responsibility.
The method may further include comparing, by a processor of the system or a processor of the second robot, the one or more capabilities of the second robot to the second responsibility summary, wherein the second robot may be able to perform at least the portion of the second responsibility based on the one or more capabilities of the second robot matching factors for performing at least the portion of the second responsibility.
The feedback may indicate a change in at least one of a factor for performing the first responsibility or environmental data in an environment in which the robotic team is configured to operate, and wherein generating the second responsibility summary may include changing the at least one factor for performing the first responsibility based on the feedback.
The environmental data may include one or more of pressure, temperature, gas type, presence or absence of stairs, humidity, weather conditions, noise level, noise type, vibration, or substances.
A third responsibility, different from the first responsibility and the second responsibility, may be assigned to the first robot or the second robot by a processor of the system, and wherein the responsibility schedule may be updated by the processor of the system based on assigning the second responsibility and the third responsibility.
The method may further comprise: determining, by a processor of the system, whether the second robot is capable of performing the second duties; and assigning the second responsibility to at least a third robot different from the first robot and the second robot if the second robot's ability prevents the second robot from completing the second responsibility.
According to another aspect, a computer-implemented method of managing a fleet of robots includes: receiving one or more responsibilities, each responsibilities comprising a responsibilities summary, wherein the responsibilities summary comprises environmental data from one or more sensors; for each of the one or more responsibilities, determining, based on a summary of responsibilities included in the responsibilities, one or more robots capable of performing the one or more responsibilities; scheduling one or more responsibilities to be performed by one or more robots as a primary schedule; receiving feedback indicative of a change in the environmental data, the feedback obtained by a robot of the one or more robots performing one or more of the scheduled responsibilities; and scheduling the one or more responsibilities to the one or more robots as a secondary schedule based on the feedback.
The method may further include determining, by a processor of the system, a robot capability of the first robot if the robot capability of the first robot meets a feature in a role profile defining the first role, wherein the robot capability of the first robot of the one or more robots scheduled to perform the first role may not be suitable for operation in an environment defined by a change in environment data, and the method may further include assigning, by the processor of the system, the first role to the second robot.
The processor of the first robot may obtain feedback, and the feedback may be obtained separately from data collected based on responsibilities assigned to the first robot.
The one or more responsibilities of the primary schedule may include: assigning, by a processor of the system, a first responsibility from the one or more responsibilities to the first robot, wherein the scheduling of the feedback-based secondary schedule may further include determining, by the processor of the system, a second robot capable of performing a portion of the first responsibility; and assigning a second robot to the first responsibility, wherein the second robot is configured to assist the first robot in completing the responsibility based on the change in the environmental data.
According to another aspect, a computer-implemented method of managing a fleet of robots includes: receiving a first task; determining at least one factor of the first task indicative of information related to completing the first task; receiving data indicating information related to an environment in which a first task is to be completed; determining at least one responsibility capability for completing the responsibility including the first task; wherein the at least one responsibility capability is based on one or more of the at least one factor or data; and creating a responsibility summary based on the first responsibility and one or more of the at least one factor, the data, or the at least one responsibility capability.
The responsibilities may include a plurality of tasks having a first task, and wherein the at least one factor may include an order in which the plurality of tasks are to be performed.
The method may further comprise: receiving, by a processor of a system in which the robot team is configured to operate, a second task different from the first task; determining, by a processor of the system, at least one factor of the second task, wherein the at least one factor associated with the first task and the second task may indicate an order in which the first task is completed relative to the second task; a second responsibility summary is created by the processor of the system based on the second task and at least one factor of the second task.
The method may further comprise: receiving, by a processor of the system, capabilities of one or more robots from the one or more robots; determining, by a processor of the system, whether the capabilities of the one or more robots enable the one or more robots to perform the first task and the second task in an order indicated by the factors of each of the first task and the second task; and if the one or more robots are capable of performing the first task and the second task in the specified order, assigning the first task and the second task to the one or more robots by a processor of the system.
The responsibilities may include a plurality of tasks, and the responsibilities summary includes a sub-responsibilities summary for each of the plurality of tasks.
The method may further comprise: assigning responsibilities to the first robot by a processor of a system in which the robot team is configured to operate; receiving feedback by one or more of a processor of the system or a processor from a robot in the robot team, wherein the feedback may include information about objects along a route for accessing the step location in responsibility, the location of the objects preventing the robot from accessing the location via the route; receiving, by a processor of the system, map data to determine one or more routes for accessing step locations of the task; and altering, by a processor of the system, data in the responsibility summary, wherein altering the data in the responsibility summary causes the responsibilities assigned to the first robot to be rescheduled such that the first robot performs responsibilities in an order relative to all other responsibilities assigned to the first robot based on the altered data.
The method may further comprise: determining, by a processor of a system in which the robotic team is configured to operate, the robotic team; receiving, by a processor of the system, for each robot in the robot fleet, robot capabilities defining features of the respective robot, wherein the robot capabilities are transferred from the respective robot or a memory storing the robot fleet capabilities to the processor of the system; determining, by a processor of the system, whether at least one responsibility capability of the responsibility matches a robot capability of a robot selected from the robot team; and assigning, by the processor of the system, responsibilities to the selected robot if the responsibilities capabilities of the at least one responsibilities match the robot capabilities of the selected robot.
The method may further comprise: receiving, by a processor of a system in which the robot team is configured to operate, feedback; and updating, by a processor of the system, the responsibility summary based on the feedback, wherein the feedback may indicate a change in one or more of the at least one factor, the data, or the at least one responsibility capability.
The responsibility overview may be defined in a system computer code language specific to the system used to manage the robot fleet.
The method may further comprise: determining, by a processor of a system in which the robotic team is configured to operate, the robotic team; determining, by a processor of the system, a robot computer code language for each robot in the robot fleet; and for each robot that does not support the system computer code language, converting, by the processor of the system, the responsibility summary into the robot computer code language.
According to one aspect, a computer-implemented method of managing a fleet of robots includes: receiving a first responsibility, the first responsibility including factors related to the execution of the first responsibility; determining data related to an environment in which the first responsibility is performed; instructing one or more robots in the robot fleet to perform a first responsibility; receiving feedback during performance of the first responsibility, wherein the feedback includes information related to the environment; and revising the first responsibility based on the feedback, wherein revising the first responsibility causes at least one robot from the one or more robots to alter the scheduled operation.
The method may further include defining, by a processor of a system in which the robotic team is configured to operate, a second responsibility based on the feedback.
The feedback may include information regarding objects along a route for accessing a location of a step in responsibility, the positioning of the objects preventing the robot from accessing the location via the route, the method may further include receiving, by a processor of a system in which the robot team is configured to operate, map data to determine one or more routes for accessing the location of the task, wherein changing the scheduled operation includes rescheduling responsibility assigned to the robot, including a first responsibility, such that the robot performs the responsibility in a minimum period of time.
The environmental feedback may include information about an object positioned along a route for accessing the step location in the first responsibility, the object positioned such that the object prevents the robot from accessing the location via the route, the method may further include: revising, by a processor of a system in which the robotic team is configured to operate, the responsibility schedule to remove the first responsibility from the responsibility schedule; receiving, by a processor of the system, at least one robot capability from each of the one or more robots; determining, by a processor of the system, a second robot from the one or more robots, the second robot having a robot capability that enables the second robot to (1) operate in an environment of responsibilities and (2) perform a first responsibilities to complete; and reassigning, by the processor of the system, the first responsibility removed from the responsibility schedule to the second robot.
Revising the first responsibility may include transmitting data between one or more of a processor of a system in which the robot team is configured to operate, a processor of the first robot, and a processor of the second robot; the second robot is determined by a processor of the system to perform the first responsibility, and wherein both the first robot and the second robot may be required to perform the first responsibility.
One of the first robot or the second robot may enable the other of the first robot or the second robot to access the step location of the first responsibility, and the other of the first robot or the second robot may be configured to perform the step.
According to another aspect, a computer-implemented method of managing a fleet of robots includes: receiving a first responsibility defined by a plurality of tasks to be performed during the first responsibility; determining a first factor of the first responsibility, wherein the first factor comprises a time sequence in which a plurality of tasks are to be performed; creating an initial responsibility summary based on the first factor; receiving feedback from a system, wherein the system comprises a robot team; and modifying the initial responsibility summary based on feedback, wherein the feedback comprises a change in a time sequence of the plurality of tasks.
The feedback may be received by a robot in the robot team scheduled to perform the first responsibility, and the method may further include causing, by a processor of the robot, the robot to perform the plurality of tasks in a time order different from the time order in the initial responsibility summary.
The method may further comprise: assigning, by a processor of a system in which the robotic team is configured to operate, a first responsibility to a first robot from the robotic team, wherein the feedback further includes a change in a second factor related to the plurality of tasks; receiving, by a processor of the system, a capability of the first robot from the first robot; and if the capabilities of the first robot do not enable the first robot to perform the plurality of tasks of the first responsibility based on the updated second factor, assigning, by the processor of the system, the first responsibility to a second robot different from the first robot, the second robot capable of performing the plurality of tasks of the first responsibility based on the updated information.
The first factor may include information related to a tool pack from a robot in a robot team that can be used to perform a plurality of tasks, and wherein the feedback may include a change in the tool pack.
According to one aspect, a computer-implemented method of managing a fleet of robots includes: receiving a first responsibility summary including information related to the execution of the task; determining capabilities of one or more robots in a robot team; assigning a first responsibility defined by the first responsibility summary to a first robot of the one or more robots based on the capabilities of the one or more robots and the information in the responsibility summary; instruct the first robot to perform a task based on the information in the responsibility summary; receiving data obtained by a first robot performing a task; and defining a second responsibility summary based on the received data, wherein the second responsibility summary includes information related to the execution of the task.
The information related to the execution of the task may include historical data stored in a memory of a system configured to manage the robotic team related to past responsibilities for executing the task.
The historical data may include environmental data received by a processor of a system configured to manage the robotic fleet, which may be transmitted from at least one or more sensors or from a third party connected to the system via a wired or wireless connection.
The environmental data may include a time-varying characteristic, and the method may further include receiving, by a processor of the system, periodic updates related to the environmental data from at least one or more sensors or from a third party.
The environmental data may include values of environmental measurements of the task locations, wherein a range of the values may be determined based on the historical data, and wherein the task may be performed when the measured values of the environmental measurements are within a predetermined range.
The method may further include determining, by a processor of a system configured to manage a robot team, whether the capabilities of the first robot enable the first robot to perform all of the factors of the second responsibility summary, and if the first robot is unable to perform all of the factors of the second responsibility summary, assigning, by the processor of the system, the second responsibility defined by the second responsibility summary to a second robot different from the first robot.
The method may further comprise: receiving, by at least one of a processor of a system configured to manage a robotic fleet or a processor of a robot, feedback from a system in which the robotic fleet is configured to operate; monitoring, by a processor of at least the system or a processor of the robot that received the feedback, a first responsibility; automatically updating the first responsibility summary by a processor of at least the system or a robot receiving the feedback based on a change in a factor in the first responsibility summary due to the received feedback; determining, by a processor of at least the system that received the feedback or a processor of the robot, whether the capabilities of the first robot enable the first robot to perform all factors of the first responsibility based on the updated first responsibility summary; determining, by a processor of at least the system that received the feedback or a processor of the robot, a robot from the plurality of robots having the ability to perform a second responsibility based on the updated responsibility summary if the first robot is unable to perform the first responsibility based on the updated first responsibility summary; and assigning, by the processor of at least the system or the processor of the robot that received the feedback, the second responsibility to the robot that is capable of performing the second responsibility.
The responsibility schedule of the robot capable of performing the second responsibility may be revised by the processor of the at least system receiving the feedback or the processor of the robot based on the allocation of the second responsibility.
The method may further comprise: determining, by at least one of a processor of a system configured to manage a fleet of robots or a processor of a robot, whether the capabilities of the first robot enable the first robot to perform all factors of the second responsibility summary; and assigning, by the processor of the at least system or the processor of the robot that received the feedback, the first robot and the second robot to the second responsibility if the capability of the first robot does not enable the first robot to perform each step of the second responsibility, wherein the second robot is capable of enabling the first robot to complete all steps of the second responsibility.
The second responsibilities may also be assigned to the second robot by at least one of a processor of a system configured to manage a robot team or a processor of a robot, wherein the first robot and the second robot may be configured to coordinate to accomplish the second responsibilities.
According to another aspect, a computer-implemented method of managing a fleet of robots includes: receiving a first responsibility summary, wherein the first responsibility summary includes information related to execution of the task; determining capabilities of one or more robots; assigning tasks to a first robot of the one or more robots based on the capabilities of the one or more robots and the information in the first responsibility summary; instruct the first robot to perform a task based on the information in the responsibility summary; receiving data obtained by a first robot performing a task; defining a second responsibility summary by modifying the first responsibility summary based on the received data; and instructing the first robot to perform the task based on the information in the second responsibility summary.
The method may further comprise: if the first robot is unable to perform the task based on the second responsibility summary, determining, by a processor of a system configured to manage the robot team, if the second robot of the one or more robots is able to perform the task based on the second responsibility summary; and assigning, by the processor of the system, tasks based on the second responsibility profile to the second robot based on the capabilities of the second robot.
The second responsibility may be assigned to the second robot by a processor of the system based on the proximity of the second robot to the task location.
The method may further comprise: if the first robot is unable to perform a portion of the task, determining, by at least one of a processor of a system configured to manage the robot team or a processor of the first robot, whether a second robot from the one or more robots is able to perform the portion of the task; and scheduling, by at least one of a processor of the system or a processor of the first robot, the first robot and the second robot to perform the second responsibility based on the second responsibility summary.
The processor of the first robot may continue to receive data while performing the task based on the second responsibility summary, wherein the method may further comprise: determining, by at least one of a processor of a system configured to manage a fleet of robots or a processor of a first robot, if the first robot is unable to perform a portion of a task based on the received data; defining, by at least one of a processor of the system or a processor of the first robot, a third responsibility summary based on the received data if the first robot is unable to perform the portion of the task; determining, by at least one of a processor of the system or a processor of the first robot, a second robot capable of performing a task based on the third responsibility summary; and instructing, by at least one of a processor of the system or a processor of the first robot, the second robot to perform the task based on the third responsibility profile.
The third responsibility summary may include historical data based on past execution of the task.
According to another aspect, a computer-implemented method of managing a fleet of robots includes: receiving a first responsibility summary; determining capabilities of one or more robots in a robot team; assigning tasks to a first robot from the one or more robots based on the capabilities of the one or more robots and the first responsibility profile; instructing the first robot to perform a task based on the information in the responsibility summary; receiving data obtained by the second robot performing the task, wherein the data is historical data related to the task; defining a second responsibility summary by modifying the first responsibility summary based on the history data; and instructing the first robot to perform the task based on the information in the second responsibility summary.
The tasks may include instructing, by a processor of a system configured to manage a fleet of robots, a third robot to change a responsibility schedule based on instructions from a central database.
The method may include: receiving, by a processor of a system configured to manage a fleet of robots, environmental data; and modifying, by a processor of the system, the first responsibility summary based on the environment data prior to instructing the first robot to perform the task.
If modifying the first responsibility summary based on the environment data results in the first robot being unable to perform the task, the method may further include: if the capabilities of the second robot enable the second robot to navigate the environment defined by the environment data, determining, by a processor of a system configured to manage the robot team, if the second robot is capable of performing tasks based on the environment data; and if the second robot is capable of performing the task based on the environmental data, instructing, by a processor of the system, the second robot to perform the task.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and together with the description, serve to explain the principles of the disclosed embodiments.
FIG. 1 illustrates an exemplary mobile robot in an industrial facility environment 100;
FIG. 2 illustrates tasks and components of an operations management system for an industrial facility;
FIG. 3 illustrates a generalized model of robot activity within an operations management system;
FIG. 4 illustrates components of a robotic service platform within an operations management system in accordance with one or more embodiments;
FIG. 5 illustrates a detailed architecture of a multi-vendor robotic service platform in accordance with one or more embodiments;
FIG. 6 further illustrates a detailed architecture of a multi-vendor robotic service platform and associated components in accordance with one or more embodiments;
FIG. 7 illustrates a subsystem of an operations management system including a multi-vendor robotic service platform in accordance with one or more embodiments;
FIG. 8 illustrates an environment for robot control and communication within a multi-vendor robot service platform in accordance with one or more embodiments;
FIG. 9 illustrates a process of planning, distributing, and performing robotic tasks within a multi-vendor robotic service platform in accordance with one or more embodiments;
FIG. 10 illustrates a flow chart of a generalized method for management of a robotic fleet according to one example;
FIG. 11 illustrates a flow chart of a generalized method for defining responsibilities during management of the robotic team in FIG. 10;
FIG. 12 shows a flow chart of a generalized method for assigning responsibilities during management of the robotic team in FIG. 10;
FIG. 13 shows a flow chart of a generalized method for managing responsibilities during management of the robotic team in FIG. 10;
FIG. 14 illustrates a dashboard for inputting and displaying data, according to one example;
FIG. 15 illustrates an example of the dashboard of FIG. 14, according to one example;
FIG. 16 illustrates a flow chart for assigning responsibilities based on priority according to one example;
FIGS. 17 and 18 illustrate a flow diagram for managing a fleet of robots according to one example;
FIGS. 19 and 20 illustrate a flow chart for managing a fleet of robots according to another example;
FIGS. 21 and 22 illustrate a flow chart for managing a fleet of robots according to yet another example;
FIGS. 23 and 24 illustrate a flow diagram for managing a fleet of robots according to one example;
FIG. 25 illustrates a flow chart for defining responsibilities using machine learning, according to one example;
FIG. 26 illustrates a flow chart for assigning responsibilities using machine learning, according to one example;
FIG. 27 illustrates a flow chart for receiving feedback according to one example;
FIG. 28 illustrates a flow chart for receiving feedback according to another example;
FIG. 29 illustrates a flow chart for reassigning responsibilities according to one example;
FIG. 30 illustrates user input according to one example;
FIG. 31 illustrates a template for defining responsibilities according to one example;
FIGS. 32A and 32B illustrate sensors within the facility of FIG. 1 according to one example;
FIG. 33 illustrates feedback from responsibilities according to one example; and
FIG. 34 illustrates a flow chart for reassigning responsibilities according to one example.
Detailed Description
Various embodiments of the present disclosure relate generally to management of robotic services in, for example, industrial facilities, and more particularly to receiving, defining, distributing, and managing responsibilities performed by a robotic team, managing data collected by such robots, and managing other associated systems.
The terminology used below is to be interpreted in its broadest reasonable manner, even though it is being used in conjunction with a detailed description of certain specific examples of the disclosure. Indeed, certain terms may even be emphasized below; however, any terminology intended to be interpreted in any restricted manner will be overtly and specifically defined as such in this detailed description section.
Industrial facilities such as chemical plants, refineries, power plants, manufacturing plants, etc. can be very large and can involve a large number of simultaneous processes throughout the facility. Some aspects of these processes may be managed remotely by operators in a central control room, but other aspects may require completion of tasks at various locations in the facility. These tasks may include monitoring and adjusting equipment, sensing environmental or other conditions within a facility, performing manufacturing tasks, and so forth. Some of these tasks may require 24 hours of attention per day, and others may require exposure to potentially dangerous conditions or work in confined or difficult to reach spaces. Increasingly, facility operators are turning to the use of autonomous mobile robots to accomplish these tasks. Some of these tasks may be accomplished by specialized sensors or actuators at several locations throughout the plant, but may not be prioritized due to cost, complexity, or other considerations.
System for managing a fleet of robots
Fig. 1 illustrates such an exemplary mobile robot 110 in an industrial facility 120 in an industrial facility environment 100. The operation of the industrial facility 120 may be controlled by one or more human operator operated operation management systems. Fig. 2 illustrates tasks and components of an operations management system 200 for an industrial facility 120.
The operation of the industrial facility 120 may include tasks and components related to the operation of the facility, monitoring of safety or other conditions within the facility, and maintenance of the facility. The overall operation may be managed by enterprise resource management system 202, which may control tasks such as, for example, production planning 204, production accounting 206, maintenance accounting 220, and turnaround maintenance and long-term maintenance planning 222.
Tasks for the operation and maintenance of the industrial facility 120 may be performed in an operations office or Central Control Room (CCR) 234 or in a field 236, such as within a production area of the industrial facility 120 itself.
Operation-related tasks performed within office/CCR 234 may include, for example, production scheduling 208, data checksums production results reporting 210, production operations management and reporting 212, data historian 214, panel operations 216, and process control 218.
Maintenance-related tasks performed within the office/CCR 234 may include, for example, maintenance and reliability management and results reporting 224, annual maintenance planning and scheduling 226, maintenance operations management and reporting 228, and third party contractor management 230.
The operation-related tasks performed in the field 236 may include, for example, human-performed operation tasks 238 and robot-performed operation tasks 240. Maintenance-related tasks performed in the field 236 may include, for example, human-performed maintenance tasks 242 and robotic-performed maintenance tasks 244.
The robot-performed operational tasks 240 and the robot-performed maintenance tasks 244, as well as other robot-performed tasks, may be managed by the robot management system 232. Exemplary embodiments of one or more such robotic management systems are discussed in more detail below.
Robots acting within the operations management system may require additional information and metadata about each of the robots and robot activities within the system, including robot management, tracking, and analysis of the robots and their activities. For example, when a plurality of robots cooperate with respect to an activity or when activities of a plurality of robots may collide, the robots may need information about other robots and activities thereof. FIG. 3 illustrates a generalized model 300 of robotic activity within an operations management system based on the ISA-95 standard (https:// www.isa.org/standards-and-publications/ISA-standards/ISA-standards-schemas/ISA 95) for "enterprise control system integration".
Management of robots 110 running within industrial facility 120, including operation tasks 240 performed by robots and maintenance tasks 244 performed by robots, may involve activities performed by robots 110, data related to robots 110 and performed tasks, operations performed by robot management system 232, and communication between robot management system 232 and robots 110. Such management may be structured according to a model of the robotic activity 300.
The data related to the robots 110 may include, for example, a robot definition 305 for each robot 110, a robot capability 310 for each robot 110, a robot schedule 315 for each robot 110, and a robot performance metric 320 for each robot 110.
Operations performed by the robot management system 232 may include, for example, robot resource management 330, robot definition management 350, detailed robot dispatch 325, robot dispatch 345, robot execution management 360, robot tracking 335, robot data collection 355, and robot human energy analysis 340. In systems that utilize robots provided by multiple robot suppliers, these operations may reflect differences between robots provided by different suppliers. For example, each robot vendor may make data about each robot available in a different format specific to the robot or vendor. To address this issue, the robotic management system 232 may normalize the data by pulling and interpreting a subset of the data that may be relevant to a human operator. For example, two robots from different vendors may each have an error in responsibilities involving blocking, but the robots may format and present the error in different ways depending on their platforms. The robot management system 232 may properly interpret the different information provided by robots from multiple vendors via an adapter, such as the robot-specific adapter 644 discussed below with respect to fig. 6.
During robotic resource management 330, robotic management system 232 may update or reference robotic capability information 310 to maintain information about available robotic resources and make those resources available for the tasks of robotic allocation. For example, during performance of a duty or task, the capabilities of the robot may change temporarily, such as exhaustion of a robot battery, damage or malfunction of one or more tools or sensors attached to the robot, removal or replacement of one or more tools or sensors attached to the robot, or other changes in the capabilities of the robot. Furthermore, the robot may become unable to move from the current location due to obstructions, environmental conditions (e.g., the robot encountering heavy rain without dampproof measures), damage to the robot rails, legs, or other propulsion devices, loss of connectivity with the robot management system 232 or other factory systems, and so forth. Furthermore, operation of a robot under extreme heat conditions, such as a desert environment, may overheat components of the robot. Similarly, operation of a robot in extreme cold conditions may affect the operation of joints or other moving parts. Such temperature extremes may shorten the life of the robot battery. The robot management system 232 may monitor the conditions of the robots and maintain the robot capability information 310 to reflect such changes and decide how to schedule or reschedule the robot fleet for optimal performance. In some embodiments, an operator or technician may be responsible for updating the robot capability information 310 of the robot management system 232. Each robot may also (or alternatively) report capability information, including payload status, to the robot management system 232 prior to performing duties. If the robot management system 232 determines that the payload capacity of the robot is insufficient to perform the responsibilities, the robot management system 232 may select a different robot for the responsibilities and update the particular robot capacity. The robot management system 232 may check future scheduled responsibilities against updated robot capabilities. If no other capable robots are available at the time of the job to be run, the robot payload may be changed, replaced, and/or updated to allow the job to be run. Such changes, replacements, or updates may affect the availability of the robot during initial intent of responsibility or scheduled operation. The responsibilities may indicate a period of time or other cyclical environment in which the responsibilities may be performed. In this case, the responsibilities can be rescheduled to run at a later time under conditions appropriate for the responsibilities. To avoid rescheduling, the robot management system 232 may verify the robot status (the time required will depend on facilities and personnel) before responsibility begins. The verification may require pre-inspection, which may also include inspection of cameras, microphones, sensors, payloads, and/or other capabilities of the robot. This can be done for all robots, including robots with variable and non-variable sensors.
During robot definition management 350, robot management system 232 may maintain robot definition information and may transmit robot-specific production rules 365 to robot 110. For example, in addition to the temporary changes in the capabilities of the robots discussed above, a particular robot may be subject to permanent changes in capabilities and availability, such as removal or installation of sensors or tools, or repositioning of the robot in another area of the facility, so that the robot may be used to perform responsibilities or tasks in a different limited portion of the facility. Modifications to the robot may change other capabilities of the robot, such as, for example, changes in total weight of the robot may result in changes in battery life, speed of movement, or maximum distance traveled by the robot. Such changes may be reflected in updated robot definition information. During the detailed robot schedule 325, the robot management system 232 may determine and update information about the overall robot schedule. Robot scheduling will be discussed in more detail below. During robot dispatch 345, robot management system 232 may release the selected robot 110 to perform the robot assigned task. Robot dispatch will be discussed in more detail below. During robot execution management 360, robot management system 232 may transmit robot commands 370 to robot 110 and may receive responses 375 from robot 110. Robot activity command and data collection will be discussed in more detail below. During robot tracking 335, robot management system 232 may maintain information about robot position, status, and performance during robot assigned activities or during idle periods. Robot tracking will be discussed in more detail below. During robot data collection 355, robot-specific data 380 may be received by robot management system 232 from robot 110. Robot data collection will be discussed in more detail below.
During robot performance analysis 340, robot management system 232 may perform an analysis of robot performance upon completion of the tasks assigned by the robot. The analysis may include updates to the information of the robot definition 305 and the robot capabilities 310 to improve performance of the tasks assigned by the robot. For example, the time it takes for a robot to complete a responsibility or task, or a portion of a responsibility or task (such as traveling from one place to another, traveling stairs or ramps) may be measured. Further, the accuracy of measurements and data (e.g., photographs or other data) collected by the robot may be monitored to determine whether the robot generally returns accurate measurements and data. Such information may be considered when planning responsibilities or tasks for the robot in the future, or may be used to determine that the robot requires maintenance. Furthermore, tasks assigned to robots may be modified after scheduling, during execution, or as a result of execution. The environment around the responsibilities may vary and therefore such modifications are required. For example, as a result of opening the door, a change in temperature or air composition may be detected such that different sensors are required to provide accurate and executable feedback regarding the environment surrounding the duty. As another example, the measured temperature may be lower than expected, such that opening the manual valve may require a higher torque than the initial robot assigned to the task may apply.
As described above, the operations management system may interact with a robot management system, such as a Robot Service Platform (RSP), to manage the robot 110 and tasks assigned by the robot. Fig. 4 illustrates components of a Robotic Service Platform (RSP) 400 within or otherwise in communication with an operations management system in accordance with one or more embodiments.
Activities of robots 110 within operations management system 200 may involve various industrial operations 410 including, for example, pipeline inspection, pressure vessel inspection, supervision, first responders, emissions detection, tank inspection, and subsea platform monitoring. The identified operation 410 is merely exemplary; principles of the present disclosure may relate to any facility adapted for specialized services to be performed by a robotic team in a dynamic environment. Such services may be repetitive and/or dangerous to human operators who may have previously relied on human ability to recognize and adjust dynamic environments to play such roles. Such dynamic environments may require a robot service platform to identify and adjust the activities of individual robots or robot teams to meet changing environments or to operate in environments that may not be designed for robotic operation while managing robot teams with varying technologies and operational capabilities. For example, suitable facilities may include chemical and pharmaceutical manufacturing, mining, food and beverage production, water and wastewater treatment, and the like. To support these operations, the RSP 400 may receive commands from an external control system 420, such as an operations management system 200 or an Industrial Automation (IA) system, and may include, for example, modules for interacting with the industrial automation system 430, for data aggregation and analysis 440, for security and security 460, for coordinating operations, collecting data and controlling robots 470, and for robot fleet management 480. Access to these modules may be provided to a human operator through an integrated human machine interface HMI 450. The industrial automation system interface 430 can provide services for industrial process automation, possibly in combination with external Industrial Automation (IA) systems. The data aggregation and analysis module 440 may receive data from tasks assigned by the robot and may support the aggregation and analysis of the data. Data aggregation and analysis will be discussed in more detail below. The security and security module 460 may ensure facility operation and security of the respective robots 110. The reconciliation-collection-control module 470 may provide overall management of robot-assigned activities, reconciliation of robot-assigned activities, and collection of data returned from robot-assigned activities. The robot team management module 480 may provide management of the robot team 110 including, for example, maintaining information regarding the current status, location, availability, and performance of the respective robots 110, coordinating related or potentially conflicting activities of the plurality of robots 110, and coordinating required maintenance and/or removal of the respective robots 110. Additional details regarding robot management are provided herein.
Managing robot activity in a large or complex industrial facility may require coordination of robots provided by multiple suppliers, each with different and possibly incompatible command and control or data management and communication protocols. Thus, as the number of robots and tasks allocated by the robots increases, it may become more and more complex to integrate robot fleets into industrial facilities. Fig. 5 illustrates a detailed architecture 500 of a multi-vendor robotic service platform 510 for addressing these issues in accordance with one or more embodiments.
The multi-vendor robotic service platform 510 may interact with external control systems, such as an Operations Management (OM) system 200 or an Industrial Automation (IA) system 505, through an IA system integrated Application Programming Interface (API) 515. The multi-vendor robotic service platform 510 may provide access to end-user applications 530, such as operational management related to overall operation of the facility, maintenance management related to maintenance activities of the facility, field inspections related to direct inspections of portions of the physical facility, and workflow management related to coordinating multiple human and robotic activities within the facility, to persons managing, controlling, or otherwise supervising the robotic team through an engineering and operational human-machine interface (HMI) 525. The instruction for activity from OM 200 may be a Work Instruction (WI) that may be directly assigned to a person. Such instructions may be in a human-readable format. The multi-vendor robot service platform 510 may convert WI into robot commands. If there are multiple types of robots from multiple vendors, the conversion may be performed differently for each type of robot, as discussed elsewhere in this disclosure. If the activity is performed by a person, WI may be assigned to the operator in the context of shift planning. If activity is to be assigned to a robot, the robot service platform 510 may autonomously convert WI and assign it to one or more robots, as discussed elsewhere in this disclosure. In other embodiments, if an alert generates robot responsibilities, such as from an external application, then action may be taken to ensure that a capable robot is available. For example, if the alarm is a critical alarm, the robotic service platform 510 may determine to suspend the current responsibilities and process the critical alarm responsibilities if no robots or people are available. Alternatively, the operator may manually take over the robot to process the alarm. The external system may send a request for task execution and the robot service platform 510 may reply with robot availability or responsibility start information. The human and robotic tasks may each include the capability requirements of the human or robot assigned to the task, such as the requirement of being able to work at a specified height above the ground, or the capability of using a particular tool. Further, the tasks may have requirements related to the operational status of the facility, such as time of day limits, requirements that the plant is not running when the tasks are performed, or changes in government regulations or other regulations governing the operation of the facility. The robot service platform 510 may separate the request received from the OM system 200 or IA system 505 into a request that may be a robot-compatible task that may be sent to the relevant robot to perform and a request that may be returned to the OM system 200 or IA system 505 to be assigned to the robot-incompatible task of the person.
The robot team management module 545 may interact with various robot platforms 560 and sensors 550 (e.g., environmental sensors) that support interactions with the robot 110. The robotic platform 560 may include a platform for interacting with an open source robotic operating system 555, such as a Robot Operating System (ROS), and vendor-specific (non-open source) robots 565. The robot team management module 545 may also interact directly with ROS robots 555 and non-ROS robots 565, rather than via the robot platform 560. Interactions between the robot team management module 545 and the robot 110 may include, for example, robot commands for responsibility and task completion based on information from the control and coordination function 540. Interactions between the robot team management module 545 and the robot 110 may also include, for example, collection of data and other information from the robot 110 and sensors 550, which may be provided to the data collection and aggregation module 535 for further processing. Such processing may include storing the collected and processed data in data store 520 and/or providing the collected and processed data to end user application 530. The data stored in the data store 520 may also be provided to the (IA) system 505 through the IA system integration API 515. The coordination function 540 may convert end user application functions into robotic system functions. For example, if an end user requests to check a meter, coordination function 540 may translate the request into robotic responsibilities for taking a picture of the meter. For example, the coordination function 540 may determine robotic responsibilities, including detailed steps such as navigating from the current position of the robot to the position of the meter, taking an image of the meter, and returning an image of the meter or analysis of meter readings based on the image. Other requested tasks will translate to other appropriate operations. To support robots from multiple vendors, metering responsibilities may be preconfigured to support multiple robots by the robotic fleet management module 545. For example, detailed operations corresponding to robots from multiple suppliers may be predetermined and stored. This may allow robotic responsibilities based on the loop task request to be quickly dispatched or scheduled based on the loop. The coordination function 540 may also coordinate robotic tasks with operations performed by other people and robotic operators, such as steps taken by operators in a control room, automated steps in an IA platform, and field operations. Each of these components works in a tightly synchronized manner to ensure proper operation of the facility. In conventional facilities, a large portion of this synchronization may be done verbally; in facilities where robot teams are applied, the coordination function 540 may integrate the robots and automation systems into the synchronization. In some cases, full control of the robot at all times may not be possible, and may allow the robot to operate semi-autonomously to complete tasks using any particular capabilities of the robot. In the case where multiple robots are assigned responsibilities, coordination function 540 may assign some responsibilities with an appropriate time delay to avoid collisions between robots operating in the same region.
In facilities with robots from multiple suppliers, additional vendor-specific and robot-type specific capabilities in a multi-vendor Robot Service Platform (RSP) may be utilized, such as robot team management, which may be provided by a robot team management module 545, robot activity control, such as may be provided by a control and coordination function 540, and data analysis, such as may be provided by a data collection and aggregation module 535. Fig. 6 illustrates a detailed architecture 600 of a multi-vendor Robotic Service Platform (RSP) in accordance with one or more embodiments.
The robot services platform 672 may include, for example, external system interfaces 624, navigation control and data services 626, a fleet manager 628, and a robot interface 630, among others.
The external system interface 624 may include, for example, a process control server 632 that may receive information about the processes and tasks to be completed from the process control client 604 under the direction of a human operations manager or operator 602. For example, the received responsibilities and tasks may include activities related to the overall operation of the facility, such as facility inspection, data collection, facility control (e.g., changing control settings), etc., to be performed by a person or robot. The external system interface 624 may also include, for example, a first web server 634, which may provide data reports to the web client 606 for display to a human operator or analyst 608. The external system interface 624 may also include, for example, a second web server 636 that may provide data feeds to the historical data store 612.
The historical data store 612 may provide data to the report generator 610, which may provide additional reports to the human operator or analyst 608. Historical data store 612 may also provide data to archive module 618 for storage in database 620 within robotic service platform 672. Database 620 may provide the data to Machine Learning (ML) and Artificial Intelligence (AI) module 614 for further analysis. The analysis results from ML and AI modules 614 may be stored in database 620. Database 620 may also provide data to historical data store 612 for access by report generator 610. Report generator 610, history data store 612, and ML and AI module 614 may be cloud-based or may operate separately from robotic service platform 672.
Navigation control and data services 626 may include, for example, a robot-specific adapter 644 that may provide platform-independent command, control, and data services for robots 110A, 110B, and 110C. The robot-specific adapters 644 may include the following: an adapter for transmitting data collected by the robot 110 (e.g., such as photographs, videos, environmental data, sensor readings, etc.); an adapter for transmitting data about the robot 110 (e.g., such as robot movements and positions, robot health and status, robot capabilities, etc.); and an adapter for transmitting command and control information to and from the robot 110 (e.g., such as multi-step responsibility instructions, individual operation instructions (e.g., navigating to a destination, taking photographs, taking video recordings, taking sound recordings, taking environmental measurements, taking substance temperature measurements, taking air temperature measurements, taking humidity measurements, determining instrument readings, measuring the presence or concentration of gases or chemicals, emitting light of a particular wavelength, intensity and/or emission pattern, emitting sound of a particular tone, intensity and/or emission pattern, emitting radio frequency homing beacons, transmitting stored data via a radio frequency or wireless data network, connecting to a power supply, connecting to a radio frequency or wireless data network, connecting to a data network port, modifying settings of a valve or other control such as by adjusting the position of a valve, blade, locally controlled pump or driver, taking manual product samples, pressing buttons, changing switch positions, moving objects, stopping all operations, returning to home, activating a particular sensor, etc.)). The robot-specific adapter 644 may also receive information from the robot 110 including, for example, information about the status and completion of tasks or responsibilities, non-data task completion information (e.g., completion of responsibilities or abandoned, etc.).
Command/control and data information can generally be maintained within RSP 672 in one or more general internal formats. This may allow for internal operation of RSP 672, as well as interfaces and information external to RSP 672, such as shared with other components of an operations management system (OM), to be independent of the robot or robot type in an active state in the facility. Thus, the complexity of using different robot fleets within the facility may be reduced. To this end, each adapter may retrieve information from RSP 672 in one or more generic internal formats, convert the information into specific information appropriate for the particular robot to which it is to be transferred, and then transmit the converted information to the particular robot. For example, the definition of robot tasks or responsibilities may be converted from a generic internal format to a format that conforms to the intended command protocol of the assigned robot. Instead, data transmitted from the robot may be received by the adapter in a robot-specific format and converted by the adapter into a generic internal format for use by RSP 672 and other components of an operation management system (OM) such as operation management system (OM) 700 depicted in fig. 7. A common internal format for defining robotic tasks or responsibilities may include a list of measurements or data to be captured within the facility. The adapter 644 may convert this list into robot-specific instructions, which may include, for example, the direction to physically travel to the data capture location and which instruments to use. Vendor-specific robot data may also be parsed by the adapter 644 to detect generalized or useful information to be displayed to an operator. For example, data such as battery life may be displayed in different ways (percentage/residual voltage/residual time) that may be parsed and normalized.
Some such adapters 644 may be limited to only being adapted to communicate with a particular robot depending on the type of robot or vendor, while other adapters may be compatible with a variety of robot types. The adapter 644 may be considered a fleet-specific adapter compatible with multiple robots of the same type and operates simultaneously or cooperatively. The adapter 644 may be manually encoded based on a generic internal data format and information regarding the data and command protocols of a particular vendor and robot. Alternatively, such adapters may be automatically generated based on algorithms or artificial intelligence processes. The adapter may be configured to provide an equivalent level of basic control over multiple robot types. This may include, for example, the ability to move to a specific point, data capture, and robot metric reporting. The adapter may also include additional functionality depending on robot vendor specific software constraints for each robot type. For example, if the robot is unable to record its metrology data, the adapter may record the metrology data of the robot to maintain standard adapter-level compatibility with other components of the platform. Generic features from different robot types may be abstracted into a generic internal data format, such as map data, simulation data on a 3D model, etc. The OM system 700 or IA system 505 may allocate a Work Instruction (WI) checklist consisting of instructions to record information corresponding to various identifiers, which may be in a human-readable format, such as a spreadsheet or the like. The RSP 672 may convert the WI look-up table into detailed information, such as absolute or relative position and orientation information, such as GPS coordinates, location specific visual, electronic or magnetic identifiers, etc., for execution by the robot. Unstructured data types recorded by the robot, e.g., media formats such as images, sound, or video, may be further processed by RSP 672 using a data processing application into structured data types, such as text or digital data compatible with storage and processing in OM system 700 or IA system 505.
The navigation control and data service 626 may also include, for example, a fleet management module 638 to receive information from the process control server 632 regarding responsibilities and tasks to be completed. The fleet management module 638 may determine which of the robots 110A, 110B, and 110C should perform each task or responsibility and may provide detailed information about the robot tasks and responsibilities to the robot-specific adapter 644. The fleet management module 638 may also provide information to the process control server 632 regarding the progress or completion of responsibilities and tasks. The navigation control and data service 626 may also include, for example, a data management module 640, which may receive data from robots 110A, 110B, and 110C through a robot-specific adapter 644. The data management module 640 may also include a Machine Learning (ML) and Artificial Intelligence (AI) module 642, which may further analyze data received from robots 110A, 110B, and 110C.
The external system interface 624 and the navigation control and data service 626 may provide information regarding tasks dispatched to and performed by robots 110A, 110B, and 110C, task results returned by robots 110A, 110B, and 110C, and other information related to the management of robots 110A, 110B, and 110C, as well as responsibilities performed by robots 110A, 110B, and 110C to database 620. For example, records of robot tasks and task results may be stored in log 622 within database 620.
The fleet manager 628 may manage the fleet of robots 110A, 110B, and 110C, involve scheduling and dispatching responsibilities and tasks to the robots 110A, 110B, and 110C, monitoring health and maintenance status of the robots 110A, 110B, and 110C and their components, scheduling maintenance of the robots 110A, 110B, and 110C, and so forth. The fleet manager 628 may include a vendor-specific fleet management module 646, which may provide management of the robots 110A, 110B, and 110C that are specific to a vendor of one or more of the robots 110A, 110B, and 110C. A separate vendor-specific fleet management module 646 may be provided for each vendor-specific interface, such as vendor-specific interfaces 648 and 650. The fleet manager 628 may assign responsibilities and tasks to the robots 110 based on the responsibilities requirements and the capabilities of the robots. For example, if the fleet manager 628 determines that responsibilities may be supported only by the robot type of type a, the fleet manager 628 may assign responsibilities only to robots in the robots 110A, 110B, and 110C of type a. For example, responsibilities may require traversing obstacles, such as stairs, that can only be traversed by a particular type of robot. For another role that may be assigned to robots of type A or B, the fleet manager 628 may assign roles to robots of types A or B, 110A, 110B, and 110C, based on the availability of each robot 110A, 110B, and 110C. The availability of each robot 110A, 110B, and 110C may be determined by the fleet manager 628 using metrics such as, for example, communication status and battery life. In some embodiments, the operations management system 200 may specify responsibilities or tasks to be performed by a particular robot or robots in a particular robot group. If a group of robots is specified, the fleet manager 628 can assign responsibilities to selected robots in the specified group of robots. Furthermore, the robot type or some properties of the robot from a particular vendor may have a known reliability, and the fleet manager 628 may more or less directly monitor the robot application based on the known reliability.
The robot interface 630 may include, for example, a first vendor-specific robot interface 648, a second vendor-specific robot interface 650, and a fleet server 660. Each of the first vendor-specific robot interface 648 and the second vendor-specific robot interface 650 may provide an interface to a vendor-specific robot of the robot. For example, a first vendor-specific robot interface 648 may interact with robot 110A via a Robot Operating System (ROS) interface, and a second vendor-specific robot interface 650 may interact with robot 110B via a proprietary Remote Procedure Call (RPC) interface. The number and type of vendor-specific robotic interfaces provided by the robotic interface 630 is not limited and may depend on the number and type of robots in the robotic team managed by the robotic service platform 672. A vendor specific interface is typically required to access these robots. That is, robots from each vendor typically have proprietary software and are gated using each vendor's Application Programming Interface (API). For example, if a vendor uses a gRPC call to access its unique API to control a robot from the vendor, there may be no other alternative to communicate with or control the robot. The fleet server 660 may interact with a robot coordinator fleet, such as the robot 110C, through a fleet client 664. The robot interface 630 also may interact with the real-time communication module 662 to provide additional communication streams to, for example, the robots 110A, 110B, and 110C, the human operation manager, or the operator 602, such as through the process control server 632 or the process control client 604 or other components of the RSP 672. For real-time communications, the robot interface 630 may support multiple types of protocols for streaming data transmission. For example, webRTC is a streaming protocol for high quality video/sound streams that can be used for such communications. Such streaming may include, for example, receiving data to be displayed to an operator and performing real-time processing of the data using Artificial Intelligence (AI) or Machine Learning (ML) to support functions such as anomaly detection, for example. Real-time communication may be important for fleet management in order to perform different tasks. Some robots may have the ability to operate for a short period of time during lost communications, while others may not. The fleet server 660 may require frequent communication with each robot based on the capabilities of the robot and tracking the robot during responsibilities.
The vendor-specific fleet manager module 646 and the fleet server 660 may be cloud-based or may operate separately from the fleet manager 628 and the robot interface 630.
Robots 110A, 110B, and 110C may be provided by different suppliers and may be of multiple types and may also have different instruments, tools, and sensors available. Each robot may incorporate a robot management and fleet management module of RSP 672 to maintain information regarding the health and status of robot 110, including battery or fuel level, capacity, location, failure, maintenance requirements, and the like.
Examples of robots 110A, 110B, and 110C may include fully autonomous, preprogrammed for a particular task, motion, route, or activity, or directly manually controlled robots. Robots 110A, 110B, and 110C may be stationary or mobile, and mobile robots may include wheeled, tracked, double tracked, quad tracked, or multi-tracked or other propulsion devices. Robots 110A, 110B, and 110C may be provided with tools, sensors, or other hardware for performing tasks and tasks, such as articulated arms, handles, claws, wrenches, drives, hammers, crowbars, cameras, microphones, chemical detectors, noise sensors, vibration sensors, and the like. Robots 110A, 110B, and 110C may include digital and physical memory, such as for photographs, videos, sounds, environmental readings, environmental samples such as soil, chemicals, and the like. Robots 110A, 110B, and 110C may include various communication capabilities, including analog (radio and video transmissions, etc.) and digital (Wi-Fi, bluetooth, other near field communications, etc.).
A robotic service platform such as RSP 672 may interact with and rely on other subsystems of the operations management system. Fig. 7 illustrates a subsystem of an operations management system 700 in accordance with one or more embodiments, as well as a multi-vendor RSP that can operate as part of, separate from, or in concert with the operations management system 700.
The operations management system 700 may include a number of subsystems to support the operation and management of industrial facilities. Such subsystems may include, for example, an operations support system 702, a patrol management system 704, a task management system 706, a process control system 710, and a robotic management system or Robotic Support Platform (RSP) 672.
The process control system 710 may provide operational control and monitoring for the facility through a control subsystem that may be distributed within the facility or located outside the facility. The process control system 710 may include, for example, a collaboration information server 722 that may support sharing information between the process control system 710 and other subsystems of the operation management system 700. The process control system 710 may also include: a field control system 728 that can control processes within the facility and collect data from those process data via field instruments; and a safety control system 730 that can monitor safety conditions of the process within the facility and can ensure that the process returns to a safe state in the event of a deviation. The process control system 710 may coordinate with the operations support system 702, the patrol management system 704, the task management system 706, the robotic management system, or a Robotic Support Platform (RSP) 672.
The operation support system 702 may provide services that support the overall operation of the facility. The operation support system 702 may include, for example: a collaboration information server 722 that may support sharing information between the operation support system 702 and other subsystems of the operation management system 700; a process information management module 714 that can store and manage information about processes and procedures for operating the industrial facility; and a process execution management module 716, which can manage processes and execution of processes.
The patrol management system 704 may provide services for periodic patrol and monitoring of facilities with respect to operation, security, and security. The patrol management system 704 may include, for example, a collaboration information server 722, which may support sharing information between the patrol management system 704 and other subsystems of the operations management system 700. The patrol management system 704 may further include: a checklist administration module 718 that can administer checklists to ensure adequate coverage of all operations, security and security protocols; a checklist execution management module 724 that may manage execution of tasks to meet checklist requirements, such as may be determined by the checklist management module 718; and a schedule management module 720 that can schedule completion of the checklist tasks. Checklist and associated checklist tasks may be assigned to human or robotic assets. The allocation and scheduling of robot-allocated tasks will be discussed in more detail below.
The task management system 706 can provide creation, distribution, and monitoring of tasks within a facility. The tasks may be human-assigned or robot-assigned. The task management system 706 may include, for example, a collaboration information server 722 that may support sharing information between the operational task management system 706 and other subsystems of the operational management system 700. The task management system 706 may further include: a trigger management module 725 that can generate new tasks triggered by the incoming information; and a task execution management module 726 that may control the allocation and execution of tasks. The management of tasks assigned by the robot will be discussed in more detail below.
RSP 672 may provide for the management and operation of various types of robotic fleets provided by multiple vendors. RSP 672 may include, for example, collaboration information server 722, which may support sharing information between RSP 672 and other subsystems of operation management system 700. RSP 672 may also include robot team management module 628, robot data management module 640, robot generic interface modules (such as robot specific adapter 644), database 620, robot data analysis modules (such as data management module 640), and Machine Learning (ML) and Artificial Intelligence (AI) modules 642, all of which are discussed above with reference to fig. 6. RSP 672 may also include, for example, robot interface modules specific to different types of robots, such as a robot a interface 742 that interacts with robot 110A of type a, a robot B interface 744 that interacts with robot 110B of type B, and a robot C interface 746 that interacts with robot 110C of type C. The robot interface may be directly connected to the robot 110, such as shown for robot B interface 744 and robot C interface 746, or may be connected to the robot 110 through an external robot controller, such as robot control 748 connecting robot a interface 742 and robot 110A. Such external robotic controllers may be cloud-based servers, as shown in fig. 7, or may be connected through various types of computer networks. An exemplary connection between the robotic support platform and the robot 110 will be discussed below with reference to fig. 8.
Fig. 8 illustrates an environment 800 for robot control and communication within a multi-vendor robot service platform in accordance with one or more embodiments.
The operations management system 805 may include many systems and modules for overall management of industrial facilities, such as those shown in fig. 2 and discussed above, for example. As described above, the operation management system 805 may interact with a Robot Support Platform (RSP) 840 to provide information regarding responsibilities and tasks related to the operation of a plant that may be performed by robots, such as robots 110A and 110B.
The connections between the operation management system 805, RSP 840 and robots 110 may be provided differently depending on the needs and capabilities of the facility and robots 110. For example, in one embodiment, RSP 840A may be provided as a cloud-based service that connects to robot 110A using a global network 830, such as the internet. In another embodiment, cloud-based RSP 840B may use virtual private network 835 to securely connect to internet service provider network 825 and from there to robot 110A. In yet another embodiment, RSP 840C may be provided as an application running on a local computer, such as local personal computer 845. RSP 840C may connect to robot 110A using local web-server 845 and local network 820. In yet another embodiment, RSP 840C may utilize web server 850 to connect to vendor-specific robot server 815 and from there to robot 110B. The vendor-specific robotic server 815 is depicted as a cloud-based service, but the vendor-specific robotic server 815 may be provided on, for example, a local personal computer 845, a different local computer, or a remote computer accessible via a global network such as the internet, or the like. In another embodiment, cloud-based RSP 840D may connect to vendor-specific robot server 815 and from there to robot 110B.
Fig. 9 illustrates a process 900 of distributing, scheduling, and executing robotic tasks within a multi-vendor Robotic Service Platform (RSP) (e.g., RSP 672 in fig. 6) in accordance with one or more embodiments. In operation 910, an operations management system (OM) (e.g., OM 700 in fig. 7) may determine that it is desirable to undertake one or more tasks for operation or maintenance of an industrial facility. At least one of the tasks may be specified to be completed by the robot in operation 910 or later in the process by the RSP. In operation 920, OM may transmit the robot-specified task to the RSP. In operation 930, the RSP may convert the received robot-specific task into one or more robot responsibilities. Converting the received robot-specified task to a robot responsibility may include determining factors of the responsibility, data related to the environment in which the task is located, and/or responsibility capabilities for completing the task, as described herein. The definition of robotic responsibilities will be discussed in more detail below. In operation 940, the RSP may schedule one or more robot responsibilities to include, for example, selecting an appropriate one or more robots and assigning responsibilities to the selected robots. The selection and assignment of robots to particular tasks will be discussed in more detail below. In operation 950, the RSP may manage (e.g., initiate, monitor, and otherwise manage) one or more robotic responsibilities. Managing robot responsibilities may include, for example, dispatching robots, monitoring the progress of robots during responsibilities, and responding to events during responsibilities, such as by assigning additional or alternative robots to responsibilities. Managing robot responsibilities will be discussed in more detail below. In operation 960, the RSP may receive data generated by one or more robotic responsibilities. Processing data generated by robotic responsibilities and/or other feedback received from the facility before, during, or after the responsibilities will be discussed in more detail below. In operation 970, the RSP may update the task received from OM based on the received data. In operation 980, OM may update various information about the operation of the facility and the task completed. This information may include plant operation and maintenance status, and may be displayed through various user interfaces, such as, for example, shift reports or facility status dashboards. OM may also use the completion status of the task and the data generated by the task to plan future tasks in new iterations starting from operation 910.
Method for defining, distributing and managing responsibilities
Fig. 10 illustrates a flow chart 1000 for defining, assigning and managing one or more responsibilities, such as in the industrial environment facility 100 shown in fig. 1. For example, in step 1010, one or more responsibilities may be defined based on one or more tasks and factors or criteria for accomplishing each task. Responsibility may be defined by an operation management system (e.g., operation management system 700 of fig. 7), a user (e.g., via control system process control client 604, web client 606, or directly via an input device), another system in communication with RSP 672, or RSP 672. Once one or more responsibilities have been defined, the responsibilities can be assigned to a person, robot, or combination of person and robot in step 1020. As will be described herein, responsibilities may be scheduled and assigned via the robotic team manager 628 of RSP 672. In step 1030, responsibilities may be continuously or intermittently monitored by RSP 672 and, as responsibilities are monitored, changes to responsibilities, for example, may be managed based on feedback from a human and/or robotic operator. Examples of defining, assigning, and managing responsibilities are described herein.
According to one example, the robotic management system RSP 672 in fig. 6 may define one or more responsibilities. One or more responsibilities may define one or more tasks to be performed by an operator (e.g., a human operator, a robotic operator, or a combination of a human operator and a robotic operator). For example, to define responsibilities, one or more tasks may be manually selected or entered, such as by a user via dashboard 3000 in fig. 30, or automatically selected or entered, such as via process control client 604 or via a remote site, first web client 634, or second web client 636. A task may include the purpose of the responsibility, in other words, the final task to be performed in the responsibility. For example, the tasks may include: monitoring and/or obtaining data points, status or conditions (e.g., pressure, temperature, humidity, other weather conditions, noise levels and/or types, vibrations, sensor values, locations, levels of matter (e.g., fuel, gas, fluid, etc.), altitude, weight of an environment, machine or facility, obtaining samples (soil, air, products of a machine or facility, etc.), battery power, etc.); performing all or part of an operation at or within a machine or facility (e.g., flipping a switch, pressing a button, moving an item, rotating a valve, performing a step of assembling an item (e.g., within an assembly line or otherwise), adjusting a workpiece or other item, securing a broken workpiece or other item (e.g., tapping a gauge to provide an appropriate reading), lifting an item, performing maintenance on an item (e.g., replacing other portions of a circuit board or device), etc.; and/or any other work necessary or desired within the facility.
While responsibilities may include a single task, responsibilities may include multiple tasks that are performed together. Multiple tasks may be performed simultaneously and/or may be performed serially. In some cases, one task may have to be performed before another task. In this case, responsibilities may indicate the order of tasks.
The tasks may be determined by the operation management system 700, RSP 672, or any other system or module in communication with the operation management system 700 and/or RSP 672. As further described herein, the operations management system 700 and/or RSP 672 may define responsibilities based on factors, data, and responsibilities (as further described herein), etc., necessary or desired to complete a task. Responsibilities may include all informational instructions necessary or desirable to perform a task.
For example, dashboard 3000 (see fig. 30) may allow a user to input factors, data, and responsibility capabilities, as described herein. Dashboard 3000 may include categories that display various information related to responsibilities. For example, creation time 3005 may indicate when to create responsibilities. The description 3010 may provide detailed information of responsibilities or names of responsibilities. Task type 3015 may include types of tasks such as regular tasks, special tasks, maintenance tasks, urgent tasks, and the like. Status 3020 may indicate whether a task is in progress, complete, or whether a task has been defined but has not yet been initiated (e.g., a draft). The name and/or type of robot scheduled for responsibilities may be indicated under the robot name 3025, and the planned end time and start time 3030 may indicate when responsibilities will begin and/or end.
The user may create new responsibilities using new functionality 3040 and may enter the information described herein to define the responsibilities. Alternatively, the data may be automatically populated, for example, from the historian 612 or some other memory. The user may also use the search 3050 to search for tasks, responsibilities, or other data within dashboard 3000, and/or organize tasks and/or responsibilities via a list of job instructions 3060. Job instruction list 3060 may allow a user to access a drop down menu to select which categories to view and/or which categories to use to organize dashboard 3000.
Referring to FIG. 31, a job instruction 3100 (e.g., a responsibility summary) can be created by a user, for example, via new function 3040 in FIG. 30. Job instruction 3100 may include various inputs such as details of responsibilities and tasks 3110, planned start and end times 3120, and operator status categories 3130. Accessory 3140 can be included in job instruction 3100. The operator can use the attachment 3140 for additional data regarding responsibilities, tasks (e.g., tools to be used for the tasks), and the like. Job instruction 3100 may include a variety of layouts and information and is not limited. Furthermore, this information may be updated and/or confirmed by a user or automatically, as described herein.
Fig. 11 shows a flow chart 1100 for defining responsibilities. In step 1110, RSP 672 may receive the task. The tasks may be received by RSP 672 via process control client 604, via input directly to RSP 672 (e.g., via a user interface (e.g., dashboard 1400 in fig. 14)), via a remote system (e.g., a wired or wireless device connected to operation management system 700 or RSP 672), via a memory (e.g., historian 612 in fig. 6 that stores historical data) that interacts with operation management system 700 and/or RSP 672, or a combination thereof. Tasks may be sent to RSP 672 in response to user commands, or may be sent automatically, e.g., based on feedback received by RSP 672 and/or operation management system 700. As another example, tasks may be received based on feedback from one or more operators (e.g., robot 110 and/or a human operator) performing responsibilities and managed as described herein.
Upon receipt of the task, factors for defining responsibilities may be determined in step 1120. Such factors may include information necessary or useful to perform the task. For example, if the task is to take a sensor reading, factors may include, but are not limited to, the following sensor information: type (analog, digital, etc.), location, altitude, map of the area surrounding and leading to the sensor (sufficient information to access the sensor), environmental conditions at the sensor location (e.g., temperature, pressure, etc.), etc. Another factor may include the timing of the task, such as the time the task must begin and/or the time the task must complete. Additional factors may include steps that may or may not be performed in order to accomplish a task. For example, referring again to taking a sensor reading as a task, it may be necessary to activate a switch before taking a sensor reading. In this case, the factor may include a step of activating the switch before reading the sensor. Other steps (factors) may include opening a door, climbing stairs, traversing a tunnel, turning on a light, etc.
Factors may also include routes to the sensor and factors within those routes. For example, the factors may indicate whether there is a low hanging object along the route, whether there is an opening to navigate, and/or include the size of one or more channels along the route. For example, factors may include a minimum opening along the route. Factors may also include various routes for reaching the task location. For example, a map may provide a number of possible routes for reaching a task location, regardless of how the operator reaches the location. For each possible route, factors may include factors associated with the route, such as the dimensions of the path and/or objects along the route.
In step 1130, dynamic data (e.g., time-varying data) may be received based on one or more factors. Such dynamic data may include particular values or parameters necessary or useful for task execution based on one or more factors. For example, referring again to taking sensor readings as a task, if the factor is location and the sensor is mobile, data regarding the sensor location may be needed. The data may be obtained by any suitable positioning means such as GPS, cameras, etc. If the sensor is fixed (not movable), then additional information about the location may not be needed, as this information will be included as a factor. As another example, if temperature is a factor, a value (data) of temperature may be obtained, for example, via a thermometer adjacent to the sensor.
Dynamic data may also be managed by operating management system 700. For example, an operator may update a sensor profile with data that may be used to run one or more responsibilities. Alternatively or additionally, the sensor profile may also be updated automatically using data from other systems (e.g., remote or external systems), or based on feedback from sensors within the facility and/or feedback from other responsibilities. Dynamic data (e.g., weather) may also be obtained and updated by RSP 672 via a third party (e.g., weather application). Such dynamic feedback may also be provided by sensors attached to the robot and/or sensors in the facility. For example, if the robot senses water, such as rain, and the robot is unable to work in the rain, the robot may terminate responsibility and travel to a predetermined covered location. The robot (or other communication device) may also communicate the termination of responsibility to RSP 672.RSP 672 may reassign responsibilities to another robot that has the ability to perform responsibilities, including the ability to operate in water. Alternatively, RSP 672 may schedule the same robot to complete duties after a rain outage. In some cases, RSP 672 may use weather data from third parties and/or other sensors in the facility to terminate responsibilities if the robot is unable to determine that there is rain, e.g., the robot itself does not have a sensor and/or edge processor to detect and/or terminate responsibilities.
More generally, data can be received from sensors in and around the facility where RSP 672 manages the robotic team. The data may include dynamic measurements or information such as the location of movable objects in the facility, map information of the facility, and/or other environmental conditions associated with the facility. For example, the environmental conditions at the task location and/or the environmental conditions on the route to the task location may include data. Environmental conditions may include ambient temperature, air content or air quality, conditions of one or more surfaces at or on the route to the task location, and the like. The environmental condition may be a condition along a single route to the task location or may include environmental conditions along multiple routes to the task location.
In some cases, it may be useful to provide additional sensors to supplement or compare to existing sensors, allowing for more in-depth analysis of sensor data or accuracy checking of existing sensors. For example, if an existing sensor indicates an abnormal change in data readings, the robot may carry and place the sensor in a position suitable for temporarily reading the data. The robot placement sensor may be instructed by RSP 672 as described herein, or the robot may decide to do so autonomously or semi-autonomously. The additional sensor may transmit its data readings over a wired or wireless connection and may transmit the data readings to the robot or elsewhere for analysis and/or comparison with readings from existing sensors.
The data may also include information about dynamic (e.g., movable, variable) objects along the route of the task location. For example, a container, person, or other object may be positioned along a route. The objects may be associated with one or more sensors that may indicate when and where the object is moving, or may obtain the position by other means such as imaging. Further, the data may indicate that there are no more intended objects along the route. Alternatively or additionally, any type of data, such as image data, may be received as feedback from one or more of the sensors, the person, or the robot, as described herein. The image data may, for example, display the time and place of the object moving along the route and may be included as data when defining the responsibility summary. The image data may for example display information important for human activity in an area, such as whether there is a fire extinguisher, availability of appropriate personal protective equipment (hard caps, safety boots, etc.) in the field, etc.
In step 1140, the responsibility capability of the operator for performing the responsibility is determined based on the factors determined in step 1120 and/or the data received in step 1130. The responsibility capability may include features of the robot that allow the robot to perform responsibility and complete tasks. For example, if a reading is to be obtained from an analog sensor, the sensor may have to be imaged. Thus, responsibility will include imaging capability. An additional responsibility capability may be to obtain sensor values from the imaging sensor. This may be accomplished, for example, by accessing an image analysis program for determining sensor readings based on the images. Instead of the responsibility capabilities of the robot, the image analysis program may be associated with the operations management system 700, RSP 672, and/or a remote system connected to the operations management system 700 or RSP 672. For example, image data analysis, as well as other types of analysis of raw data collected by an operator, may be performed by the robotic data analysis module 642 of the RSP 672. In some cases, responsibilities may require that a robot be unassigned or may not be assigned to responsibilities based on the characteristics of the robot. For example, a responsibility capability may indicate that at least a portion of the responsibility must be performed within the dimensions of a given area; routes to responsibility may require traversing small spaces or require certain tools or kits. If the robot is too large to navigate to a route of responsibility or if the robot is not equipped with a particular tool or toolkit, RSP 672 may be provided, for example, to designate the robot as not being able to assign the responsibility.
Responsibility capability may include features associated with the robot so that the robot may operate in certain environments. For example, if temperature is a factor and the data indicates a very high temperature, then the responsibility capability may be the ability of the robot to withstand temperatures at or above a certain temperature. Responsibility capability may also include, for example, the ability to reach a location of a particular elevation. Responsibility capability may also include the ability to descend stairs, enter limited spaces of limited size, or otherwise have a route to the location. The responsibility capability may also include availability of the robot at a particular time and/or for a specified amount of time based on factors defining the start time, end time, and/or duration of the responsibility.
Once RSP 672 determines or receives the factors, data, and/or responsibility abilities, a responsibility summary may be created in step 1150. The responsibility summary may be a data file that includes factors, data, and responsibility capabilities defining the responsibility. The data file may be in any format, such as a format compatible with the operations management system 700, RSP 672, and/or any other system. As described above, the data file may be compatible with one or more robot types, or may be modified to be compatible with one or more robot types.
In step 1160, the responsibility summary may be output directly to a robot or other system for viewing and/or further processing by the person. For example, the responsibility summary may be output to a device having a display, such as a wired or wireless device, so that the user may view the responsibility summary. The display may include a Graphical User Interface (GUI) that may display each of the factors, data, and/or responsibility capabilities in the responsibility summary. The responsibility summaries may also be stored in memory, for example, memory associated with RSP 672 (e.g., to be stored in historian 612 of fig. 6), memory associated with operations management system 700 and/or one or more operators, such as memory associated with a robot or a device operated by a person. The responsibility summary can also be modified before or after being output by RSP 672 to be compatible with one or more other systems, such as operations management system 700, a robot, etc.
For example, the output responsibility summary can cause RSP 672 system to generate a plurality of files to be compatible with all systems associated with RSP 672 (e.g., robotic systems and/or any other systems). Alternatively or additionally, RSP 672 may query all systems to determine the file format of the output responsibility profile. For example, as shown in fig. 6, robot type a 110A and robot type B110B may be able to receive a role profile with an unchanged file format. In contrast, robot type C110C may require that the responsibility summary be transferred from RSP 672 via fleet client 664 to modify the responsibility summary file format to be compatible with robot type C110C. As will be described herein, responsibilities may be assigned to robots based on a schedule. Thus, the responsibility summary is not provided to the robot until after the responsibility is assigned, as described in the flow diagram 1200 of FIG. 12. Alternatively, as described above with reference to FIG. 6, the responsibility summaries may be converted from a general internal format to a format appropriate for the particular robot or robot type by the robot-specific adapter 644.
The responsibility file may contain one or more of the following: a role identification (e.g., name, number, etc.), role priority, and/or identification of any other role scheduled to be completed simultaneously with the role; metadata including credentials and/or contact information of the author, summaries of time and date of job generation, descriptions of the job, etc.; the brand and model of the robot, including the current version of the software used by the robot, the weight or size of the robot, etc.; a payload currently loaded on the robot and/or a payload usable by the robot; references (e.g., bar codes, QR codes, markers) associated with the robot; minimum battery/energy requirements for the robot to complete the duties, and any reserves built in to address any steering of the robot during the duties; map of the facility (two-dimensional and/or three-dimensional); a route for performing responsibility and/or a task along the route; start time and end time or time range for performing duties; any personnel and/or other robots necessary to complete the task; credentials of an operator or other robot capable of starting, stopping, and/or modifying responsibilities; a communication protocol; responsibility specific and/or general alarms or warnings that may occur during responsibility, and operational indications when alarms trip; preconditions for initiating, performing, and/or completing responsibilities (e.g., not raining during the responsibilities or during at least a portion of the responsibilities); sensor location and details of the sensor on the robot and any sensors with which the robot is configured to interact during responsibility; or any optional or auxiliary responsibilities may be performed based on feedback received during the responsibilities.
In some examples, the responsibility files may be generated based on a robot type, such as a manufacturer of the robot, or the like. Each robot type may include a particular method of recreating the sequence of actions based on the list of instructions. In some cases, multiple robots may be designed to recreate a sequence of actions in a similar manner. Each robot may generate a file, such as a directory of instruction files, which the robot may transmit to RSP 672 via robot-specific adapter 644. RSP 672 may select the responsibility file based on the type of robot. With the generation of the responsibility summary, the responsibility capability can inform the RSP 672 (or operator) what functions the robot must be able to satisfy to perform the responsibility. For example, if the task includes taking a photograph, the responsibility profile may indicate how many photographs to take, the resolution of the imager used to obtain the photograph, what object to take, etc., as described herein. RSP 672 may provide the operator with a robot-specific method so that the operator may generate a responsibility summary based on the robot-specific language. If multiple robots are required for responsibility, the operator may repeat the process of generating a summary of the responsibility for each robot in the robot-specific language. Alternatively, RSP 672 may automatically generate the responsibility summary. In some cases, a robot language that is generic to the robot may be used such that a single responsibility summary may be generated using the generic robot language. In some examples, RSP 672 may confirm the capabilities of the robot before assigning responsibilities to the robot to ensure that the robot capabilities have not changed. For example, the payload of the robot may have changed or the sensor may fail.
Assigning one or more responsibilities is described with reference to flow diagram 1200 in fig. 12. In step 1210, one or more job summaries may be received. The responsibility summary received in step 1210 may be the responsibility summary output during step 1160 in fig. 11. Alternatively or additionally, one or more responsibility summaries can be received via user input (e.g., via a wired or wireless device), via memory (e.g., based on historical data explained herein (e.g., from the historian 612 of fig. 6)), and/or from another system (e.g., a third party system). As described herein, each responsibility summary can include one or more tasks, factors necessary or useful for performing the tasks, dynamic data based on the one or more factors, and/or responsibility capabilities of the robot for performing the responsibility. Responsibility may be assigned based on factors of responsibility and robot type.
The ability of a robot to perform a role or the ability of multiple robots to perform a role includes the ability of one or more robots to satisfy all of the features of the role. In contrast, if the capability of the robot does not satisfy each feature of the responsibilities, the robot cannot perform the responsibilities. As described herein, multiple robots that cannot individually satisfy all of the features of a role but can together satisfy all of the features of the role may be jointly assigned roles. Such a decision whether the robot is able to perform duties may be determined by RSP 672 or the robot. For example, RSP 672 may initially assign responsibilities to one or more robots based on the features of the responsibilities, but the robots may determine one or more features during the responsibilities that they cannot perform responsibilities based on feedback, as described herein.
It should be appreciated that multiple robots may be able to perform the same duties. In this case, responsibilities may be assigned to any of those robots that are capable of performing the responsibilities. Alternatively, responsibilities may be assigned to one of a plurality of robots based on one or more priorities of responsibilities. The priority of a responsibility may be an attribute that has a desired level of importance to the responsibility, as opposed to the features required by the robot to perform the responsibility and to complete the task of the responsibility. The priorities may include, for example, one or more of a desired execution quality of the responsibilities and their tasks, a desired execution efficiency level of the responsibilities/tasks, a desired cost of completing the responsibilities/tasks, and the like. Each of these responsibility priorities can have a threshold associated with it, such as a quality threshold or a cost threshold required for responsibility execution. For example, responsibilities may be characterized as requiring the highest level of quality of responsibilities or tasks to be performed, to perform responsibilities or tasks within a minimum length of time, to perform responsibilities or tasks with the least energy, to perform responsibilities or tasks with the least error or failure or request for further assistance/human intervention, and/or to perform responsibilities or tasks at the lowest cost (e.g., if a robot is used in a rental arrangement). One or more robots may be selected based at least in part on the additional criteria. For example, responsibilities may include designations such as "highly critical" such that it requires very high quality output without failure. In this case, the robot having the least failure may be selected instead of another robot capable of performing responsibilities but having a higher failure rate. As another example, if the priority of responsibilities includes performing the responsibilities in the most energy-efficient manner, responsibilities may be assigned to a robot based in part on the ability of the robot to operate using lower energy.
In some cases, certain responsibility priorities may not be decisive in assigning responsibility to robots. In other words, priority of responsibilities may be desired, but is not required. In the example given above, the priority may include using lower energy to complete the duties. However, if the only robot that is able to perform the responsibilities uses higher energy, then the responsibilities may be assigned to one of these high energy robots, regardless of priority, because the required energy level may be less important than the time that the responsibilities must be completed.
It should also be appreciated that the priority may not be a numerical value. For example, the responsibility priorities may include some threshold, e.g., performing the responsibility if the energy usage of the robot is below a predetermined threshold, or the responsibility priorities may include the order or timing of performing the responsibility, e.g., performing the responsibility at some point in a series of responsibilities. Once all responsibilities are assigned, one or more responsibilities schedules can be generated based on all possible iterations of assigning multiple responsibilities to the robot. A responsibility schedule can be selected that satisfies as many priorities as possible for the various responsibilities.
In step 1220, the data and responsibility capacities can be updated, if desired, based on the time lapse since the creation of the responsibility summary and/or the receipt of the responsibility summary. Factors and responsibility capacities may be updated in step 1220 in a similar manner as outlined in step 1120. Further, when assigning responsibilities, additional information may be available that is not available when determining responsibilities. For example, the data may be modified based on feedback received from sensors and/or operators, as described in flow chart 2700 in fig. 27 and/or flow chart 2800 in fig. 28. In this case, the data may be updated based on the currently available information and may ensure that responsibilities are properly assigned.
All robots may be determined in step 1230. This includes any robot at or available to the facility. For example, the operations management system 700 may store robots that are to be assigned to facilities, or robots that are assigned and may be in maintenance but not available. This may also include robots renting or renting to facilities. Determining all robots in step 1230 may also include determining all robots in a third party robot team. In this case, any robot determined in step 1230 may include identifying all potential robots, regardless of availability and/or capabilities. It should be appreciated that while all robots available to the facility may be identified, not all robots can be assigned. For example, it may be determined that certain robots are not desired to be allocated based on high rental costs, availability according to contractual terms, use costs, and the like.
Once all potential robots are identified, the characteristics and/or capabilities of each robot (referred to as "robot capabilities" for ease of understanding) may be determined in step 1240. The robot capabilities may be received directly from the robot or may be received from a system storing the robot capabilities. Additionally or alternatively, the robot capabilities may be updated periodically or in real-time based on feedback received from the robot and/or other sensors or inputs. It should be appreciated that the robotic capabilities may be based at least in part on additional factors such as time, location, order of operation, etc. (e.g., a camera operating during the day may not be available for use at night, storage of data or physical samples may require uploading or discarding at a particular location, etc. before additional data collection can take place). In some cases, the robot may transmit data to and/or receive data from RSP 672 even when the robot is not performing duties. As described herein, RSP 672 may reassign responsibilities based on the data and/or the robot may receive updates to the responsibilities it is scheduled to perform or receive additional responsibilities based on the data. Further, the real-time monitoring of RSP 672 may determine whether a robot is no longer able to perform one or more responsibilities assigned to the robot. For example, the battery level may have fallen below an acceptable threshold for performing responsibilities, and RSP 672 may reassign one or more responsibilities to another robot that is capable of performing that one or more responsibilities.
Responsibility (as described above) can be separate and distinct from robotic capabilities. While the responsibility capability includes features that may fulfill the responsibility, the robot capability may include features associated with a particular robot. For example, the responsibility capability may include reading a sensor at a height of 50 feet. The robotic capabilities that can meet this duty capability may include the ability of an imager/camera to fly or climb the sides of a building. Furthermore, while there may be a single robot whose robotic capabilities meet a set of responsibilities, two or more robots each having separate robotic capabilities may be required to accomplish the responsibilities. For example, responsibilities may require a first robot with a camera and a second robot with a rotor and a sensor capable of lifting the first robot 50 feet from the ground.
The robot capabilities may include availability of the robot in general and/or at a particular time. For example, the robot may be assigned multiple responsibilities, or the robot may not be available for maintenance or repair at a particular time due to low battery or other conflicts. The robot capabilities may also include information related to damage to the robot. In this case, the robot may operate with less than full capabilities, but may be able to perform some functions. In this case, the robot capability information may include which nominal capabilities of the robot are unavailable, and the time at which the full capabilities may be restored if known. Similarly, the robot capabilities may include the remaining battery power and may indicate how long the robot may be used before having to recharge. The robotic capabilities may also include a human operator supporting or monitoring the availability of certain responsibilities or tasks within a single responsibilities (e.g., based on inherent hazards or critical activities, etc.).
The robotic capabilities may also include the ability of the robot to perform various activities, such as carrying a load on a weight, reaching a height, maneuvering or rotating an item through an opening having a particular size, capturing images, sensing noise levels and types, grasping the item, applying force to the item, and the like.
The robotic capabilities may also include tools associated with the robot. For example, the robot may have a gripping mechanism that may operate in a manner similar to a human hand to turn a lever or the like. Alternatively or additionally, the robotic capabilities may include the ability of the robot to acquire and/or use tools. For example, a wrench may be provided for use by the robot. However, if the robot does not include a gripping mechanism or similar mechanism for holding and operating the wrench, the wrench may not be a robot capability associated with that particular robot. The robot capability information may also include specifications of tools available to the robot, such as the size or torque capability of the wrench.
The robot capabilities may also include the ability of the robot to withstand certain environmental conditions, including certain temperatures, pressures, humidity, etc. For example, the robot may include materials that are suitable for high temperature applications but are not suitable for immersion in fluids. It should be appreciated that these are examples of robotic capabilities, and that the robotic capabilities may include other features associated with the robot. It should also be appreciated that the condition of the robot (e.g., aging, degradation, etc. of certain elements on the robot) may be included in the robot capabilities.
Once the robot capabilities are determined, one or more robots may be assigned to each responsibility in step 1250. Robots may be assigned based on the characteristics of responsibilities, data, and matching of responsibilities capabilities with robot capabilities. In this matching step, certain of the responsibility capacities may have higher weights (more important in completing the responsibility) than others, so that robots having robot capacities matching those of higher weights are considered. For example, a robot may have to have the robot capability to reach a higher elevation to fix the antenna, but the most preferred robot is not able to image or record the fixation of the antenna (desired duty capability). Thus, responsibility can be weighted or categorized (necessary, desired, etc.) to optimize the allocation of robots.
Factors other than such matching may affect the assignment of robots to responsibilities. For example, robots may be assigned to responsibilities such that as many responsibilities as possible may be achieved within the time frame assigned for each responsibilities. If each responsibility has a time-span in which the responsibility should be completed, the responsibility may be assigned such that the time-span factor is met for the maximum possible number of responsibilities. Alternatively, responsibilities may be assigned such that they reach the highest possible or threshold level of quality. For example, if each responsibility has a quality factor associated with the responsibility summary, the responsibility may be assigned to the robot in an attempt to have as many responsibilities as possible achieve a predetermined threshold level of quality metric. The allocation of robots may take into account the number and quality of responsibilities completed, weighting them to achieve proper allocation of robots. The robot may also assign based on the position of the task and/or distance from the starting point of the robot, the quality of the route of the task, etc. Based on the type of robot, the robot may not be tolerant of certain conditions, such as dust, wind, temperature and/or temperature differences, the presence of water and/or gas, etc. In some cases, an alarm may be triggered by the robot and/or RSP 672 to cause the responsibility to break and/or to cause a different robot (e.g., a "first responder" robot specifically responsible for an event) to begin responsibility. Responsibility may be assigned to one or more robots based on the payloads of the robots and/or payloads attachable to the robots. Responsibilities may also be assigned to one or more robots based on their proximity in time and/or space, predicted weather forecast, priority data, etc.
Once responsibility is assigned, a schedule is generated and output in step 1260. For example, the schedule may be output to a device having a display, such as a wired or wireless device, so that the user may view the schedule of responsibilities. The display may include a GUI that displays a dashboard (e.g., dashboard 1400 in fig. 14) that may display each robot and one or more responsibilities to be assigned to each robot. The schedule may also be stored in a memory, such as a memory associated with RSP672, operation management system 700, one or more robots, and/or any other remote devices. The schedule may also be modified before or after being output by RSP672 to be compatible with one or more other systems, such as operation management system 700, a robot, etc., as described in step 1160 of fig. 11. For example, the output schedule may cause RSP672 to generate a plurality of schedule files to be compatible with all systems associated with RSP 672. Alternatively or additionally, RSP672 may query all systems to determine the file format of the output responsibility summary and/or responsibility schedule. In this way, the schedule may be provided to multiple systems and/or multiple robots, which may help manage responsibilities, as will be described herein.
Managing one or more responsibilities may be described with reference to flowchart 1300 in fig. 13. In step 1310, a schedule of responsibilities and robots to which the responsibilities are assigned may be received. The schedule of assigned responsibilities may include the schedule generated and output in step 1260 of fig. 12. Alternatively or additionally, the schedule may be received or modified by a user input (e.g., via a wired or wireless device), via memory (e.g., based on historical data), and/or from another system (e.g., a third party system). As described herein, the schedule may include responsibilities and robots assigned to the responsibilities. The schedule may also include any other information associated with each responsibility and/or each robot that may be used to manage the responsibility. Upon receipt of the responsibility schedule, responsibility may be initiated according to the time specified in the schedule.
After determining the schedule, and/or upon initiating one or more responsibilities, feedback can be received 1320 from the robot, responsibilities, and/or environment in which the robot is located, or from another remote location. Feedback may be any information sent from the robot to RSP 672 and/or to any other robot, sensor, or other device within or outside the system (e.g., the system within facility 100 of fig. 1). Feedback may include information related to the overall status of the responsibilities, such as whether the responsibilities began, when the responsibilities began, and/or whether the responsibilities completed. Feedback may also include any information/data received from sources other than robots (e.g., sensors), including information/data related to the performance of duties.
The feedback may include continuous feedback or may be intermittent feedback based on the availability of communication by the robot, person, and/or facility. For example, the environment of the facility may reduce the ability of communication signals to propagate through the environment, and robots or other feedback devices may only be able to communicate at certain locations in the facility. Alternatively or additionally, the data transmission may be reduced to a predetermined time range to reduce the amount of data received and/or transmitted by RSP 672, sensors, robots, other feedback devices, and the like. In some cases, if communication with RSP 672, operations management system 700, etc., is lost, the robot may be instructed to shut down or otherwise stop moving. This may be a useful safety feature, for example, if the robot is unable to communicate robot-to-robot and/or if the robot does not have a sensor for detecting the presence of an object along the route of responsibility.
The feedback may also include the state of the robot. This may include the status of the robot performing the duties and/or may include the status of other robots or people in contact with the robot. For example, in performing responsibilities, a first robot performing responsibilities may encounter a second robot (or person) and communicate the state of the second robot to RSP 672. The first robot may communicate the status of the second robot, e.g., stationary, moving, malfunctioning, etc. This data may be used to alter responsibilities and/or a responsibilities schedule, as described herein. Alternatively or additionally, the first robot may provide instructions from RSP 672 to the second robot. For example, the second robot may be in a position that is unable to communicate with RSP 672. In some examples, RSP 672 may transmit instructions to all robots to provide instructions to the second robot as to whether and when any robot is in contact with the second robot. In this case, when the first robot (or any other robot or transmission device) is in communicative contact with the second robot, instructions from RSP 672 may be transmitted to the second robot. Similarly, the first robot may receive information from the second robot for transmission to RSP 672 or another device, and the first robot may transmit the information upon communicative contact with the device. In this way, all robots may act as communication nodes in the facility, which may improve the functionality of the management system and/or the scheduled responsibilities within the system.
The feedback may also include changes in environmental conditions and/or information from sensors along the responsibility route. For example, the robot may determine smell, hear noise, sense vibration, or learn an alarm from sensors at locations along the responsibility path. This feedback may be provided to RSP 672, may be communicated to another robot, may result in defining other responsibilities, and/or may be undertaken by the robot sensing the alarm, as will be described herein.
Additional details regarding the use of feedback are included herein with reference to flowchart 2700 of fig. 27 and flowchart 2800 of fig. 28.
In step 1330, RSP 672 may manage responsibilities based on the received feedback. Management of responsibilities may include modifying a schedule or responsibilities based on feedback. Schedule or role modification may include continuous looping such that a schedule or role (including termination) may be continuously modified based on feedback from the robot, changes to roles within the schedule, and/or increases or decreases in roles to the schedule. In this example, the schedule may be continuously modified. Alternatively, modifying the schedule and responsibilities may include terminating the schedule (or responsibilities) once all scheduled responsibilities (or particular responsibilities) are complete. Rescheduling or reassigning responsibilities based on feedback, which may include new responsibilities, is explained with reference to fig. 27.
Exemplary GUI/method for defining, assigning and managing responsibilities
In some examples, a user may define, assign, and/or manage responsibilities via inputs directly to a wired or wireless device, e.g., via a dashboard. An example of a dashboard 1400 is shown with reference to fig. 14. The user may select or modify responsibilities, robots, and/or a responsibilities schedule via a Graphical User Interface (GUI), such as dashboard 1400. Dashboard 1400 or similar devices may allow a user to input commands remotely, e.g., via web client 606 in FIG. 6, or directly, e.g., via process control client 604. Dashboard 1400 can display various titles, such as robots 1410, sensors 1420, responsibilities 1430, and/or alarms/events 1440. These titles may indicate subcategories that may be selected via touch, mouse, etc. for receiving, transmitting, and/or viewing information related to responsibilities and/or other events within the facility. The categories and/or subcategories may include images, numbers, and/or words that indicate each category or subcategory. For example, the user may select the sub-category configuration robot type 1412, the selection of which may take the user to another screen listing all available robot types. The user may select the type of robot to be used for a particular role, or may automatically select a robot based on a role profile, as described herein. Alternatively or additionally, configuring the robot type 1412 may allow the user to select a person performing the responsibilities.
Configuration robots 1414 may also be displayed under category 1410. The selection of the configuration robot 1414 may allow a user to select various functions or capabilities of the robot, such as including a particular sensor package. The configuration robot 1414 may also allow a user to select packages, such as particular tools and/or sensors, for use by personnel during responsibility. It should be appreciated that functions or capabilities may be automatically selected based on, for example, the responsibility profiles described herein.
A third category, robot state 1416, may be displayed under category 1410. The robot state 1416 may display the state of the robot, for example, via the list 1500 shown in fig. 15. For example, when the robot state 1416 is selected, the first robot 1510 and the second robot 1520 may be displayed in a list format on the dashboard 1400. The robot status 1416 may display various robot capabilities or other information related to each robot, such as a robot name 1530, a robot status 1532, a robot status 1534, a communication status 1536, a battery level and battery status 1538, odometer readings 1539, a responsibility status 1540, a responsibility progress 1542, a most recent (or last) communication update 1544 with the robot, and/or whether the robot has video capabilities 1546. It should be appreciated that this list is merely an example, and that any robot capabilities or other information related to the robot are shown in fig. 15. It should also be appreciated that the robot state 1516 is not limited to a robot and may be used to display the capabilities or status of any operator (e.g., a robot or a person).
With continued reference to FIG. 14, four subcategories may be displayed under the sensor 1420 category. For example, a subcategory configuration area 1422, configuration sensors 1424, sensor states 1426, and a graphical view 1428 may be displayed. Selecting each of these subcategories may change the display to show various information. For example, selecting the configuration area 1422 may configure the sensor based on where the sensor is located, e.g., zeroing out the sensor used in a particular area. This may be done manually or automatically based on feedback. Similarly, select configuration sensor 1424 may configure a sensor. For example, the sensor may be an imaging sensor, and when the configured sensor 1426 is selected, the capabilities of the image sensor, such as brightness, saturation, etc., may be altered automatically or manually by the user. Selecting the sensor state 1426 may provide a state of the sensor similar to observing the state of the robot when the robot state 1416 is selected. For example, a list of one or more sensors may be displayed, and upon selection of sensor state 1426, the states or capabilities of those sensors may be displayed. Selecting graphical view 1428 may display the sensor as graphical, e.g., displaying the temperature sensor as a thermometer, displaying the pressure sensor as a pressure gauge, etc. Selection graphical view 1428 may also graphically display other information for one or more sensors.
Under the category named responsibilities 1430, the user may select any of a configuration responsibilities template 1432, a configuration responsibilities list 1434, a running responsibilities 1436, a responsibilities history 1438, or a responsibilities history (validation) 1439. Selecting the configuration responsibility template 1432 may allow the user to change the factors, data, and/or responsibility capabilities that will be associated with the responsibility or may change the manner in which the factors are displayed. For example, it may be important to consider the ambient temperature when defining responsibilities. In this case, the user may choose to configure the responsibility template 1432 and may modify the responsibility template to include temperature as a factor in defining the responsibility.
Selecting the configuration responsibility list 1434 may allow the user to manually modify the scheduled responsibility or responsibilities. For example, upon selection of the configuration responsibility list 1434, a list of responsibilities and the robots to which each responsibility is assigned may be displayed. The user may view the robot and responsibilities and may change the responsibilities schedule. For example, responsibilities may include taking sensor readings, and a user may know that a sensor is located at a certain height. The user can see responsibility for scheduling robots that do not reach the sensor height to acquire sensor data. As will be described herein, this data may be saved for future use and the responsibility summary may be automatically updated as data is acquired from the sensor in the future.
Selecting the run responsibility 1436 may cause the responsibility to begin, or may cause the responsibility to be scheduled to begin, for example, at a particular date and time in the future. Further, if responsibilities are scheduled in the future, responsibilities can be manually or automatically selected to begin in advance via running responsibilities 1436 based on feedback. The select responsibility history 1438 may allow the user to display which responsibilities have been initiated, what data has been acquired, and any alarms or other feedback from the responsibilities. The selection responsibility history (validation) 1439 may allow for data to be validated manually or automatically. For example, in the case of acquiring an image of an analog sensor, it may be necessary to confirm the reading of the sensor.
The use of dashboard 1400 can allow a user to define, assign, and/or manage responsibilities in any of the manners described herein. It should be appreciated that some or all aspects of definition, assignment and/or management of responsibilities may be accomplished manually or automatically. Dashboard 1400 may also enable a user to confirm or revise any data received by the system. Alternatively, dashboard 1400 can be used only for administrators to view the status of all responsibilities.
Priority-based allocation responsibilities
Responsibility may be automatically assigned based on a number of factors in step 1240 of flowchart 1200 of fig. 12. The allocation of responsibilities is described, for example, with reference to flowchart 1600 in fig. 16. According to one approach, RSP 672 may order a set of responsibilities into a certain order based on a desired priority. As described herein, the desired priority may be based on the relative importance of the responsibilities, the timing of the responsibilities, the cost of the responsibilities, the desired quality required for the tasks of each responsibilities, and the like. The order of responsibilities may take into account more than one priority. In step 1610, factors, data, and responsibility abilities may be parsed from the responsibility summary. Then, in one exemplary method, each robot in the fleet may be evaluated one at a time. For example, in step 1620, the robot capabilities of the first robot may be compared to factors, data, and responsibility capabilities parsed from the first responsibility summary. In the event that there is a sufficient match between the robot capabilities and factors, data and responsibility capabilities, then in step 1630, a first responsibility may be assigned to the first robot. If there is a sufficient match with the first robot, then in step 1640, a determination may be made as to whether all responsibilities have been assigned. If any responsibilities remain to be assigned, flowchart 1600 returns to step 1620 until either all responsibilities have been assigned or it is determined that all responsibilities cannot be assigned (as described below).
In the event that a match is not sufficient, the next robot in the queue (e.g., the second robot) may be compared in step 1650 to determine if a sufficient match exists. If so, responsibility may be assigned to the second robot and the flowchart 1600 loops back to step 1640 to determine if there are any more responsibility assignments. If the determination in step 1650 is negative, flowchart 1600 flows to step 1670 to determine if any robots remain. If there are robots to be searched, all robots are searched to determine whether responsibilities can be completed by any robot. Each robot may be evaluated until responsibility is assigned or no robots remain.
In the event that a second role cannot be assigned to the robot, the robot previously assigned to the previous role (e.g., the first role) may be evaluated in step 1680. If a second responsibility can be assigned to the robot assigned to the first responsibility, then the second responsibility can be assigned to the first robot and the process can loop back to step 1620. If the second responsibility cannot be assigned to the previously assigned robot, the responsibility remains unassigned. If multiple responsibilities are assigned to a single robot, the method may select from any of a plurality of options, including leaving multiple responsibilities for a single robot and scheduling them at different times, or determining if any other unassigned robot is capable of completing one of the assigned responsibilities.
Once all responsibilities have been evaluated, a schedule of responsibilities assigned to the robot may be determined. In step 1698, the robot schedule may be output, for example, as described with reference to step 1260 in flowchart 1200 of fig. 12.
Exemplary method of managing a robot team
Referring to flow chart 1800 in fig. 18, an example of defining, assigning, and managing daily shifts is shown with reference to fig. 17. For example, as an alternative to fully automating allocation of responsibilities to robots as described above, the administrator 1801 or other personnel may assist in allocating one or more responsibilities to robots and/or human operators. This may be done wirelessly or via a wired connection. For example, the robot may include a GUI or keyboard for inputting data, the robot may include one or more data ports for receiving wired transmissions, and/or the robot may receive data via wireless communications.
In step 1810, manager 1810 may access a plurality of responsibilities. The administrator 1801 may then initiate a Field Assistant (FA) program/module 1802 to create a responsibility plan to cause the responsibility group to examine the facility.
In step 1825, the administrator may first determine if the responsibilities are appropriate for the person or robot. Any of the steps described above with reference to fig. 12 and/or fig. 16 may be used by the administrator in this determination. The administrator may work with the aid of a computer program or algorithm.
If it is determined that the task is appropriate for a person, the flow diagram 1800 may proceed to step 1830. The human operator 1806 may download a list of responsibilities assigned to the human operator 1806. For example, the human operator 1806 may access a portable device that displays responsibilities. Once the responsibilities have been downloaded, the human operator 1806 may select and execute the responsibilities in step 1840. The portable device may accompany the human operator 1806 when performing duties. In this way, the human operator 1806 may provide real-time or intermittent feedback to RSP 1803 (e.g., RSP 672 in fig. 6) based on the responsibilities performed. The human operator 1806 may also provide additional feedback, for example, based on five sensations of its own or based on other sensors available to the human operator. As described herein, this feedback may be used to automatically update future responsibility summaries and/or reassign robots that are scheduled to perform other responsibilities.
In the event that determination is made in step 1825 that the task is a robotic task, the flow diagram 1800 proceeds to step 1850. In step 1850, the RSP 672 periodically refers to the responsibility information on the FA 1802 to determine which tasks have been assigned to the robot. In some cases, the RSP 1803 may query the FA 1802 continuously, or the RSP 1803 may be scheduled to query the FA 1802 at certain intervals. Alternatively, the FA 1802 can push the task to the RSP 1803 when the task is scheduled to begin.
Once the responsibility information is relayed to the RSP 1803, in step 1860, the responsibility assigned to the robot during the shift can be downloaded from the RSP 1803 to the robot 1805. Responsibilities may include a summary of responsibilities with any information (e.g., factors, data, and/or responsibilities) described herein.
In step 1870, RSP 1803 may receive robotic capabilities from a robot or other system. The robotic capabilities may include any such robotic capabilities discussed in this disclosure. For example, the responsibility capability may include the status of the robot 1805, such as all system functions, battery power, kits associated with the robot, and any other information. In response to this information, the RSP 1803 may send responsibility information, such as certain triggers for performing the responsibility and/or other information, to the robot control system 1804 (e.g., such as the robot team management 480 in fig. 4).
In step 1890, the robot control system 1804 may send responsibility information to the robot. This information may be transmitted wirelessly via communication nodes throughout the facility or via wired connections (e.g., via data ports in the robot). In this case, RSP 1803 may control the robot to perform responsibilities via robot control system 1804.
Referring to flowchart 2000 in fig. 20, an example of assigning and managing responsibilities is shown with reference to fig. 19. For example, in step 2005, a human manager or automation system (e.g., RSP 672) determines whether the responsibility is to assign a task to robot 1805 or person 1806. The determination may be made based on the responsibility summary or may be made based on historical data. For example, a responsibility summary may indicate one or more responsibility abilities to complete a task that a person's ability may satisfy.
If the responsibilities are designated as those of person 1806, person 1806 may assume the task of ensuring that it has the correct toolkit or other required equipment, including any sensors and/or other tools for performing the responsibilities, in step 2010. In addition, person 1806 may then perform the duties and determine the results of the duties' tasks. Once responsibility is complete, the person 1806 can upload the task results to the RSP 1803 (e.g., RSP 672), the operations management system 700, or another remote system.
If the responsibility is the responsibility assigned to the robot 1805, then in step 2030, the robot 1805 may perform the responsibility based on the responsibility summary. For example, the robot may parse information from the responsibility summary, including factors, data, and/or responsibility capabilities associated with the responsibility, such as whether the sensor should be imaged, whether a valve should be opened, and so forth.
After the robot 1805 performs the duties, the robot may transmit the results of the duties' tasks to the robot control system 1804 in step 2040. For example, robot 1805 may transmit image data with information related to sensor readings, or information confirming that the valve is open. This information may be sent wirelessly from the task location, or may be sent via a wired connection when the robot 1805 returns to the home base.
In step 2060, RSP 1803 may store any information, such as task results, in database 1807. It should be appreciated that any information transmitted by the robot 1805 may be considered feedback, such as visual or audio information 1808, and may be used to define future responsibilities (e.g., factors or data for future responsibilities). The information received during step 2060 may also be used to determine whether additional responsibilities should be scheduled and the scope of those responsibilities.
In step 2070, the administrator 1801 may view the data transmitted by the robot 1805 in step 2050 and may confirm the results. For example, the administrator 1801 may view the image of the sensor and confirm that the reading was properly parsed from the image. Alternatively or additionally, the administrator 1801 may perform additional steps on the data, such as entering the data into a database or transmitting the data to another system. For example, in step 2080, the administrator 1801 may send the results of the task to a Field Assistant (FA) 1802, which may allow the user to view the results and/or may provide feedback for modifying and/or creating new responsibilities.
Referring to flowchart 2200 in fig. 22, an example of assigning and managing responsibilities is shown with reference to fig. 21. For example, in step 2202 of flowchart 2200, a human manager or automation system (e.g., RSP 672) determines whether it is responsibility to assign a task to robot 1805 or person 1806. The determination may be made based on the responsibility summary or may be made based on historical data. For example, a responsibility summary may indicate one or more responsibility abilities to complete a task that a person's ability may satisfy.
If responsibility is to be assigned to person 1806, then in step 2205, person 1806 may examine equipment, such as sensors, to determine the outcome of the task. For example, if the task is to determine a reading on a pressure sensor, the person 1806 may read and record the value of the sensor. The value may be recorded digitally, for example via a handheld electronic device or with a writing implement and paper.
In step 2210, the person 1806 may upload the results of the task. For example, person 1806 may upload the results from the handheld electronic device wirelessly or via a wired data port. Alternatively, person 1806 may enter the results to a terminal at the headquarter when responsibility is complete.
If it is determined at step 2202 that the role is for the robot 1805, then at step 2215 the robot 1805 may perform the role based on the role profile. For example, the robot 1805 may parse information from the responsibility summary, including factors, data, and/or responsibility capabilities associated with the responsibility, such as whether the sensor should be imaged, whether the valve position should be changed, and so forth.
After the robot 1805 performs the responsibilities, the robot may transmit the results of the responsibilities via the robot control system 1804 in step 2220. For example, robot 1805 may transmit image data with information related to sensor readings, or information confirming that a valve is activated. This information may be sent wirelessly from the task location, or may be sent via a wired connection when the robot 1805 returns to the home base. RSP 1803 (e.g., RSP 672) may receive the task result in step 2225.
In step 2230, RSP 1803 may store any information (e.g., task results) in database 1807. It should be appreciated that any information transmitted by the robot may be considered feedback and may be used to define future responsibilities. This information received during step 2225 may also be used to determine whether additional responsibilities should be scheduled and the scope of those responsibilities.
In step 2235, RSP 1803 may determine that data in the task result should be analyzed. For example, the data may be related to sensors, such as visual or audio data 1808, and may be analyzed by the administrator 1801.
In step 2240, the user application may return the results of the analysis. For example, the user application may automatically determine the sensor reading from the image data.
In step 2245, the administrator 1804 may examine the image data and may confirm the analysis from the user application. For example, the quality of the image data may be insufficient for the user application to provide an analysis with sufficient certainty.
In step 2250, the RSP 1803 may transmit the task result to the Field Assistant (FA) 1802 or the user application 2201. This may allow the user to view the results and/or may provide feedback for modifying responsibilities and/or creating new responsibilities.
Referring to flowchart 2400 in fig. 24, an example of assigning and managing responsibilities is shown with reference to fig. 23. For example, in step 2405, a human manager or automation system (e.g., RSP 672) determines whether the responsibility is to assign a task to the robot 1805 or the person 1806. The determination may be made based on the responsibility summary or may be made based on historical data. For example, a responsibility summary may indicate one or more responsibility abilities to complete a task that a person's ability may satisfy.
If responsibility is to be assigned to person 1806, then in step 2410, person 1806 may examine equipment, such as sensors, to determine the outcome of the task. For example, if responsibility includes determining a reading on the pressure sensor, person 1806 may read and record the value of the sensor. The value may be recorded digitally, for example via a handheld electronic device or with a writing implement and paper.
In step 2420, person 1806 may upload or transmit the task results. For example, person 1806 may release or transmit data related to the sensor value using a handheld device. Alternatively, person 1806 may access the terminal and transcribe data related to the sensor values. This information may be transmitted to a central location, such as a Field Assistant (FA) system 1802.
If it is determined in step 2405 that the role is that of the robot 1805, then in step 2430 the robot 1805 may perform the role, e.g., based on the role profile. For example, the robot 1805 may parse information from the responsibility summary, including factors, data, and/or responsibility capabilities associated with the responsibility, such as whether the sensor should be imaged, whether a valve should be opened, and so forth.
After the robot 1805 performs the responsibilities, the robot 1805 may determine, in step 2440, a result of performing the task associated with the responsibilities. For example, the sensors 3210 and 3220 in fig. 32A and 32B, respectively, may be imaged. The robot 1805 may query the RSP 1803 to gain access to Artificial Intelligence (AI) capabilities 2401 for determining readings from the sensors 3210 and 3220, or any other image or audio data 1808 in fig. 23. Alternatively or additionally, the robot 1805 may load an AI program 2401 thereon for automatically determining the values of the sensors 3210, 3220 based on image analysis. In this way, the value of the sensor can be obtained.
After determining the task result, the robot 1805 may transmit image data having information related to the sensor readings in step 2450. This information may be sent wirelessly from the task location or may be sent via a wired connection when the robot returns to the home base. RSP 1803 may receive the responsibility results in step 2450. The information may include results (e.g., readings of the sensor) as well as any raw data or information collected during responsibility (e.g., image data of the sensor). It should be appreciated that any information transmitted by the robot may be considered feedback and may be stored in the database 1807 and used to define future responsibilities. This information received during step 2450 can also be used to determine whether additional responsibilities should be scheduled and the scope of those responsibilities. For example, feedback may be displayed in responsibility history 3300 in FIG. 33. The feedback may include information associated with the task, e.g., first information 3320 associated with reading the first sensor information and second information 3330 associated with reading the second sensor information. In some examples, responsibility history 3300 can also include robot information 3310. Alternatively or additionally, the information may be retrieved from a memory (e.g., the history store 612 in fig. 6). The robot data may be used to determine whether the robot is a suitable robot for performing this responsibility (e.g., whether the robot performs the responsibility or a task therein, has better or worse results, reliability, time or energy efficiency, etc., as compared to other robots), and may be used as feedback to update planned or future responsibilities, such as which robots are assigned to certain responsibilities.
In step 2460, RSP 1803 may receive information from the robot. In step 2470, RSP 1803 may store any information in the database. In step 2480, the RSP 1803 may transmit the responsibility results to the Field Assistant (FA) 1802. This may allow the administrator to view the results and/or may provide feedback for modifying responsibilities and/or creating new responsibilities.
Use of artificial intelligence and machine learning
Referring to flowchart 2500 in FIG. 25, historical data can be used to define responsibilities. For example, the memory may store any one or more of tasks, factors, data, and responsibility capabilities. In some cases, one or more responsibilities and tasks associated with those responsibilities may be stored in a memory (e.g., historian 612). In step 2510, tasks and/or responsibilities may be accessed from memory. Alternatively or additionally, tasks or responsibilities may be entered in the manner described with reference to step 1110 in FIG. 11.
After a task is established, factors related to the task (such as those described in step 1120) may be determined by accessing historical data associated with the task in step 2520. Alternatively or additionally, RSP 672 may access operations management system 700 or another database to determine whether factors associated with the task have been updated. For example, the task may include steps associated with finding and reading a manometer. The historical data may indicate that the manometer is an analog sensor. RSP 672 may access one or more databases and may determine that the manometer is updated to the digital sensor.
In step 2530, RSP 672 may access memory (e.g., historian 612) to determine historical data regarding data associated with factors of the task. Further, in the case of updating factors based on new information associated with responsibilities, the data may be automatically updated to include new data associated with these updated factors. For example, in the case where the manometer has changed from analog to digital, the data may include details for accessing information about the digital manometer.
In step 2540, the responsibility abilities available to complete the responsibility may be determined and/or updated. Responsibility capability may be determined by accessing historical data. Alternatively or additionally, responsibility capability may be automatically updated based on the revised factors and/or revised data. For example, if the manometer changes from an analog sensor to a digital sensor, the duty capacity for reading the sensor may change. In this case, the responsibility capability may be automatically updated based on information in the operations management system 700 and/or by accessing information on the internet, for example, via the manufacturer's website, etc.
In step 2550, the responsibility summary can be automatically updated based on any new information determined in steps 2510 to 2540.
In step 2560, a responsibility summary can be output as described in step 1150 of FIG. 11.
The flowchart 2600 of fig. 26 describes a method of generating an updated responsibility schedule based on historical data. One or more responsibilities, each having a summary of responsibilities, as depicted at step 2610 in fig. 26, may be received. As described herein, the responsibility summary can be based on the responsibility selected by the user. Alternatively or additionally, in step 2610, responsibility and responsibility summaries may be determined using artificial intelligence and machine learning employing historical data. For example, the responsibilities of the maintenance facility may be assigned by a person on a predetermined basis or automatically by the operations management system or RSP 672. However, once these responsibilities are first scheduled, RSP 672 may automatically receive these responsibilities from memory (e.g., historian 612) in step 2610. Further, responsibilities may be selected based on new job instructions. For example, responsibilities with a single task (e.g., inspection sensor, actuation button, turn lever, etc.) may be adjusted or combined to create responsibilities with multiple tasks based on one or more task requests.
In some cases, RSP 672 may record the success rate and/or completion time of one or more responsibilities supported by the various robot types. RSP 672 may adjust the allocation of responsibilities based on this historical data by revising the allocation algorithm to allocate responsibilities to robots to perform responsibilities with higher success rates, to complete responsibilities faster, etc. In some examples, if, for example, a picture is taken at a sub-optimal resolution, it may be determined that the responsibility is inconclusive or deemed to be failed. In this case, RSP 672 may modify the assignment algorithm to schedule similar responsibilities to robots with better imaging packages. RSP 672 may also select the power consumption of all robots by adjusting the allocation algorithm to allocate responsibilities to robots with lower or higher power outputs based on the power requirements of each robot. For example, if high quality images are desired, but these robots require high power, assigning responsibilities to the robots to deliver high quality images may increase the overall power of the robots.
Further, in step 2620, feedback may be sent from sensors and robots distributed throughout the facility to RSP 672. Based on this feedback, responsibilities can be generated to address maintenance issues that arise on a scheduled basis. For example, it may be determined that condensation forms on a particular pipe and water continuously forms on the floor below the pipe. Feedback from the sensors or robots can provide this information to RSP 672 and can define responsibilities (e.g., in flowchart 1100 of fig. 11) to purify water on a predetermined basis. For example, having water on the floor may change environmental factors, which may affect the overall fleet schedule. In this case, it may be specified to perform cleaning of the water on the floor prior to other tasks in that area of the facility.
In step 2630, it may be determined whether the new responsibilities should occur within a responsibilities schedule (e.g., responsibilities that have been previously scheduled). If new responsibilities can occur within the schedule, it can be determined where in the schedule the new responsibilities should occur. For example, as described above, cleaning water on the floor may be scheduled before other nearby tasks. Alternatively, flowchart 2600 may proceed to step 2660.
In step 2640, it may be determined where in the schedule new responsibilities should occur. For example, this may be determined based on the responsibility factors, data, and/or responsibility capabilities in the already scheduled responsibilities and the new responsibilities. For example, responsibilities may require traversing the area of water on the floor, and the assigned robot cannot traverse the water. The duty of the cleaning water may be scheduled in advance.
In step 2650, responsibilities may be reassigned or rescheduled as described herein (see, e.g., fig. 28). Once responsibilities have been rescheduled, a revised responsibilities schedule can be generated, for example, as described in step 2860 of FIG. 28. Alternatively, a new responsibility schedule may be generated, wherein it is determined that the responsibilities are not part of the previously scheduled responsibilities.
Use of feedback for defining, assigning and managing responsibilities
Referring to flowchart 2700 in fig. 27, the robot may receive feedback during responsibility. During responsibility, one or more sensors or other devices for collecting information may provide feedback in step 2710. These sensors/devices may include image sensors, pressure sensors, or any other device. The sensors may be sensors associated with the facility and/or with one or more robots within the facility. These sensors may monitor the state, capabilities, or other information of the robot directly (e.g., sensors mounted on the robot) or indirectly (e.g., sensors associated with the facility). The feedback may include robot-specific feedback. For example, the robot may provide battery status, health status (e.g., sensor failure), location within the facility, or other similar feedback. The sensor may also determine environmental information of the facility. For example, the feedback may include the pressure or temperature of the environment. The feedback may also include the location of one or more objects within the facility. For example, the imaging or positioning device may determine the position of a robot, person, or other object within the facility. The person may also provide feedback directly or indirectly. For example, a person may provide feedback via a portable handheld device or via a fixed terminal within the facility. Alternatively or additionally, the sensor may be attached to the person and/or may be embedded within the person's clothing or equipment. In this way, robots and facilities may be monitored and responsibilities and/or responsibilities schedules may be altered as desired.
In step 2720, feedback may be transmitted from the sensor or other device to RSP 672. Data can be transmitted directly or indirectly from the sensor to RSP 672. For example, as will be described herein, the sensor may be connected to RSP 672 via a wireless or wired connection. In some cases, the wireless connection may be intermittent due to a predetermined communication schedule, or due to interference from objects throughout the facility. In this case, feedback may be transmitted indirectly from the sensor to RSP 672. Such indirect communication may be via another robot, person, or other communication interface.
In step 2730, RSP 672 and/or the robot may receive feedback. The user may monitor the feedback and may change the responsibilities or responsibilities schedule, or the responsibilities or responsibilities schedule may be automatically changed by the robot or RSP 672. For example, a robot that receives feedback may automatically change responsibilities based on the feedback. Altering responsibilities or a responsibilities schedule (e.g., fig. 11 or 25) based on feedback is described herein.
In step 2740, RSP 672 may store the feedback in a storage device. This feedback can be used to define future responsibilities. For example, the environmental data in the responsibility summary may be updated based on recent feedback from a particular location. In other examples, the temperature of the facility may change consistently throughout the day, and the environmental data may be updated based on the timing.
In step 2750, RSP 672 may transmit the modified responsibilities and/or modified responsibilities schedule to one or more robots. For example, based on the received feedback, the responsibilities and/or the responsibilities schedule may be altered, and the revised responsibilities summary may be transmitted to the robot accordingly. RSP 672 may transmit the modified responsibilities or responsibilities schedule to the robot in any of the manners described herein.
Referring to flow chart 2800 of fig. 28, a change in responsibilities and/or a responsibilities schedule can be modified based on feedback. For example, the flowchart may be adapted to alter the responsibilities or responsibilities schedule from step 2550 in fig. 25.
In step 2810, feedback is received by the robot or RSP 672. The feedback may include information about the tasks, data, responsibility or factors of the robot's ability. In step 2820, RSP 672 determines whether the feedback is feedback related to one or more of the factors of the task, data related to the factors, responsibility or robotic capabilities, or a combination of these.
In step 2830, RSP 672 determines which responsibilities the feedback affects. For example, the feedback may include factors of the task. In this case, a single responsibility, such as the responsibility associated with the task, may be the only responsibility affected by the feedback. If the feedback includes environmental data, the feedback may affect multiple responsibilities. For example, if the pressure in a certain area of a facility changes, this may affect a number of responsibilities associated with that area. Feedback may also include inability to reach a particular location within the facility (e.g., based on size or facility limitations), someone working in a particular vicinity (e.g., work performed by the robot may cause injury to the person), equipment damage or malfunction (e.g., valves, sensors, buttons, etc.), environment (snow, rain, gas, sunlight, etc.), and so forth. Responsibilities may also be application dependent, e.g., responsibilities may be assigned to a subset of robots based on type of responsibilities (robot a may be assigned to responsibilities that include button actuation, robot B may be assigned to responsibilities that include moving levers, etc.). Feedback and historical data may be used to optimize future responsibilities, e.g., reassign future responsibilities to different robots, such that a future responsibilities schedule may be based on robot usage to improve efficiency, reduce cost, and energy usage, etc.
In step 2840, the responsibility summary for each responsibility affected by the feedback is modified by RSP 672. For example, the affected parts of the responsibility summary (e.g., factors, environmental data, responsibility or robotic capabilities) are modified.
In step 2850, RSP 672 determines whether the modification of the responsibility summary affects the responsibility schedule, e.g., allocation of robots, timing of responsibilities, etc. If the answer is no, operation proceeds to step 2870. If the modification of the responsibility summary does affect the responsibility schedule, then flowchart 2600 flows to step 2860.
In step 2860, the responsibility schedule is revised. For example, responsibilities may be reassigned, timing changes for responsibilities, etc., as described in flowchart 2600 of fig. 26.
In step 2870, a responsibility schedule or revised responsibility schedule is generated and transmitted to the robot. For example, the responsibility schedule may be generated and transmitted as described in step 1260 of fig. 12.
Method for reassigning responsibilities
Referring to flow chart 2900 in FIG. 29, a method of reassigning responsibilities is described. In step 2910, feedback may be received in any manner described in flowchart 2700 of fig. 27 or flowchart 2800 of fig. 28.
In step 2920, RSP 672 may receive one or more responsibilities or one or more responsibilities schedules. This may include any responsibilities obtained during step 1210 in the assigned responsibilities flowchart 1200 of fig. 12, or any responsibilities schedule that generates output in step 1260. Alternatively or additionally, in step 2920, a responsibility schedule generated by the remote system or received from the person, for example, via an input device, may be received.
In step 2930, RSP 672 determines, based on the feedback, whether any parameters (factors, data, responsibility capabilities) of any responsibility summary have been changed. If the responsibility summary is not affected, flowchart 2900 proceeds to step 2970. If at least one responsibility summary is affected, flowchart 2900 proceeds to step 2940.
In step 2940, RSP 672 modifies and updates one or more of the responsibility summaries. For example, the environmental data feedback may alter the environmental data in one or more responsibility summaries.
In step 2950, RSP 672 determines whether the change responsibility summary changes the responsibility schedule. For example, RSP 672 may determine whether one or more robots assigned to each role may continue to perform the role based on the same role schedule. If changing the responsibility summary does not change the responsibility schedule, flow 2900 proceeds to step 2970. If changing the responsibility summary does change the responsibility schedule, flow 2900 proceeds to step 2960.
The responsibility schedule may be altered or updated in step 2960. For example, responsibilities may be assigned to a responsibilities schedule in the manner described in flowchart 1200 of fig. 12 and/or flowchart 2600 of fig. 26. As in step 1260 of fig. 12, a new or revised responsibility schedule may be output and assigned to the robot.
In step 2970, a confirmation message or alert may be transmitted to all robots, administrators' displays, etc. in the responsibility schedule to confirm that the responsibility schedule has not been revised. In addition, a responsibility summary of one or more responsibilities that were revised but did not affect the responsibility schedule may be transmitted to the corresponding robot. Alternatively, RSP 672 may determine that no acknowledgement message needs to be sent to the robot and/or administrator or other personnel to attempt to reduce the amount of data transferred. In some cases, responsibilities may be designed such that the robot has limited communication with RSP 672 to reduce costs, etc. In this case, the robot may be instructed to operate independently (based on the responsibility summary and/or the responsibility schedule) to perform one or more responsibilities unless an alarm or emergency occurs. In some examples, status checks may be sent to or from RSP 672 periodically to inform RSP 672 of the status of the robot and/or responsibilities. Based on this feedback, the schedule may be modified as described herein.
Another method of reassigning responsibilities is described with reference to flow chart 3400 in fig. 34.
In step 3410, the first robot may begin performing primary responsibilities. In step 3420, the first robot may receive feedback in any manner described herein. Based on the feedback, the first robot may determine an action plan in step 3430. For example, the first robot may determine that it is unable to perform responsibilities based on feedback, or that it may be able to perform conditional responsibilities (received from RSP 672 or preloaded on the robot).
In step 3440, the robot completes the conditional responsibilities and begins executing the primary responsibilities again. In some cases, the robot may begin performing the primary responsibilities from the place where it left off, or may need to perform the entire primary responsibilities again.
Further, in step 3450, an alarm may be triggered, indicating an event occurring within the facility. In step 3460, it is determined whether the first robot or the one or more second robots should perform the alarm-generated responsibilities based on, for example, the proximity of the robots to the location, the priority assigned to the responsibilities of all robots, etc.
In step 3470, the primary responsibility may be performed by the first robot. The first robot may begin another duty, or may return to the shelter or other "base" area (the return to the base may be included in the primary duty, or may be a separate duty preprogrammed into the robot).
While exemplary control systems for defining, distributing and managing robots and robot fleets have been described, it should be understood that the specific arrangement of elements in these systems is not limited. As described herein, a control system for defining, distributing and managing robots and robot teams is included. Defining, assigning, and managing various robots may be improved by the systems and methods described herein.
It will be apparent to those skilled in the art that various modifications and variations can be made to the disclosed systems and methods without departing from the scope of the disclosure. It is to be appreciated that the disclosed systems may include a variety of suitable computer systems and/or computing units incorporating a plurality of hardware components, such as processors and non-transitory computer readable media, for example, that allow the systems to perform one or more operations during a method according to the description herein. Other aspects of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the specification and examples be considered as exemplary only.
It should be appreciated that the various systems may include any computing device. The computing device may include input and output ports that connect to input devices and output devices such as a keyboard, a mouse, a touch screen, a monitor, a display, and the like. Of course, various system functions may be implemented in a distributed fashion across multiple similar platforms to distribute processing load. Alternatively, the system may be implemented by appropriate programming of a computer hardware platform.
In one embodiment, any of the disclosed systems, methods, and/or graphical user interfaces may be performed or implemented by a computing system consistent with or similar to the description herein. Although not required, aspects of the disclosure are described in the context of computer-executable instructions, such as routines executed by a data processing apparatus (e.g., a server computer, wireless device, and/or personal computer). Those skilled in the relevant art will appreciate that aspects of the disclosure may be practiced with other communications, data processing, or computer system configurations, including: internet appliances, hand-held devices (including personal digital assistants ("PDAs")), wearable computers, all forms of cellular or mobile phones (including voice over IP ("VoIP") phones), dumb terminals, media players, gaming devices, virtual reality devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, set top boxes, network PCs, minicomputers, mainframe computers, and the like. Indeed, the terms "computer," "computing device," and the like are generally used interchangeably herein and refer to any of the above-described devices and systems, as well as any data processor.
Aspects of the disclosure may be embodied in special purpose computers and/or data processors that are specially programmed, configured, and/or constructed to perform one or more of the computer-executable instructions explained in detail herein. While aspects of the disclosure, such as certain functions, are described as being performed on only a single device, the disclosure may also be practiced in distributed environments where functions or modules are shared among different processing devices that are linked through communication networks, such as a local area network ("LAN"), a wide area network ("WAN"), and/or the internet. Similarly, the techniques presented herein involving multiple devices may be implemented in a single device. In a distributed computing environment, program modules may be located in both local and/or remote memory storage devices.
Aspects of the present disclosure may be stored and/or distributed on non-transitory computer readable media including magnetically or optically readable computer disks, hard-wired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, biological memory, or other data storage media. Alternatively, computer-implemented instructions, data structures, screen displays, and other data according to aspects of the present disclosure may be distributed over the internet and/or other networks (including wireless networks) for a period of time on a propagated signal (e.g., electromagnetic waves, acoustic waves, etc.) on a propagated medium, and/or they may be provided on any analog or digital network (packet switched, circuit switched, or other scheme).
Program aspects of the technology may be considered to be "products" or "articles of manufacture" typically in the form of executable code and/or associated data carried on or embodied in a type of machine-readable medium. "storage" type media includes any or all of the tangible memory of a computer, processor, etc., or its associated modules, such as various semiconductor memories, tape drives, disk drives, etc., which can provide non-transitory storage for software programming at any time. All or part of the software may sometimes communicate over the internet or various other telecommunications networks. Such communication may enable, for example, loading of software from one computer or processor to another computer or processor, such as from a management server or host computer of a mobile communication network to a computer platform of a server and/or from a server to a mobile device. Thus, another type of medium that may carry software elements includes optical, electrical, and electromagnetic waves, such as those used over wired and optical landline networks, and over various air links, on physical interfaces between local devices. Physical elements carrying such waves, such as wired or wireless links, optical links, etc., may also be considered as media carrying software. As used herein, unless limited to a non-transitory tangible "storage" medium, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.
It will be apparent to those skilled in the art that various modifications and variations can be made in the disclosed systems, methods, and apparatus without departing from the scope of the disclosure. Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the invention being indicated by the following claims.
Claims (20)
1. A system for management of a fleet of robots, the system comprising:
a plurality of robots including a first robot of a first robot type and a second robot of a second robot type; and
a robot service platform, the robot service platform comprising:
a first data adapter corresponding to the first robot type and configured to convert responsibility feedback collected by the first robot into a generic data format compatible with the robot service platform;
A second data adapter corresponding to the second robot type and configured to convert responsibility feedback collected by the second robot into the generic data format;
a first control adapter corresponding to the first robot type and configured to convert generic role control information into first robot role control information compatible with the first robot;
a second control adapter corresponding to the second robot type and configured to convert the generic role control information into second robot role control information compatible with the second robot; and
a responsibility manager configured to:
selecting the first robot from a plurality of robot responsibilities based on its capabilities and characteristics of the selected robot responsibilities;
transmitting first robot role control information to the first robot via the first control adapter, the first control adapter having converted the first generic robot role control information into the transmitted first robot role control information; and
Responsibility feedback of the first data adapter from data transformations obtained by the first robot is received from the first data adapter.
2. The system for management of a robotic team as in claim 1, wherein the responsibility manager is further configured to:
selecting the second robot from the plurality of robots to perform the selected robot responsibilities based on the capabilities of the second robot and the characteristics of the selected robot responsibilities;
transmitting second robot role control information to the second robot via the second control adapter, the second control adapter having converted the first generic robot role control information into the transmitted second robot role control information; and
responsibility feedback collected by the second robot is received via the second data adapter.
3. A system for management of a robotic team as in claim 2, wherein the responsibility manager considers the first robot and the second robot to perform at least a portion of the selected robot responsibilities and selects the second robot to perform the at least a portion of the selected robot responsibilities in place of the first robot.
4. A system for management of a robotic team as in claim 2, wherein the responsibility manager considers the first robot and the second robot to perform at least a portion of the selected robot responsibilities and selects the second robot to perform the at least a portion of the selected robot responsibilities in cooperation with the first robot.
5. The system for management of a robotic team as in claim 2, wherein the second data adapter is different from the first data adapter and the second control adapter is different from the first control adapter.
6. The system for management of a robotic team as in claim 1, wherein the received responsibility feedback includes one or more of the following information: information that the selected robot responsibility is fully completed, information that the selected robot responsibility is partially completed, information that the selected robot responsibility is not initiated, health information about the first robot, status information about the first robot, capability information about the first robot, photographs, videos, environmental data, sensor readings, movement data about the first robot, and position data about the first robot.
7. The system for management of a robotic team as in claim 1, wherein the first robotic responsibility control information transmitted to the first robot includes one or more of a multi-step responsibility instruction or a separate operation instruction including one or more of: navigating to a destination, taking a photo, recording video, recording sound, measuring environment, measuring temperature of a substance, measuring temperature of air, measuring humidity, determining instrument readings, measuring presence or concentration of a gas or chemical, emitting light of a specific wavelength, intensity and/or emission pattern, emitting sound of a specific tone, intensity and/or emission pattern, emitting a radio frequency homing beacon, transmitting stored data via a radio frequency or wireless data network, connecting to a power source, connecting to a radio frequency or wireless data network, connecting to a data network port, modifying settings of a valve or other control, adjusting the position of a valve, blade, locally controlled pump or actuator, taking a product sample, pressing a button, changing a switch position, moving an object, stopping all operations, returning to an original position or activating a sensor.
8. A robotic service platform for management of a robotic team, the robotic service platform configured to:
selecting a first robot from the plurality of robots to perform robot responsibilities;
selecting a first data adapter from a plurality of data adapters corresponding to a first robot type of the first robot, the first data adapter configured to convert responsibility feedback collected by the first robot into a generic data format;
selecting a first control adapter corresponding to the first robot type from a plurality of control adapters, the first control adapter configured to convert generic role control information into first robot role control information compatible with the first robot;
transmitting the first robot role control information to the first robot via the first control adapter, the first control adapter having converted the first generic robot role control information into the transmitted first robot role control information; and
responsibility feedback collected by the first robot is received via the first data adapter.
9. A robotic service platform for management of a robotic team as in claim 8, the robotic service platform further configured to:
Selecting a second robot from the plurality of robots to perform the robot responsibilities;
selecting a second data adapter from the plurality of data adapters corresponding to a second robot type, the second data adapter configured to convert responsibility feedback collected by the second robot into the generic data format, wherein the second data adapter is different from the first data adapter;
selecting a second control adapter corresponding to the second robot type from the plurality of control adapters, the second control adapter configured to convert generic role control information into second robot role control information compatible with the second robot, wherein the second control adapter is different from the first control adapter;
transmitting the second robot role control information to the second robot via the second control adapter, the second control adapter having converted the first generic robot role control information into the transmitted second robot role control information; and
responsibility feedback collected by the second robot is received via the second data adapter.
10. A robot service platform for management of a robotic team as recited in claim 9, wherein the robot service platform considers the first robot and the second robot to perform at least a portion of the selected robot responsibilities and selects the second robot to perform the at least a portion of the robot responsibilities in place of the first robot.
11. A robot service platform for management of a robotic team as recited in claim 9, wherein the robot service platform considers the first robot and the second robot to perform at least a portion of the selected robot responsibilities and selects the second robot to perform at least a portion of the robot responsibilities in cooperation with the first robot.
12. A robot service platform for management of a robotic team as recited in claim 8, wherein the robot service platform selects the first robot to perform the robotic responsibility based on the capabilities of the first robot and the characteristics of the robotic responsibility.
13. A robot service platform for management of a robotic team as recited in claim 9, wherein the robot service platform selects a second robot to perform the robotic responsibility based on the capabilities of the second robot and characteristics of the robotic responsibility.
14. A robot service platform for management of a robotic team as recited in claim 8, wherein the received responsibility feedback collected by the first robot includes one or more of the following information: information that the selected robot responsibility is fully completed, information that the selected robot responsibility is partially completed, information that the selected robot responsibility is not initiated, health information about the first robot, status information about the first robot, capability information about the first robot, photographs, videos, environmental data, sensor readings, movement data about the first robot, and position data about the first robot.
15. A robot service platform for management of a robot team as recited in claim 8, wherein the first robot responsibility control information transmitted to the first robot includes one or more of a multi-step responsibility instruction or a separate operation instruction including one or more of: navigating to a destination, taking a photo, recording video, recording sound, measuring environment, measuring temperature of a substance, measuring temperature of air, measuring humidity, determining instrument readings, measuring presence or concentration of a gas or chemical, emitting light of a specific wavelength, intensity and/or emission pattern, emitting sound of a specific tone, intensity and/or emission pattern, emitting a radio frequency homing beacon, transmitting stored data via a radio frequency or wireless data network, connecting to a power source, connecting to a radio frequency or wireless data network, connecting to a data network port, modifying settings of a valve or other control, adjusting the position of a valve, blade, locally controlled pump or actuator, taking a product sample, pressing a button, changing a switch position, moving an object, stopping all operations, returning to an original position or activating a sensor.
16. An adapter for management of a robotic fleet, comprising:
a data adapter corresponding to a first robot type and configured to convert data collected by the first robot into a generic data format compatible with a robot service platform; and
a control adapter corresponding to the first robot type and configured to convert generic role control information into robot role control information compatible with the first robot.
17. The adapter of claim 16, wherein the responsibility feedback collected by the first robot comprises one or more of the following information: information that robot responsibilities are fully completed, information that the robot responsibilities are partially completed, information that the robot responsibilities are not initiated, health information about the first robot, status information about the first robot, capability information about the first robot, photographs, videos, environmental data, sensor readings, movement data about the robot, and position data about the robot.
18. The adapter of claim 16, wherein the robot duty control information comprises one or more of a multi-step duty instruction or a separate operation instruction comprising one or more of: navigating to a destination, taking a photo, recording video, recording sound, measuring environment, measuring temperature of a substance, measuring temperature of air, measuring humidity, determining instrument readings, measuring presence or concentration of a gas or chemical, emitting light of a specific wavelength, intensity and/or emission pattern, emitting sound of a specific tone, intensity and/or emission pattern, emitting a radio frequency homing beacon, transmitting stored data via a radio frequency or wireless data network, connecting to a power source, connecting to a radio frequency or wireless data network, connecting to a data network port, modifying settings of a valve or other control, adjusting the position of a valve, blade, locally controlled pump or actuator, taking a product sample, pressing a button, changing a switch position, moving an object, stopping all operations, returning to an original position or activating a sensor.
19. The adapter of claim 16, wherein the adapter is configured to communicate with a particular type of robot or a robot produced by a single manufacturer.
20. The adapter of claim 16, wherein the adapter is configured to communicate with multiple types of robots or robots produced by multiple manufacturers.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US63/152734 | 2021-02-23 | ||
| US17/245104 | 2021-04-30 | ||
| US17/245,104 US20220269284A1 (en) | 2021-02-23 | 2021-04-30 | Systems and methods for management of a robot fleet |
| PCT/IB2022/051425 WO2022180487A1 (en) | 2021-02-23 | 2022-02-17 | Systems and methods for management of a robot fleet |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117255978A true CN117255978A (en) | 2023-12-19 |
Family
ID=89128098
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202280029914.3A Pending CN117255978A (en) | 2021-02-23 | 2022-02-17 | System and method for management of a robot team |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117255978A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117707053A (en) * | 2024-02-05 | 2024-03-15 | 南京迅集科技有限公司 | Industrial control visual movement control system and method based on AI visual analysis |
-
2022
- 2022-02-17 CN CN202280029914.3A patent/CN117255978A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117707053A (en) * | 2024-02-05 | 2024-03-15 | 南京迅集科技有限公司 | Industrial control visual movement control system and method based on AI visual analysis |
| CN117707053B (en) * | 2024-02-05 | 2024-04-26 | 南京迅集科技有限公司 | Industrial control visual movement control system and method based on AI visual analysis |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11169651B2 (en) | Method and apparatus for controlling a process plant with location aware mobile devices | |
| US10913154B2 (en) | Systems and method for robotic learning of industrial tasks based on human demonstration | |
| US20220269285A1 (en) | Systems and methods for management of a robot fleet | |
| US20220300011A1 (en) | Systems and methods for management of a robot fleet | |
| WO2019234249A1 (en) | Mobile vehicles in manufacturing | |
| GB2513958A (en) | Supervisor engine for process control | |
| CN110134081B (en) | Control system based on robot capability model | |
| US20070282781A1 (en) | Method To Retrieve Data For An Equipment, Plant Or A Process | |
| CN117255978A (en) | System and method for management of a robot team | |
| US20240094712A1 (en) | Robot staging area management | |
| US20240189997A1 (en) | Building a robot mission based on a robot-agnostic graphical user interface (gui) | |
| US20240189996A1 (en) | Building a robot mission based on mission metrics | |
| US20240189998A1 (en) | Building a robot mission based on mission inventory | |
| WO2024121618A1 (en) | Systems, methods, and devices for building a robot mission | |
| INNO et al. | D1. 4a AUTOWARE Guidelines |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |