Classifications

• • A—HUMAN NECESSITIES
  • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
  • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
  • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
  • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
  • A47L9/2857—User input or output elements for control, e.g. buttons, switches or displays
• • A—HUMAN NECESSITIES
  • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
  • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
  • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
  • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
  • A47L9/2805—Parameters or conditions being sensed
  • A47L9/2826—Parameters or conditions being sensed the condition of the floor
• • A—HUMAN NECESSITIES
  • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
  • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
  • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
  • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
  • A47L9/2836—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means characterised by the parts which are controlled
  • A47L9/2852—Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles
• • A—HUMAN NECESSITIES
  • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
  • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
  • A47L9/00—Details or accessories of suction cleaners, e.g. mechanical means for controlling the suction or for effecting pulsating action; Storing devices specially adapted to suction cleaners or parts thereof; Carrying-vehicles specially adapted for suction cleaners
  • A47L9/28—Installation of the electric equipment, e.g. adaptation or attachment to the suction cleaner; Controlling suction cleaners by electric means
  • A47L9/2894—Details related to signal transmission in suction cleaners
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B19/00—Programme-control systems
  • G05B19/02—Programme-control systems electric
  • G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
  • G05B19/042—Programme control other than numerical control, i.e. in sequence controllers or logic controllers using digital processors
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
  • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
  • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
  • G05D1/0016—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement characterised by the operator's input device
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
  • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
  • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
  • G05D1/0044—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
  • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
  • G05D1/20—Control system inputs
  • G05D1/22—Command input arrangements
  • G05D1/221—Remote-control arrangements
  • G05D1/222—Remote-control arrangements operated by humans
  • G05D1/223—Command input arrangements on the remote controller, e.g. joysticks or touch screens
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
  • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
  • G05D1/20—Control system inputs
  • G05D1/22—Command input arrangements
  • G05D1/221—Remote-control arrangements
  • G05D1/225—Remote-control arrangements operated by off-board computers
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
  • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
  • G06F3/0486—Drag-and-drop
• • A—HUMAN NECESSITIES
  • A47—FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
  • A47L—DOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
  • A47L2201/00—Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
  • A47L2201/04—Automatic control of the travelling movement; Automatic obstacle detection
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B2219/00—Program-control systems
  • G05B2219/20—Pc systems
  • G05B2219/25—Pc structure of the system
  • G05B2219/25419—Scheduling
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B2219/00—Program-control systems
  • G05B2219/30—Nc systems
  • G05B2219/45—Nc applications
  • G05B2219/45098—Vacuum cleaning robot

Definitions

• This specification relates generally to scheduling and control systems for autonomous cleaning robots.
• One exemplary system described herein is a mission timeline for scheduling and controlling a mobile cleaning robot.
• Cleaning robots include mobile robots that autonomously perform cleaning tasks within an environment, e.g., a home. Many kinds of cleaning robots are autonomous to some degree and in different ways.
• the cleaning robots include a controller that is configured to autonomously navigate the cleaning robot about the environment such that the cleaning robot can ingest debris as it moves.
• An autonomous cleaning robot may be controlled directly by a user (e.g. by pushing a button on the robot) or remotely (e.g. by a mobile application).
• a mobile application can be used to monitor the status of the autonomous cleaning robot as it executes a cleaning mission and make changes to the cleaning mission as it is executed.
• a mobile application may include a mission timeline displaying a status and schedule of the cleaning mission.
• a mobile application may also include a map to allow the user to visualize a location of the robot, which rooms have been cleaned, and which rooms are scheduled to be cleaned during the cleaning mission.
• the user can, through the mobile application, change cleaning parameters, scheduling, or a cleaning status of the autonomous cleaning robot.
• the autonomous cleaning robot executes the cleaning mission, operational events and a status of the robot may be displayed allowing the user to continue to monitor, and change if desired, the cleaning mission.
• an autonomous cleaning robot includes a drive configured to move the cleaning robot across a floor surface in an area to be cleaned and a controller.
• the controller is configured to receive data representing an editable mission timeline including data representing a sequence of rooms to be cleaned, navigate the cleaning robot to clean the rooms following the sequence, track operational events occurring in each of the rooms cleaned, and transmit data about time spent navigating each of the rooms navigated included in the sequence.
• the controller is configured to track time to complete cleaning each room included in the sequence and, upon completion of cleaning each room, transmit the time for display.
• the data representing a sequence of rooms to be cleaned comprises an indication of at least a first room to be cleaned and a second room to be cleaned and wherein the controller is configured to: clean the first room to be cleaned, transmit data about the time spent cleaning the first room, clean the second room to be cleaned, and transmit data about the time spent cleaning the second room.
• the controller is configured to, during a cleaning operation, receive a re-ordered sequence of rooms to be cleaned.
• the controller is configured to, subsequent to receiving a sequence of rooms to be cleaned, receive an indication to cancel cleaning a room in the sequence of rooms to be cleaned, and clean the rooms in the sequence except for the cancelled room.
• the controller is configured to transmit data representing a tracked operational event to be remotely represented on a graphical representation of an editable mission timeline.
• the controller is configured to transmit data representing a tracked operational event to be presented on a graphical user interface.
• tracking an operational event comprises tracking where the cleaning robot performed additional cleaning in response to detected dirt. In some implementations, tracking an operational event comprises detecting completion of cleaning a room.
• the operational event is a bin emptying event and the controller is further configured to compute an elapsed time to address the bin emptying event and to initiate transmission of data indicating the elapsed time.
• the operational event is a bin emptying event and the controller is further configured to initiate transmission of data indicating the operational event to be presented on a graphical representation of an editable mission timeline.
• the controller is configured to receive data indicating a set of cleaning parameters to be used in a room and the set of cleaning parameters includes at least one of a vacuum power, an edge cleaning setting, a multiple pass setting, and a wet or dry mopping parameter.
• the controller is configured to determine a charge level of a battery of the robot, transmit data representing the charge level, and receive a charging instruction comprising a charging period required to charge the battery to allow the cleaning robot to complete cleaning the rooms in the sequence.
• controller is configured to navigate the cleaning robot to a charging station based on the charging period and transmit data indicating an elapsed charging time for display.
• a handheld computing device includes one or more input devices, a display, and a processor.
• the processor is configured to initiate data transmission and reception with an autonomous cleaning robot, and present a first graphical representation of an editable mission timeline.
• the editable mission timeline includes an arrangement of a plurality of icons, each icon representing a room to be cleaned by the autonomous cleaning robot, to represent a sequence of rooms to be cleaned, and, for each icon of the plurality of icons, a corresponding representation to reflect a time spent cleaning each respective room.
• At least one of the plurality of icons are user-manipulatable, using the one or more input devices, to adjust the sequence of rooms to be cleaned. In some implementations, only the icons representing rooms remaining to be cleaned are user-manipulatable.
• the processor further configured to present a graphical representation of a map indicating rooms cleaned and rooms to be cleaned.
• one or more events detected by the autonomous cleaning robot are presented on the graphical representation of a map.
• an event detected by the autonomous cleaning robot includes an event where the robot detected excess dirt and performed additional cleaning.
• an event detected by the autonomous cleaning robot includes an event where the robot indicates completion of cleaning a room.
• a graphical representation of the event detected by the autonomous cleaning robot is inserted in the graphical representation of the editable mission timeline.
• the processor is configured to present the first graphical representation of an editable mission timeline for the autonomous cleaning robot and a second graphical representation of an editable mission timeline for another autonomous cleaning robot.
• the processor is configured to initiate
• the processor is configured to initiate receiving, from the robot, data representing an elapsed time spent cleaning at least one room and to present the elapsed time in the first graphical representation of an editable mission timeline.
• the processor is configured to initiate receiving, from the autonomous cleaning robot, data representing that the autonomous cleaning robot has completed cleaning a room, and to change an appearance of the icon corresponding to the cleaned room in the first graphical representation of the editable mission timeline.
• changing the appearance of the icon comprises changing a color, a shape, a size, or a position of the icon.
• the processor is configured to determine an area of interest from one or more images captured by the handheld computing device and transmit data representing a cleaning instruction for the area of interest to the robot.
• the user-manipulatable icons may be dragged and dropped within the editable mission timeline.
• the processor is further configured to initiate transmission of data representing a revised order of icons in the editable mission timeline.
• the user-manipulatable icons may be clicked on and removed from the editable mission timeline.
• the icons corresponding to cleaned rooms are fixed in the editable mission timeline.
• the mission timeline can display an estimated remaining cleaning time to a user.
• the mission timeline allows the user to remotely manipulate the order in which rooms are to be cleaned by the autonomous cleaning robot.
• a user may delete, re-order, or add rooms to the cleaning schedule of the cleaning robot on the mission timeline. For example, if a user is having guests over to their home and needs the kitchen, living room, and bathroom cleaned before guests arrive, the user can re-order the cleaning schedule and/or delete rooms except for these rooms such that the cleaning of these rooms is completed in time.
• the mission timeline can provide an interface that allows a user to select cleaning parameters for individual rooms and monitor operational events, like detecting dirt, occurring during cleaning.
• the user interface also includes a map for displaying cleaned rooms and locations of operational events.
• the robots and techniques described herein, or portions thereof, can be controlled by a computer program product that includes instructions that are stored on one or more non-transitory machine-readable storage media, and that are executable on one or more processing devices to control (e.g., to coordinate) the operations described herein.
• the robots described herein, or portions thereof, can be implemented as all or part of an apparatus or electronic system that can include one or more processing devices and memory to store executable instructions to implement various operations.
• FIG. 1 is a view of a system for controlling an autonomous cleaning robot using a handheld computing device.
• FIG. 2 is a view of an underside of an embodiment of an autonomous cleaning robot.
• FIG. 3 is a flow chart depicting a process to transmit data between a user interface and an autonomous cleaning robot.
• FIGS. 4A-4H are screenshots of an embodiment of a user interface for controlling an autonomous cleaning robot including an editable timeline.
• FIG. 5 is a screenshot of an embodiment of a user interface for controlling multiple autonomous cleaning robots.
• FIGS. 6A-6B are screenshots of a user interface showing a cleaning map.
• FIG. 7A is a perspective view of a room with a dirty location.
• FIG. 7B is a screenshot of a user interface for directing the robot to clean the dirty location.
• An autonomous cleaning robot may be controlled directly by a user (e.g. by pushing a button on the robot) or remotely (e.g. by a mobile application).
• a mobile application can be used to change cleaning parameters, scheduling, or a cleaning status of the autonomous cleaning robot.
• the mobile application may include an editable mission timeline displaying a selection and an order of rooms to be cleaned along with time estimates for completing cleaning in each room. Based on the displayed order and time estimates, a user may change the order or selection of rooms or cleaning parameters for cleaning each room.
• the autonomous cleaning robot receives information from the mobile device and executes the cleaning mission summarized on the timeline, operational events and a status of the robot may be sent from the robot to be displayed such that the user can continue to monitor, and change if desired, the cleaning mission.
• an autonomous cleaning robot 102 is located on a floor surface 104 within a room 106.
• the autonomous cleaning robot 102 is configured to communicate with a handheld computing device 120.
• the handheld computing device 120 may be, for example, a smart phone, a tablet, a smart watch, a laptop computer, etc.
• the handheld computing device 120 includes a processor 114 configured to initiate data transmission and reception with the autonomous cleaning robot 102 (either directly or via the internet) and present a graphical representation of an editable mission timeline.
• An editable mission timeline is a user-manipulable graphical display showing a sequence of rooms to be cleaned. The timeline can optionally include associated time estimates to clean each space.
• the autonomous cleaning robot 102 includes a controller which is configured to receive data representing the editable mission timeline and navigate the autonomous cleaning robot 102 across the floor surface 104 and clean the rooms following the sequence of the mission timeline.
• the autonomous cleaning robot 102 includes a vacuum assembly 108 and uses suction to ingest debris 116 as the autonomous cleaning robot 102 traverses the floor surface 104.
• the autonomous cleaning robot 102 may be a mopping robot which may include a cleaning pad for wiping or scrubbing the floor surface 104.
• the robot 102 includes a body 200 movable across the floor surface 104 and is powered by a battery.
• the body 200 includes a front portion 202a that has a substantially rectangular shape and a rear portion 202b that has a substantially semicircular shape.
• the front portion 202a includes, for example, two lateral sides 204a, 204b that are substantially perpendicular to a front side 206 of the front portion 202a.
• the robot 102 includes a drive system including actuators 208a, 208b mounted in the body 200 and operably connected to the drive wheels 210a, 210b, which are rotatably mounted to the body 200.
• the robot 102 includes a controller 212 that operates the actuators 208a, 208b to autonomously navigate the robot 102 about a floor surface 104 during a cleaning operation.
• the robot 102 includes a caster wheel 211 that supports the body 200 (e.g. the rear portion 202b of the body 200) above the floor surface 104, and the drive wheels 210a, 210b support the front portion 202a of the body 200 above the floor surface 104.
• a vacuum assembly 108 is also carried within the body 200 of the robot 102, e.g., in the rear portion 202b of the body 200.
• the controller 212 operates the vacuum assembly 108 to generate the airflow 110 and enable the robot 102 to ingest the debris 116 during the cleaning operation.
• the airflow 110 generated by the vacuum assembly 108 is exhausted through a vent 213 in a rear portion 202b or a conduit connected to a cleaning head of the robot 102.
• the cleaning head includes, for example, one or more rollers (e.g. brushes or compliant rubber rollers) that engage the floor surface 104 and sweep the debris 116 into the cleaning bin 100.
• the airflow 110 exhausted to the cleaning head can further improve pickup of debris from the floor surface 104 by increasing an amount of airflow proximate the cleaning head to agitate the debris 116 on the floor surface 104.
• the cleaning robot 102 is a self-contained robot that autonomously moves across the floor surface 104 to ingest debris.
• the cleaning robot 102 for example, carries a battery to power the vacuum assembly 108.
• the cleaning head of the robot 102 includes a first roller 212a and a second roller 212b.
• the rollers 212a, 212b include, for example, brushes or flaps that engage the floor surface 104 to collect the debris 1 16 on the floor surface 104.
• the rollers 212a, 212b for example, counter rotate relative to one another to cooperate in moving debris 116 toward a plenum 112, e.g., one roller rotates counterclockwise while the other rotates clockwise.
• the plenum 112 in turn guides the airflow 110 containing the debris 116 into the cleaning bin 100. As described herein, during the travel of airflow 1 10 through the cleaning bin 100 toward the vacuum assembly 108, the debris 116 is deposited in the cleaning bin 100.
• the robot 102 to sweep debris 116 toward the rollers 212a, 212b, the robot 102 includes a brush 214 that rotates about a non-horizontal axis, e.g., an axis forming an angle between 75 degrees and 90 degrees with the floor surface 104.
• the brush 214 extends beyond a perimeter of the body 200 such that the brush 214 is capable of engaging debris 116 on portions of the floor surface 104 that the rollers 212a, 212b typically cannot reach.
• the brush 214 is capable of engaging debris 116 near walls of the environment and brushing the debris 116 toward the rollers 212a, 212b to facilitate ingestion of the debris 116 by the robot 102.
• the user can control the robot directly by pushing a button on the robot or can control the robot remotely via a mobile application on a mobile device.
• the user can modify when and where the robot cleans.
• the autonomous cleaning robot 102 navigates the home executing cleaning behaviors, while monitoring its status and transmitting status data to the mobile device.
• FIG. 3 is a flow chart depicting a process of exchanging information among an autonomous mobile robot, a cloud computing system, and a mobile device.
• a cleaning mission may be initiated by pressing a button on the robot 708 or may be scheduled for a future time or day.
• the user may select a set of rooms to be cleaned during the cleaning mission or may instruct the robot to clean all rooms.
• the user may also select a set of cleaning parameters to be used in each room during the cleaning mission.
• the autonomous cleaning robot 708 tracks (710) its status, including its location, any operational events occurring during cleaning, and a time spent cleaning.
• the autonomous cleaning robot 708 transmits (712) status data (e.g. one or more of location data, operational event data, time data) to a cloud computing system 706, which calculates (714), by a processor 742, time estimates for areas to be cleaned. For example, a time estimate could be calculated for cleaning room 106 by averaging the actual cleaning times for the room that have been gathered during multiple (e.g. two or more) prior cleaning missions for the room.
• the cloud computing system 706 transmits (716) time estimate data along with robot status data to a mobile device 704.
• the mobile device 704 presents
• the robot status data and time estimate data may be presented on the display of the mobile device as any of a number of graphical representations editable mission timeline and/or a mapping interface (as shown in FIGS. 4A-4H, 5, and 6A-6B).
• a user 702 views (720) the robot status data and time estimate data on the display and may input (722) new cleaning parameters or may manipulate the order or identity of rooms to be cleaned.
• the user 702 may, for example, delete rooms from the robot's 708 cleaning schedule. In other instances, the user 702, may, for example, select an edge cleaning mode or a deep cleaning mode for a room to be cleaned.
• the display of the mobile device 704 is updates (724) as the user inputs changes to the cleaning parameters or cleaning schedule. For example, if the user changes the cleaning parameters from single pass cleaning to dual pass cleaning, the system will update the estimated time to provide an estimate based on the new parameters. In this example of single pass cleaning vs. dual pass cleaning, the estimate would be approximately doubled.
• the cloud computing system 706 calculates (726) time estimates for areas to be cleaned, which are then transmitted (728) (e.g. by a wireless transmission, by applying a protocol, by broadcasting a wireless transmission) back to the mobile device 704 and displayed. Additionally, data relating to the calculated
• time estimates are transmitted (746) to a controller 730 of the robot. Based on the inputs from the user 702, which are received by the controller 730 of the robot
• the controller 730 generates (732) a command signal.
• the command signal commands the robot 708 to execute (734) a behavior, which may be a cleaning behavior.
• a behavior which may be a cleaning behavior.
• the controller continues to track (710) the robot's status, including its location, any operational events occurring during cleaning, and a time spent cleaning.
• live updates relating to the robot's status may be additionally provided via push notifications to a mobile device or home electronic system (e.g. an interactive speaker system).
• the controller 730 Upon executing (734) a behavior, the controller 730 checks (736) to see if the received command signal includes a command to complete the cleaning mission. If the command signal includes a command to complete the cleaning mission, the robot is commanded to return to its dock and upon return sends information to enable the cloud computing system 706 to generate (738) a mission summary which is transmitted to, and displayed (740) by, the mobile device 704.
• the mission summary may include a timeline and/or a map.
• the timeline may display, the rooms cleaned, a time spent cleaning each room, operational events tracked in each room, etc.
• the map may display the rooms cleaned, operational events tracked in each room, a type of cleaning (e.g. sweeping or mopping) performed in each room, etc.
• Operations for the process 700 and other processes described herein can be executed in a distributed manner.
• the cloud computing system 706, the mobile robot 708, and the mobile device 704 may execute one or more of the operations in concert with one another.
• Operations described as executed by one of the cloud computing system 706, the mobile robot 708, and the mobile device 704 are, in some implementations, executed at least in part by two or all of the cloud computing system 706, the mobile robot 708, and the mobile device 400.
• FIGS. 4A-4H are screenshots of an embodiment of a user interface for controlling an autonomous cleaning robot including an editable timeline.
• the user interface 300a includes a status indicator 302 and a cleaning mode indicator 304.
• the status indicator 302 indicates a current status of the autonomous cleaning robot 102.
• a cleaning mode indicator 304 may indicate an overall cleaning mode to be used during the cleaning mission.
• a cleaning mode may be, for example, a standard cleaning mode, a deep cleaning mode, a quick cleaning mode, etc.
• the user interface 300a also includes a clean button 306 which the user may select to start a cleaning mission for the autonomous cleaning robot 102.
• the user interface 300b displays, during a cleaning mission, the status indicator 302, a pause button 308, and an editable mission timeline 310.
• the status indicator 302 includes the current location of the autonomous cleaning robot 102, an elapsed time, and an estimated time remaining in the cleaning session.
• the robot sends information about a current cleaning time and location. This information is used to generate the information included in the status indicator.
• the elapsed time and remaining time are displayed in the status indicator.
• the elapsed time and remaining time in the status indicator are associated with the full cleaning mission (e.g. how long the robot has been cleaning in total for the mission and how long expected to clean all remaining areas on the schedule).
• the pause button 308 allows the user to pause the cleaning session.
• the editable mission timeline 310 includes room icons 312, room labels 314, room cleaning time estimates 315, and elapsed cleaning times 316.
• the user interface 300 displays an elapsed cleaning time 316 for that room adjacent to the room label 314.
• a room cleaning time estimate 315 is displayed in the editable mission timeline 310 to inform a user of an approximate time to clean the corresponding room.
• the time estimates 315 may be based on, for example, data received from the robot from previous cleaning missions, an input of approximate square footage of the space to be cleaned, a test run through all rooms for purposes of calculating time estimates, etc. Based on these time estimates 315, a user may find it desirable to re-order or remove rooms from the cleaning schedule represented by the editable mission timeline 310.
• the robot Upon completion of cleaning in one space and beginning to clean in another, the robot sends information about the time and new location to the cloud computing system and the cloud computing system sends information to the mobile device to update the displayed timeline.
• the data can cause the mobile device to update the timeline to highlight the current space and provide the total time spent cleaning the prior space, replacing the previously provided estimate.
• the icons 312 are user-manipulatable (e.g. by dragging and dropping) using an input device, for example, a touch screen of the handheld computing device 120 on which the user interface 300b is displayed. As such, the user can easily re-order or remove rooms from the cleaning schedule while the autonomous cleaning robot 102 is cleaning.
• the icons 312 may be selectable by the user and allow the user to select one or more cleaning parameters for the corresponding room.
• a cleaning parameter may be, for example, a vacuum power level, a single versus a multiple pass setting, an edge cleaning setting, a wet versus dry cleaning setting, etc.
• the user interface 300c shows the editable mission timeline 310 where the icon 312b, representing the dining room, has been deleted by the user. For example, the user can select the icon of the "X" to remove the room.
• the mobile device sends information to the cloud computing system and the cloud computing system calculates a new total time to completion.
• the cloud computing system sends the updated time to cause the mobile device to present, on the display, the new schedule.
• the cloud computing system also sends, and the robot receives, information to cause the robot to adjust its cleaning routine to skip the dining room.
• the remaining time estimate displayed in the status indicator 302 is updated to reflect the deletion of the dining room's estimated cleaning time. For example, here, the removal of the dining room icon 312b from the editable mission timeline 310, reduces the overall cleaning estimate by 45 minutes.
• a user may delete room icons 312 from the editable mission timeline 310 to shorten the overall cleaning time or to focus a cleaning mission in particular rooms.
• the user interface 300d shows the editable mission timeline 310 where the icons 312c and 312d, corresponding to the closet and bathroom, respectively, have been re-ordered compared to the order shown in FIG. 4C. As such information is sent from the mobile device and received by the robot to cause the controller 212 of the autonomous cleaning robot 102 will instruct the autonomous cleaning robot 102 to clean the bathroom before cleaning the closet.
• the user interface 300e shows an icon 312e corresponding to an operational event, here a bin evacuation event (e.g. when the robot docks and an evacuation unit removes the dirt and debris in the robot's dust bin into a collection volume or bag of the evacuation unit).
• a bin evacuation event e.g. when the robot docks and an evacuation unit removes the dirt and debris in the robot's dust bin into a collection volume or bag of the evacuation unit.
• the icon 312e informs the user that while the autonomous cleaning robot 102 was in the room
• the robot determines that the dust bin is full or near its full capacity. The robot then traverses the floor to find the docking station or evacuation unit. Upon locating the unit, the dust bin is evacuated
• the robot sends a signal to the cloud computing system.
• the cloud computing system sends information to cause the mobile device to display the indicator.
• the robot After evacuation, the robot returns to clean the next room or space on the schedule.
• An operational event label 314e and an elapsed time 316e are also included in the editable mission timeline 310 informing the user of a description of the operational event and the associated time to resolve it.
• the time information sent from the robot to be displayed can be either a time spent to complete the bin evacuation or a time to drive the robot to the evacuation station, evacuate the bin, and drive the robot back to the cleaning location.
• the robot is controlled to complete the room it is currently cleaning prior to evacuating the bin.
• An operational event icon 312e may be displayed in a different shape, color, outline, etc., from the room icons 312a- 312d. Additionally, icons representing rooms that have been completed, here 312a corresponding to the living room, may be displayed in a different shape, color, outline, etc. from rooms remaining to be cleaned. In some implementations, errors may also be included in the mission timeline 310. An icon, an error label, and an elapsed time may be shown to describe the error.
• the user interface 300f (which may, in some examples be reached by scrolling down from interface 300e shown in FIG. 4E) also includes a graphical representation of a map 318 of a floor plan of an area to be cleaned.
• the floorplan is split into rooms 319. Rooms 319 that have been cleaned may be displayed in a different color, outline, fill pattern, etc. from rooms remaining to be cleaned.
• the map 318 also includes a robot icon 320 representing the location of the autonomous cleaning robot 102.
• the robot icon 320 is moved within the map 318 as the autonomous cleaning robot 102 moves through the rooms to be cleaned. As the robot moves through the rooms or spaces on the schedule, information relating to the robot's position and cleaning status are sent to the cloud computing system.
• the cloud computing system sends information to cause the mobile device to display the robot icon 320 on the map 318 in a location corresponding to the robot's location.
• the cloud computing system also sends information to cause the mobile device to display the rooms 319 based on the robot's cleaning status (e.g. if the robot's cleaning status indicates that cleaning a room has been completed, the room may be colored differently from rooms in progress or remaining to be cleaned).
• the map also includes a dock icon 322 corresponding to the location of a charging dock for the autonomous cleaning robot 102.
• the map 318 includes a pencil icon 323 which, when selected, allows a user to draw on a portion of the map 318.
• the user may draw on the map 318 to split areas on the map 318 into separate rooms. For example, a user may draw along dotted line 330 to split room 319c into two separate rooms. When rooms are split, the user may be prompted to select an icon and/or a name for the split rooms. The user may also be prompted to select cleaning parameters to be used in cleaning the split rooms. In some implementations, the user may select a boundary between rooms on the map 318 to merge rooms together into one room.
• a user may select boundary 332 between rooms 319a and 319b to merge the rooms 319a and 319b into one room.
• the user may be prompted to select an icon and/or a name for the merged room.
• the user may also be prompted to select cleaning parameters to be used in cleaning the merged room.
• the user may draw on the map to indicate that the area indicated requires additional cleaning.
• the mobile device sends information relating to the area requiring additional cleaning (e.g. a location of the area, cleaning parameters to be followed) to the cloud computing system and the cloud computing system sends the information to the robot.
• the controller generates a command signal to clean the area based on the transmission from the cloud computing system and the robot cleans the area according to the specified cleaning parameters.
• the additional cleaning may be added to the cleaning schedule and performed after the robot completes other scheduled cleaning in the cleaning mission. Scheduling the additional cleaning may be performed by the cloud computing system, the controller, or by the user by using the mobile device.
• the map 318 could also include indicators for differing floor types or for locations of thresholds within the area to be cleaned.
• the user interfaces 300g and 300h include a graphical representation of a map 318 which displays cleaned rooms 324 and uncleaned room
• the map 318 also displays a robot icon 320 representing the location of the autonomous cleaning robot 102.
• the user interface 300 also displays a cleaning summary 328 which includes the total cleaning mission time, a measurement for the area cleaned (e.g. a number of rooms, a square footage, area measurement, etc.), and a summary of dirt detect events.
• the summary of dirt detect events may include information relating to locations where the robot detected excess dirt. For example, a summary of dirt detect events may be displayed as indicators on a map interface with each dirt detect event corresponding to an indicator located in a room on the map corresponding to the room (or location within a room) where excess dirt was detected.
• a summary of dirt detect events may be displayed as icons on a post-completion mission timeline with each dirt detect event corresponding to an icon associated with a room icon on the post-completion mission timeline.
• a mission timeline 310b is also included with the cleaning summary 328 and provides a summary of the order of rooms cleaned and the time spent cleaning each room.
• this post- completion mission timeline 310b is not editable, as the cleaning mission is completed.
• the mission timeline 310b is editable such that the user can modify the order or selection of rooms to be cleaned and save the order to be used in a subsequent cleaning mission.
• a user interface 300i may include a first editable mission timeline 410a and a second editable mission timeline 410b with each of the editable mission timelines corresponding to different autonomous cleaning robots.
• editable mission timeline 410a represents a cleaning mission for a mopping robot
• editable mission timeline 410b represents a cleaning mission for a sweeping robot.
• the editable mission timelines 410a and 410b also include a predicted time 415 for cleaning each room. The predicted times may be computed by averaging the actual cleaning times for the room that have been gathered during multiple (e.g. two or more) prior cleaning missions for the room. The predicted times can be updated as the robot progresses through the cleaning mission or as cleaning parameters or the sequence of rooms to be cleaned are adjusted.
• the icons 412 may be manipulated by a user to change the sequence of cleaning the rooms in the mission by rearranging, or deleting, the icons.
• information is sent to the robot to adjust the robot's schedule accordingly to remove or reorder the cleaning operations.
• the mobile device Upon selection of one of the icons 412, the mobile device sends information to the cloud computing system and the cloud computing system calculates a new total time to completion.
• the cloud computing system sends the updated time to cause the mobile device to present, on the display, the new schedule.
• the cloud computing system also sends, and the robot receives, information to cause the robot to adjust its cleaning routine.
• the icons 412 may be selectable by the user and allow the user to select one or more cleaning parameters for that room.
• a cleaning parameter may be, for example, a vacuum power level, a single versus a multiple pass setting, an edge cleaning setting, a wet versus dry cleaning setting, etc.
• information is sent to the robot to adjust the robot's cleaning parameters accordingly to add or change the cleaning parameter for at least a portion of the cleaning mission.
• the mobile device Upon selection of one of the icons 412, the mobile device sends information to the cloud computing system and the cloud computing system calculates a new total time to completion. The cloud computing system also calculates new estimated completion times for each room on the mission timeline and transmits the new estimates to the mobile device for presentation on a display.
• the timelines 410a and 410b may be configured such that each room to be cleaned by the mopping robot, the rooms in timeline 410b, is cleaned by the sweeping robot before the mopping robot cleans the room.
• the processor 114 is configured to order the rooms in the timelines 410a and 410b such that each room is cleaned by the sweeping robot before it is cleaned by the mopping robot. For example, the estimated time of completing sweeping the kitchen by the sweeping robot must be before the estimated time for the mopping robot to begin mopping the kitchen.
• cleaning patterns of a sweeping robot and a mopping robot may be coordinated such that the sweeping and mopping robot may simultaneously clean the same room, however the mopping robot only mops portions of the room that have been previously swept by the sweeping robot.
• the processor 114 is configured to only include rooms with hard floor surfaces (e.g. tile, hardwood, laminate, etc.) in the timeline 410a for the mopping robot.
• Pause buttons 408a and 408b may be used to pause a cleaning mission for the mopping robot and the sweeping robot, respectively. When either of the pause buttons 408a or 408b are selected, a signal is sent to the corresponding cleaning robot to suspend cleaning.
• the timelines 410a and 410b may correspond to two robots configured to perform similar cleaning tasks (e.g. two sweeping robots).
• timelines 410a and 410b may be generated based on the rooms selected to be cleaned.
• timeline 410a for a first robot may include icons corresponding to rooms on an upper floor of a home whereas timeline
• the timelines 410a and 410b for a second robot may include icons corresponding to rooms on a lower floor of a home.
• the timelines 410a and 410b may be generated to minimize an estimated cleaning time of the rooms selected to be cleaned. For example, if the two robots performing the cleaning have docks that are located on opposite sides of a home, the timelines may select rooms nearby to each robot's dock to be included in that robot's timeline.
• a user may configure and save custom cleaning mission plans for multiple robots. For example, a user may save a "high traffic" cleaning mission plan which selects the kitchen and living room to be cleaned by one robot and the master bedroom and hallway to be cleaned by a second robot. When this "high traffic" cleaning plan is selected, the mission timelines for the first robot and the second robot are generated based on the rooms chosen for each in the cleaning plan.
• the user interfaces 300j and 300k present a map 518 including robot icons 520a and 520b, corresponding to locations of the mopping and sweeping robots, respectively (both based on information received from the robot).
• the map 518 displays cleaned areas 524 and uncleaned areas 526 based on data received from the cleaning robots during the cleaning mission.
• the map 518 also displays swept and mopped areas 530 where both the sweeping robot and the mopping robot cleaned the floor surface. Different colors or shading styles may be used to show different types of cleaning (e.g. sweeping and mopping). For example, in FIG. 6A, the solid shading of cleaned areas 524 and 530 represent swept areas and the diagonal line shading of area 530 represents mopped areas.
• Symbols 531 are included on the map 518 to show locations where the cleaning robot performed extra cleaning because excess dirt was detected in that area during the cleaning mission.
• the robot cleans, if the robot detects dirt, the robot performs additional cleaning and sends information relating to the additional cleaning to the cloud computing system.
• the cloud computing system sends the information to the mobile device, which presents a location of the additional cleaning on the display of the mobile device.
• the mobile device may present the location as a symbol on a map or as an icon on a mission timeline.
• the user interface 300 may display a cleaning summary including a total elapsed time 532 for completing the cleaning mission, a number of dirt events (e.g.
• an animation including a play/pause toggle may be included to show a sped-up animation of the mission including cleaning progress and the location and timing of operational events that occurred during the mission.
• selecting an icon on the mission timeline may cause the mapping interface to be displayed to show progress of the robot to the point in the mission represented by the icon.
• an optimized charge and resume function may be included.
• the optimized charge function would compare the estimated time remaining in a cleaning run to the estimated remaining charge of a battery of the autonomous cleaning robot.
• the type of cleaning to be done in each room e.g. edge cleaning, high power vacuum, etc.
• the optimized charge function can direct the robot to stop cleaning and charge for a set time period before finishing the cleaning mission. For example, the autonomous cleaning robot may have 45 minutes of cleaning remaining, but only 20 minutes of battery charge.
• the optimized charge function would command the robot to stop cleaning and charge for a set time period which would fill the battery enough to clean for the remaining 45 minutes of the cleaning mission, but not up to a full charge. For example, charging the battery for an extra 25 minutes of cleaning may only take 15 minutes, whereas charging a battery all the way full may take an hour. As such, the optimized charge would allow the autonomous cleaning robot to finish the cleaning mission 45 minutes earlier than if it charged to a full battery.
• FIG. 7 A is a perspective view of a room 106 with debris 116 on the floor surface 104.
• a user using a handheld computing device, like the mobile phone 600 shown in FIG. 7B, may take a photo of the debris
• a circle 602 is displayed on a screen of the mobile phone 600.
• the user may position the circle 602 to surround the debris 116.
• the circle 602 may also be stationary on the screen and the user would move the mobile phone 600 such that the debris 116 is positioned within the circle.
• a user may draw, (e.g. by using a touch screen) a boundary around the debris 116.
• Photographs taken by the user are analyzed by a processor of the mobile device 600 or the cloud computing system (the images are sent by the mobile device to the cloud computing system for analysis) using an image recognition protocol.
• the image recognition protocol compares the photographs taken by the user of the debris 116 on the floor surface 104 to reference images.
• the reference images may be images taken by a camera located on the robot.
• the processor can locate the debris 116 within a room.
• the processor can mark the locations with debris 116 for additional cleaning during a next cleaning mission.
• the processor can send information relating to the areas marked for additional cleaning (e.g.
• the additional cleaning may include multiple passes over the area, edge cleaning, or spot cleaning.
• the additional cleaning may be displayed on an editable mission timeline as discussed in FIGS. 4A-4F. Additionally or alternatively, the additional cleaning may be presented on a graphical representation of a map as shown in FIGS. 6A-6B. Additional cleaning that has been requested via photographs taken by the user may be marked with a different symbol than the symbol 531 used to show where additional cleaning was performed based on excess dirt being detected during the cleaning run.
• a mobile device as described herein may include a smart phone, a cellular phone, personal digital assistant, laptop computer, tablet, smart watch, or other portable computing device capable of transmitting and receiving signals related to a robot cleaning mission.
• the mobile device as described herein is configured to present, on a display, information relating to a robot cleaning mission and receive an input from a user.
• a cloud computing system as described herein is a computing system external to the mobile device and to the robot that provides computing processing resources as required within the scheduling and control system.
• the cloud computing system is configured to receive and transmit signals between the mobile device and the controller of the robot and is configured to process data received from either the mobile device or the controller of the robot.
• Operations associated with implementing all or part of the object detection techniques described herein can be performed by one or more programmable processors executing one or more computer programs to perform the functions described herein.
• the mobile device, the cloud computing system, and the robot's controller may all include processors programmed with computer programs for executing functions such as transmitting signals, computing time estimates, or interpreting signals.
• a computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
• the controllers and mobile devices described herein can include one or more processors.
• processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer.
• a processor will receive instructions and data from a read-only storage area or a random access storage area or both.
• Elements of a computer include one or more processors for executing instructions and one or more storage area devices for storing instructions and data.
• a computer will also include, or be operatively coupled to receive data from, or transfer data to, or both, one or more machine-readable storage media, such as mass PCBs for storing data, e.g., magnetic, magneto-optical disks, or optical disks.
• Machine-readable storage media suitable for embodying computer program instructions and data include all forms of non-volatile storage area, including by way of example, semiconductor storage area devices, e.g., EPROM, EEPROM, and flash storage area devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD- ROM disks.
• semiconductor storage area devices e.g., EPROM, EEPROM, and flash storage area devices
• magnetic disks e.g., internal hard disks or removable disks
• magneto-optical disks e.g., CD-ROM and DVD- ROM disks.
• the robot control and scheduling techniques described herein may be applicable to controlling other mobile robots aside from cleaning robots.
• a lawn mowing robot or a space-monitoring robot may be scheduled to perform operations in specific portions of a lawn or space as described herein.
• a user could similarly monitor and/or manipulate the mission progress of these robots through a mission timeline and/or mapping interface presented on a mobile device.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Mechanical Engineering (AREA)
• Automation & Control Theory (AREA)
• Remote Sensing (AREA)
• Radar, Positioning & Navigation (AREA)
• Aviation & Aerospace Engineering (AREA)
• Human Computer Interaction (AREA)
• Computing Systems (AREA)
• Mathematical Physics (AREA)
• Electric Vacuum Cleaner (AREA)
• Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
• Business, Economics & Management (AREA)
• Health & Medical Sciences (AREA)
• Artificial Intelligence (AREA)
• Evolutionary Computation (AREA)
• Game Theory and Decision Science (AREA)
• Medical Informatics (AREA)

Priority Applications (5)

Applications Claiming Priority (2)

Publications (1)

Family

ID=64456366

Family Applications (1)

Country Status (6)

Cited By (1)

Families Citing this family (72)

Citations (4)

Family Cites Families (76)

• 2017
  • 2017-06-02 US US15/612,368 patent/US20180344116A1/en not_active Abandoned
  • 2017-10-30 EP EP17901335.4A patent/EP3631585A4/en active Pending
  • 2017-10-30 CN CN201780021339.1A patent/CN109392308B/zh active Active
  • 2017-10-30 JP JP2018550697A patent/JP2020521515A/ja active Pending
  • 2017-10-30 MY MYPI2019000835A patent/MY203969A/en unknown
  • 2017-10-30 WO PCT/US2017/059064 patent/WO2018222219A1/en not_active Ceased
• 2020
  • 2020-11-30 US US17/107,758 patent/US11638510B2/en active Active
• 2022
  • 2022-11-02 JP JP2022176550A patent/JP7694915B2/ja active Active
• 2023
  • 2023-04-17 US US18/301,767 patent/US12089801B2/en active Active
• 2024
  • 2024-09-16 US US18/886,549 patent/US20250082158A1/en active Pending

Patent Citations (4)

Also Published As

Similar Documents

Legal Events