TW201933177A - Mobile robots to generate reference maps for localization - Google Patents

Abstract

An example robot performs a scan to obtain image data of a given region. The robot performs image analysis on the image data to detect a set of undesirable objects, and generates a reference map that excludes the set of undesirable objects, where the reference map is associated with the location of the robot at the time of the scan.

Description

Mobile robot for generating reference map for positioning

The present invention generally relates to a mobile robot for generating a reference map for positioning.

In the field of robotics, robots with mobility in free space are typically positioned by themselves, meaning they determine their own position relative to a reference point or frame. Mobile robots usually use reference maps to locate themselves.

According to a feasible implementation aspect of the present invention, a robot is specifically proposed, which includes a set of image sensors, a space determination resource, a control system, and a propulsion mechanism; wherein the control system: executes one of a given area Scan to use the set of image sensors to obtain the image data of the given area; use the spatial decision resources to determine the position of the robot relative to a reference point at the time of the scan; perform an image on the image data Analysis to detect a group of unwanted objects depicted by the image data, the group of unwanted objects being at least one of a dynamic object or an undesired object of a predetermined category; and generating an exclusion group A reference map of an unwanted object, the reference map being associated with the position of the robot at the time of the scan.

The example proposes a robot that generates a reference map for enabling the robot (or another robot) to perform positioning. The reference map can be generated by the robot performing a scan of a given area to obtain image data, and then performing image analysis on the image data to detect and exclude unwanted objects from the reference map.

In some examples, an example robot includes a set of image sensors, a space determination resource, a control system, and a propulsion mechanism. The control system is operable to perform a scan of a given area and use the space to determine resources to determine a position of the robot relative to a reference point during the scan. The robot can perform image analysis on the image data to determine a group of unwanted objects depicted by the image data and generate a reference map that excludes the group of unwanted objects. In these cases, the reference map system is associated with the position of the robot at the time of the scan. As described in conjunction with the example, the reference map can act as a resource so that the robot (or another robot) can then locate itself when the robot (or the like) travels through the given area.

Once a reference map is generated for a given area, a robot (the robot that performs positioning) can use the reference map to determine its position relative to a reference frame or point. Specifically, the robot can capture (shoot) image data for a given area, and then compare the current image data with a reference map to identify visual landmarks, which may form the current image and the reference map. Comparison basis. The robot performing positioning can then determine its relative position in the given area by comparing the shape and features depicted by the selected object in the current image with the shape of the same object depicted in the reference map The location of the reference frame or reference point. By comparison, the morphologies and features that can form a basis for comparison include the pixel size for individual objects, including those required for the determination of the distance between two objects, and the approximate shape or size of the objects depicted. By comparing the relative size of an object depicted in two images (such as the current image and the reference map), the robot may be able to triangulate it in a given area relative to a previously referenced map Take a location of the image data to its own location.

The example recognizes that positioning using reference maps is computationally expensive for robots. In addition, there will be sufficient variability in the way robots operate to make individual robots misidentify stationary and long-lived objects that can be accurately relied upon.

One or more examples described herein may be implemented using instructions executable by one or more processors. These instructions can be carried on a computer-readable medium. The following provides examples of processing resources and computer-readable media in conjunction with the machines shown or described in the drawings, on which instructions to implement the examples described herein can be carried and / or executed. In particular, the various machines shown in the examples described herein include (several) processors and various forms of memory for storing data and instructions. Examples of computer-readable media include permanent memory storage devices, such as hard drives on personal computers or servers. Other examples of computer storage media include portable storage units, such as CD or DVD units, flash memory (such as those carried on smart phones, multi-function devices, or tablets), and magnetic memory. Computers, terminals, and network-enabled devices (such as mobile devices, such as cellular phones) are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable media. In addition, some examples may be implemented in the form of a computer program or a computer that can carry the program in a carrier medium.

FIG. 1 shows an example robot for generating a reference map for performing positioning. In particular, FIG. 1 depicts a robot 100 having a control system 130, a propulsion mechanism 140, a set of image sensors 150, and a space determination resource 152. As described, the control system 130 can use the image sensor 150 to obtain image data from a given area 101. The control system 130 may implement map generation logic 112 to generate a reference map 115 from the image data in the given area. When generating the reference map 115, the map generation logic 112 may utilize semantic segmentation to detect and exclude unwanted objects from the reference map 115. Such undesirable objects include dynamic objects and stationary objects, which are essentially non-persistent in their individual locations (such as objects that are easily moved). Therefore, the map generation logic 112 can generate the reference map 115 to include stationary and permanent objects, rather than dynamic or non-permanent objects. In some examples, the map generation logic 112 can increase the ratio of the number of static and persistent objects depicted in a given reference map 115 to the number of non-permanent or dynamic objects. In some implementations, the map generation logic 112 can be implemented to include only those objects that meet a critical confidence level relative to one of the stationary and persistent objects of the given area.

In addition to selecting which objects should be displayed on the reference map 115, the control system 130 also associates a landmark or a position identifier with the position of the robot 100 when the reference map 115 is captured. More specifically, the control system 130 determines that it is a specific coordinate or other position identifier for the set of sensors that generated the reference map 115. Various techniques can be used to determine the position of the robot 100 relative to a reference frame or point. For example, the robot 100 may be located in the middle of the given area, or aligned with a mark visible by the sensor. Furthermore, the robot 100 may be equipped with a space determination resource 152 so that the robot can determine its position through self-monitoring. For example, the robot 100 may include a motion sensor (such as an accelerometer, gyroscope, and / or inertial mass unit (IMU)) that the robot 100 can use to track its own position relative to a reference point. Additionally or alternatively, the space determination resource 152 may correspond to an odometer, which can track, for example, progress when the robot 100 moves from a reference point with a known position or coordinate (eg, entrance to a given room) The number of rotations of the institution 140. The control system 130 can track the odometer and determine the information from the movement sensors to determine the relative position of itself in the given area 101 when the scan for acquiring the image data is performed. The control system 130 can then associate the determined position of the robot 100 at the time the scan was performed with the reference map 115 generated from the scanned image data. In this way, by allowing the image data to be triangulated to divide the distance information from the position of the robot 100 when the scan is performed, the reference map 115 can then be used for positioning. The distance information can then be converted into a coordinate relative to the position of the robot 100 when the scan for the reference map is performed.

In some examples, the robot 100 may generate the reference map 115 at a first moment, and then use the reference map 115 at a later moment to determine its own position within a given area and / or reference frame. As an addition or alternative, a robot may transmit the reference map 115 to another robot, or to a reference map database. Furthermore, the robot can update a reference map 115 at any time when using these maps.

Among other advantages, several examples have realized that positioning in the traditional way may consume a lot of computing power and is unreliable. For example, the robot performing positioning may perform real-time image processing to perform complex operations for detecting and comparing an object currently in view and an object referring to a map. When the current video data depicts a crowded scene of multiple objects moving, these operations become more computationally intensive and untrustworthy. To alleviate the complexity of these scenes, the robot 100 can use the semantic segmentation method to generate the reference map 115 to reliably detect the dynamic and / or non-permanent objects of a scene. When these objects are detected, they can be excluded from the reference map 115, so that the reference map is less likely to be filled with unsuitable objects, otherwise the unsuitable objects may produce incorrect positioning results when subsequently used.

The propulsion mechanism 140 includes an interface 142, at least one motor 144, and a steering system 146. The interface 142 connects the propulsion mechanism 140 to the control system 130 so that the propulsion mechanism 140 can receive instructions from the control system 130. The propulsion mechanism 140 may receive instructions regarding direction and speed from the control system 130. The commands can be used to drive the at least one motor 144 and direct the steering system 146. The at least one motor 144 may include one or more motors for propelling the robot. For example, one motor can drive all the wheels of the robot 100, or each wheel can be driven by its own motor, or any other combination of wheels and motors. The steering system 146 may include mechanism components (such as shafts, connecting rods, hydraulic devices, belts, etc.) to manipulate an angle of the wheels (such as synchronous drive, articulated drive, etc.), or use a speed between multiple motors The difference (e.g., differential drive, etc.), or any combination thereof, guides the robot 100 according to instructions received from the control system 130.

The set of image sensors 150 may include one or more types of cameras, including three-dimensional image sensing cameras (eg, cameras using a distance sensor, LiDAR, stereo cameras, etc.). For example, the set of image sensors 150 may include a laser sensor, which illuminates a target with a pulse of laser light, and measures the reflected pulse with a sensor. As another example, the set of image sensors 150 may include a camera sensor that can passively obtain two-dimensional image data. Still further, the set of image sensors 150 may include a pair of stereo cameras that cooperate to produce a three-dimensional drawing of a given object in the scene. Three-dimensional information can also be obtained from movement (e.g. from a moving structure).

The control system 130 may include a memory 110 and a processor 120. The memory 110 may be in any form (eg, RAM, DRAM, etc.) and may include map generation logic 112. The map generation logic 112 may include instructions for controlling the robot 100 when the robot 100 travels through a section or area. The map generation logic 112 may also include instructions to enable the robot 100 to generate a map of the area or area that the robot 100 travels through. The map generation logic 112 may also include data (such as models, images, templates, etc.) that are referenced by the control system 130 during the map construction process to help determine the identity of the object detected by the robot 100 and determine One of each of the detected objects is scheduled to be classified. The processor 120 can access the map generation logic 112 from the memory 110 to control the robot 100 and generate the reference map 115. In addition, as shown in FIG. 1, the control system 130 may be integrated with the robot 100 (for example, using hardware, firmware, and / or software). More generally, the map generation logic 112 may be implemented as a control system 300, such as shown in the example of FIG. 3. The control system 300 can be implemented in conjunction with the robot 100 or from the far end of the robot 100.

2A to 2C illustrate an example of a robot 100 generating a reference map 115 for a given area (eg, a residential room). In FIG. 2A, the robot 100 positions itself at a known position 211 to capture image data from a scene. The image data can depict a person 202, a wastebasket 204, a copier 206 and a wall 208. The robot can simultaneously capture the image data while tracking its position relative to a reference point 210 or reference frame.

In FIG. 2B, the robot 100 uses the map logic 112 and the image data extracted from the area 201 to generate the reference map 115. The robot 100 may exclude dynamic objects (such as people 202) and non-permanent objects (such as wastebasket 204). The robot 100 may use, for example, a map generation program described with an example of FIGS. 4A and 4B to identify the person 202 and the wastebasket 204 as undesirable objects for the reference map 115. The reference map 115 may then depict one of the boundaries of the copier 206.

In FIG. 2C, another robot (or alternatively, the same robot 100) may use the reference map 115 at a different time to locate itself. The robot 100 can capture one of the current views 215 of the scene in the area 201 and use the reference map 115 to identify which object in the scene (eg, the copier 206) is to be used for positioning purposes. The robot 200 can then use the reference map 115 to triangulate the drawn features of the copier in the current view 215 to determine the relative position of the robot 200 relative to one of the robots 100 that generated the reference map 115.

FIG. 3 illustrates an example control system for generating a reference map that is associated with the position of the robot at the time of scanning. As described with some examples, a control system 300 can be implemented to use sensor data generated from a robot's sensor set, such as described with one of the examples of FIG. 1.

In FIG. 3, the control system 300 includes a memory 310 and a processor 320. The memory 310 may be in any form, including RAM, DRAM, or ROM. The memory 310 can store instructions, for example, through the installation of software (for example, an application program). The processor 320 can access instructions from the memory 310 to control the robot 100. According to some examples, the processor 320 accesses multiple sets of instructions, including: (i) a first set of instructions 312 to obtain image data from a scan of a given area; (ii) to execute an image on the image data Analyze a second set of instructions 314; and (iii) a third set of instructions 316 to generate a reference map that excludes the set of unwanted objects and is associated with the position of the robot at the time of scanning .

In some examples, the control system 300 can be implemented as an integrated component of a working robot, for example, for use with reference map construction operations that these robots routinely perform. For example, when the robot travels through a given area to construct a reference map, the control system 300 can immediately execute instructions 312-316. Alternatively, the control system 300 may be implemented as a remote or separate entity. For example, the control system 300 may receive sensor data, which is sent from the robot 100 using, for example, a wireless communication and / or network channel. In such instances, the control system 300 may use the sent sensor data to generate a reference map of the given area, and then, once the reference map is generated or updated, transfer the reference map back to the robot.

In a further variation, the control system 300 may transmit the generated reference map to a different robot than the robot used to obtain sensor data for the given area. For example, the control system may use a first robot to generate a reference map for a given area, and transmit the generated reference map to a second robot, or alternatively to a robot team. As another example, the control system 300 may be implemented on the robot 100 or a remote entity to receive the sensing for the given area from another robot, or alternatively from a sensing device or assembly器 资料。 Device information.

The controller system 300 can operate synchronously (eg, in real time) to use the sensor data being obtained from the sensor set of the robot to construct a reference map for positioning purposes. Varyingly, the instructions 312-316 may be implemented partially or entirely in an asynchronous manner. For example, in an example where the control system is integrated with the robot 100, the robot 100 may execute instructions 312, 314 at a later time, when, for example, the robot has more computing resources available, or when the robot is offline And / or 316. Similarly, in an example where the control system 300 is remote or separate, the control system 300 can execute the instructions 312-316 independently of the operation of the robot.

FIG. 4A illustrates an example method for generating a reference map associated with the position of the robot at the time of scanning. The example method shown in FIGS. 4A and 4B can be implemented using the components shown in the examples of FIGS. 1 to 3. Therefore, the references made to the elements of FIGS. 1 to 3 are for the purpose of clarifying a suitable element or component for performing one of the described steps or sub-steps.

Referring to an example of FIG. 4A, image data is scanned from one of the given areas (block 410). The image data can be obtained, for example, from a camera and a depth (or distance) sensor, a LiDAR camera, a pair of stereo cameras, and / or a combination thereof (collectively referred to as "image sensors"). In some variations, the image data is obtained in real time, for example, a robot traveling through the given area has an on-board image sensor ("sensing robot"). In some variations, the image data is obtained from the memory only after the robot travels through the given area.

In some examples, the image data is processed by the control system 300 existing on the robot that obtained the image data. In some variations, the image data is obtained by a control system 300 that exists on another robot that is in local communication (eg, a regional wireless link) with the sensing robot that obtained the image data. Furthermore, the control system 300 for processing the image data may be a remote network computer, such as a server, directly or indirectly communicating with the sensing robot.

Once the image data is obtained, image analysis can be performed to determine a set of unwanted objects. Image analysis can be performed on the 2D or 3D images captured by the set of image sensors 150. Such unwanted objects include dynamic objects and / or objects identified as belonging to a predetermined category (block 420). The image analysis performed by the control system 300 may include object detection and classification, where the object is classified by type, category, or instance. In addition, the classification action used to determine when an object does not want to be used for a reference map can be made based on a permanent determination. For example, if objects are considered to be dynamic (such as moving), dynamic in nature, or not fixed (such as non-permanent settings), these objects may not be wanted for the reference map of. In some examples, objects can be identified by type (eg, chair, table, etc.), and permanent classification can be based on predetermined characteristics associated with the object type.

In some variations, the object classification may be provided to perform image analysis to classify objects into discrete categories or groups based on similarity. Grouping of similar objects can be used to define any of multiple classification systems, which can define multiple classifications over time and reflect a permanent characteristic of whether the object is fixed or not, and other more granular classifications .

Some variations provide that objects can be assigned to multiple categories, categories, or groups based on a single possibility. The classification of such objects may be based on a reliability score or value (eg, a value between 0.0 and 1.0), which can represent a level of confidence that the classification may be correct. Therefore, for example, for an object to be classified as a fixed classification of the object, it can reflect a reliability value as to whether the object may be moved in the future.

As described with some examples, a dynamic object may be a person who is detected to be moving when a given area is sensed. In order to identify such objects, the control system 300 may compare the image data of a captured scene in a plurality of closely spaced time intervals. If an object appears in an area of a captured scene in a time interval and appears in another area of the captured scene in another time interval, it is because the object is felt in the given area It is moving at the time of measurement, so the object can be recognized as dynamic.

Although the time-based object detection method can be used to detect such dynamic objects, some examples can also use the object classification method, in which a detected object system is based on the detection of the object as depicted in a captured scene Characteristics to determine whether it belongs to a specific category form. In these cases, even if the detected object does not move while the given area is being sensed, the detected object can be recognized as dynamic. For example, a cat may continue to lie still for a while while a given area is being sensed, but the control system 300 will still recognize what the cat is and recognize it as a dynamic object.

In a similar manner, the object classification method can also be used to identify other objects that are considered undesirable for the purpose of map generation. These objects may have a characteristic that is static in nature, but are not permanently set (eg static and in the same position for a long period of time). Such unwanted objects may include objects that can be easily moved by contact with a dynamic object. For example, a desk chair in a room may be static, but may move over time. On the other hand, a large table may be assumed to be statically and permanently installed in the same room. The control system 300 may perform analysis on the image data depicting such objects to identify multiple physical characteristics (such as shapes, signature features, etc.), which are characteristics of an object type or category. Based on the determined object type or category, the control system 300 can make a determination to identify the object as undesirable for the purpose of map generation.

The control system 300 may generate a reference map that excludes the set of unwanted objects and is associated with the position of the robot at the time of scanning (block 430). The generation of the reference map for a given area may cover an initial mapping procedure or activity, as well as subsequent activities that may lead to updating the map of the given area. In particular, some examples recognize that map updating can be done when the robot senses a given area for any task or activity, and when it encounters an unknown or other object that is unexpected for its location.

When performing image analysis, the control system 300 may utilize a database containing various models, live data, and / or templates to identify the type, category, and sub-category of objects. The database of these models, live data and / or templates can also be updated with repeated use of the robot in a given area. According to some examples, the database maintained for the purpose of image analysis can be updated with objects encountered by a robot over a given area over time. In addition, when the robot is operating in a given area for purposes other than generating a reference map (such as cleaning, vacuuming, delivering packages, etc.), the robot can use the image database. In particular, the robot can maintain a database of undesired objects, and when the robot travels through a given area and encounters an unexpected object, the control system 300 can perform image analysis to use the object to match the previously encountered and Cooperative comparison of a collection of unwanted objects classified by robots. In order to perform this comparison, the control system 300 can perform object classification and / or recognition (e.g., to detect the writing features of the object and use the writing features to compare with the writing features of other objects encountered in the given area over time Compare). If the robot does not recognize an unexpected object as an object that has been previously recognized, the robot can reclassify the object according to the category (such as a table or chair) and make a decision about whether the object is not wanted based on the type of the object The judgment of the important person. Therefore, many examples recognize that the robot can update a reference map of a given area at any time when the robot is deployed to identify newly encountered objects and identify such objects as undesirable (such as dynamic or non-long) (Still still) or desired (such as long-term storage settings) for the purpose of map generation.

In addition, in some examples, for the purpose of reference map generation, the control system 300 may update the models, live data, and template images used to designate objects as wanted or undesired. For example, if the robot repeatedly encounters an object designated as undesirable for the reference map, but then detects that the object is static in its position for a long period of time, the control system 300 may reassign the object And include the object in one of the updated versions of the map of the given area. The reassignment of objects may be consistent with the control system 300 reclassifying the encountered objects into an object type different from a previous object type classification, where the reclassified object type is known to be static and persistently set. For example, a robot may initially draw a map of a room, and identify a table as an unwanted person based on the size of a table and / or its table legs (eg, a small card table). However, if the robot repeatedly encounters the table legs at the same position for a long period of time, the control system 300 can reclassify the object into a static and permanently placed type (such as a table-type fixture). In such cases, the control system 300 may update the reference map of the given area to include the table.

Conversely, if the robot recognizes that a particular object is of a static and permanently installed type (such as a table), so that the object is included in the map, but then detects that the object has moved, then the control system 300 may The object is reclassified into an unwanted type, so that it is excluded from the reference map. Alternatively, the control system 300 may designate the object type of the displaced object as unwanted, so that if the robot encounters other objects of similar appearance, those objects will also be designated as unwanted and excluded from the given area Outside the reference map.

Referring to FIG. 4B, the control system 300 may perform object detection and classification on image data depicting a given area, in order to identify unwanted objects for the purpose of generating a reference map of that area (block 460). When performing object classification, the control system 300 may use a semantic segmentation program or technique (block 450). In this procedure, a pixel by pixel analysis is performed to divide a detected object depicted in an image into foreground. Once divided, the control system 300 can recognize the shape or outer surrounding features of one of the divided objects. The control system 300 can then compare the detected shape or external features with the database of models and templates to identify a matching object type.

As described with some examples, the database of models and templates can be based in part on historical data, based on images previously captured by the robot, corresponding to images that have been previously processed and designated as wanted or undesired.

In some examples, the classification scheme may designate objects as one of statically and permanently placed, unwanted, or dynamic. The control system 300 can determine a classification of a newly detected object by comparing the template or model image of the divided object with the previously encountered object. If a newly encountered object is deemed to sufficiently conform to a previously classified object, then the classification of the earlier object may be assigned to the newly encountered object. Over time, the control system 300 may reclassify objects based on what the robot senses in a given area.

In some variations, a reliability score is associated with a conforming object type, and when there may be multiple object types, the reliability score may be used to select the most appropriate object type. For the purpose of reference map generation, when an object meets the criticality level of reliability for one of the detected object types, the object can be designated as an unwanted one.

The control system 300 may also associate objects of different classifications with different layers (block 470). For example, when objects are classified permanently, different permanent classifications of objects may be associated with different layers. Furthermore, the control system 300 can activate different layers so that the reference map only depicts selected layers corresponding to a given classification (eg, corresponding permanent classification or objects defined by the user). Based on the determined layer, the control system 300 can perform object classification and / or recognition to compare the object with other objects of the same layer and / or type.

FIG. 5 illustrates a method for operating a robot to generate a reference map. A method such as that described with the example of FIG. 5 can be implemented, for example, using an example robot such as described with the example of FIG. 1. Thus, for the purpose of illustrating suitable components for performing a described step or sub-step, reference will be made to the elements of FIG. 1.

Referring to an example of FIG. 5, a robot can operate to scan a given area to obtain image data (block 510). For example, the robot 100 may use, for example, one (or more) two-dimensional cameras (such as cameras with wide-angle lenses and fisheye lenses) and / or one (or more) three-dimensional image sensors (such as LiDAR (light ) Sensors, a pair of stereo cameras and / or cameras with distance sensors) provide a set of image sensors to obtain image data.

When a spatial decision resource 152 is used to perform the scanning action, the robot 100 may determine its position (block 520). The spatial decision resource 152 may be equivalent to a sensing mechanism for detecting a reference point (such as a visual mark), for example. The resources of the spatial decision resource 152 may include, for example, a motion sensor, such as an accelerometer and / or gyroscope. Additionally or alternatively, the space determination resource 152 may include an odometer. For example, using the corresponding sensing mechanism, the robot 100 can detect its initial position relative to the reference point. The robot 100 may use an odometer for linear distance and an accelerometer and / or gyroscope for detecting lateral movement and direction change, while tracking its own movement in a given area. By tracking its own movement with reference to a reference point, the robot 100 can determine its position when performing a scan.

In some examples, the robot 100 performs image analysis on the image data obtained from a given area to detect a group of unwanted objects depicted by the image data (block 530). Such undesired objects may correspond to objects with a permanent classification indicating that individual objects are set dynamically or non-permanently (eg, the object may be moved by another object). Therefore, the image analysis can identify objects whose position is dynamic, not perpetual in nature, or other undesirable objects that are not conducive to positioning or mapping purposes.

The robot 100 generates a reference map 115 that excludes one of the group of unwanted objects detected from the image data of the given area (block 540). This reference map may be associated with the position of the robot 100 at the time of scanning. In this way, the reference map 115 can then be used to locate the robot (or another robot) based on the position of the robot 100 at the time of scanning, and objects depicted or otherwise presented in the reference map.

In some examples, the reference map is generated using a simultaneous positioning and mapping (SLAM) algorithm or a similar algorithm. These algorithms are used to map (map) an unknown environment while continuously tracking simultaneously The position of a robot in the environment.

Although some examples provide a robot to perform image analysis to detect the group of unwanted objects, other examples (such as those described with FIG. 3) may use a control system that is separate from or remote to the robot 100. Similarly, a separate or remote computer can generate the reference map based on the exclusion of the detected group of unwanted objects.

I consider that the examples described in this article extend to the individual elements and concepts described in this article, independent of other concepts, ideas or systems, and that some examples are intended to include many elements described anywhere in this application. Various combinations. Although many examples have been described in detail herein with reference to the accompanying drawings, it should be understood that these concepts are not limited to those precise examples. Therefore, it is intended that the scope of these concepts is defined by the appended patent application scope and its equivalent scope. Furthermore, it is expected that a particular feature described individually or as part of an example may be combined with other individually described features or parts of other examples, even if these other features and examples do not mention the Specific characteristics. Therefore, even if a combination is not described, it should not prevent the right to have such a combination.

100、200‧‧‧robot

101‧‧‧ (given) area

110, 310‧‧‧ memory

112‧‧‧Map (generate) logic

115‧‧‧Reference map

120, 320‧‧‧ processor

130、300‧‧‧Control system

140‧‧‧Promotion agency

142‧‧‧Interface

144‧‧‧Motor

146‧‧‧Steering system

150‧‧‧Image sensor

152‧‧‧ Space judgment resources

201‧‧‧Region

202‧‧‧ people

204‧‧‧ waste paper basket

206‧‧‧Copier

208‧‧‧ Wall

210‧‧‧Reference point, area

211‧‧‧ Known location

215‧‧‧ Current video

312 ~ 316‧‧‧Command

410 ~ 430, 450 ~ 470, 510 ~ 540‧‧‧ block

FIG. 1 illustrates an example robot used to generate a reference map for positioning.

2A to 2C illustrate an example of a robot that generates a reference map.

FIG. 3 illustrates an example control system for generating a reference map for positioning.

FIG. 4A illustrates an example method for generating a reference map for positioning.

FIG. 4B shows an example method for performing image analysis using semantic segmentation to generate a reference map that excludes undesired persons in the current image from being depicted as part of the map.

FIG. 5 illustrates an example method for operating a robot to generate a reference map.

Claims (15)

A robot that includes: A set of image sensors; A space judgment resource; A control system; and 1. Promotion organization; Among which the control system: Perform a scan of a given area to use the set of image sensors to obtain image data of the given area; Use the space determination resource to determine a position of the robot relative to a reference point at the time of the scan; Perform image analysis on the image data to detect a group of unwanted objects depicted by the image data, the group of unwanted objects being at least one of a dynamic object or an unwanted object of a predetermined category; as well as A reference map that excludes the set of unwanted objects is generated, the reference map being associated with the position of the robot at the time of the scan. The robot according to claim 1, wherein the space determination resource includes a movement sensor. The robot of claim 1, wherein the space determination resource includes an odometer. The robot according to claim 1, wherein the control system repeatedly executes one scan of the given area when operating the propulsion mechanism to move the robot within the given area, the control system for each of the scans The user determines the position of the robot relative to at least one of the reference point or a second reference point. The robot of claim 1, wherein the control system detects a plurality of objects in the given area including the set of unwanted objects, and determines a boundary area for each of the detected objects. The robot of claim 1, wherein the control system uses semantic segmentation to perform image analysis. The robot of claim 1, wherein the control system performs image analysis to determine a plurality of objects including the group of unwanted objects. Like the robot of claim 1, wherein the reference map is layered to identify objects of different classes in different layers. A control system, including: A memory for storing instructions; and At least one processor for executing these instructions to perform the following actions: Obtain image data of a given area, the image data is obtained from a scan of the given area by multiple sensors provided on a robot; Performing image analysis on the image data to determine a group of unwanted objects, the group of unwanted objects being at least one of a dynamic object or an undesired stationary object of a predetermined category; and A reference map that excludes the set of unwanted objects is generated, the reference map being associated with a position of the robot at the time of the scan. The control system of claim 9, wherein the at least one processor executes the instructions to repeatedly perform a scan of the given area when the robot moves within the given area. The control system of claim 10, wherein the at least one processor executes the instructions to determine the position of the robot relative to at least one of a first reference point or a second reference point for each of the scans. The control system of claim 9, wherein the at least one processor executes the instructions to detect a plurality of objects in the given area including the set of unwanted objects, and for each of the detected objects One determines a boundary area. The control system of claim 9, wherein the at least one processor executes the instructions to perform image analysis using semantic segmentation. The control system of claim 9, wherein the at least one processor executes the instructions to perform image analysis to determine a plurality of objects including the group of unwanted objects. A method for operating a robot. The method includes the following steps: Scanning to obtain image data from one of the given areas; Perform image analysis on the image data to determine a group of unwanted objects, the group of unwanted objects including at least one of a dynamic object or a predetermined category of unwanted objects; and A reference map that excludes the set of unwanted objects is generated, the reference map being associated with a position of the robot at the time of the scan.