WO2023144535A1 - Continuous smooth surface segmentation for object cleaning using a 3d cleaning tool - Google Patents

Continuous smooth surface segmentation for object cleaning using a 3d cleaning tool Download PDF

Info

Publication number
WO2023144535A1
WO2023144535A1 PCT/GB2023/050168 GB2023050168W WO2023144535A1 WO 2023144535 A1 WO2023144535 A1 WO 2023144535A1 GB 2023050168 W GB2023050168 W GB 2023050168W WO 2023144535 A1 WO2023144535 A1 WO 2023144535A1
Authority
WO
WIPO (PCT)
Prior art keywords
points
computer
point
implemented method
continuous surface
Prior art date
Application number
PCT/GB2023/050168
Other languages
French (fr)
Inventor
Tanis MAR HERNANDEZ
Original Assignee
Dyson Technology Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dyson Technology Limited filed Critical Dyson Technology Limited
Publication of WO2023144535A1 publication Critical patent/WO2023144535A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0219Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory ensuring the processing of the whole working surface
    • AHUMAN NECESSITIES
    • A47FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47LDOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L11/00Machines for cleaning floors, carpets, furniture, walls, or wall coverings
    • A47L11/40Parts or details of machines not provided for in groups A47L11/02 - A47L11/38, or not restricted to one of these groups, e.g. handles, arrangements of switches, skirts, buffers, levers
    • A47L11/4011Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/18Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/18Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form
    • G05B19/4097Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by using design data to control NC machines, e.g. CAD/CAM
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00Programme-control systems
    • G05B19/02Programme-control systems electric
    • G05B19/18Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form
    • G05B19/41Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by interpolation, e.g. the computation of intermediate points between programmed end points to define the path to be followed and the rate of travel along that path
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0251Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10Simultaneous control of position or course in three dimensions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2137Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/30Polynomial surface description
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/12Edge-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/181Segmentation; Edge detection involving edge growing; involving edge linking
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/187Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/64Analysis of geometric attributes of convexity or concavity
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/457Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by analysing connectivity, e.g. edge linking, connected component analysis or slices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • AHUMAN NECESSITIES
    • A47FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47LDOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/04Automatic control of the travelling movement; Automatic obstacle detection
    • AHUMAN NECESSITIES
    • A47FURNITURE; DOMESTIC ARTICLES OR APPLIANCES; COFFEE MILLS; SPICE MILLS; SUCTION CLEANERS IN GENERAL
    • A47LDOMESTIC WASHING OR CLEANING; SUCTION CLEANERS IN GENERAL
    • A47L2201/00Robotic cleaning machines, i.e. with automatic control of the travelling movement or the cleaning operation
    • A47L2201/06Control of the cleaning action for autonomous devices; Automatic detection of the surface condition before, during or after cleaning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/04Indexing scheme for image data processing or generation, in general involving 3D image data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • G06T2207/10012Stereo images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20128Atlas-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30108Industrial image inspection
    • G06T2207/30132Masonry; Concrete
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30181Earth observation
    • G06T2207/30184Infrastructure

Definitions

  • the present invention relates to a computer-implemented method of identifying one or more continuous smooth surface segments of an object to be cleaned by a 3D cleaning tool.
  • the invention also covers a related method and an input generation system for a control system for a 3D cleaning tool.
  • BACKGROUND TO THE INVENTION 3D cleaning tools are devices such as robotic devices which may be placed adjacent to an object to be cleaned, such as an item of furniture, or a fixture of a house such as a staircase or floor, and set to clean the item automatically.
  • the 3D cleaning tool In order to clean the object effectively, the 3D cleaning tool must first be able to build an electronic picture of the object to be cleaned, so that it can subsequently generate instructions to control the operation of the device. In order to ensure that the 3D cleaning tool operates effectively, it is necessary to ensure that it is able to build an accurate, useful electronic picture or digital representation of the object to be cleaned, and then that it is able to generate accurate instructions which control the operation of the device.
  • the majority of the surfaces of furniture such as tables, chairs, and sofas are in the form of continuous, smooth, surfaces. These may be cleaned e.g. by brushing, sweeping, vacuuming, or wiping. In order to plan such tasks, it is first necessary accurately to identify those surfaces.
  • the present invention relates to processes (and other related aspects) for identifying these surfaces.
  • the present invention relates to methods for identifying one or more surfaces of an object to be cleaned by applying a surface segmentation algorithm to an electronic representation of an outer surface of the object. According to the invention, the segments are identified using a predetermined curvature threshold.
  • a first aspect of the present invention provides a computer-implemented method of identifying one or more continuous smooth surface segments of an object to be cleaned by a 3D cleaning tool, the computer-implemented method comprising: receiving a 3D electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points P i , each point representative of a point on the outer surface of the object to be cleaned; applying a surface segmentation algorithm to the electronic representation of the surface S, thereby identifying one or more continuous surface segments S i of the surface S, wherein: each of the continuous surface segments S i comprises a subset of the plurality of points, and the curvature of each of the continuous surface segments S i is below a predetermined curvature threshold.
  • 3D cleaning tool as used in this application should be understood to cover any device which is adapted to clean an outer surface of a 3D object.
  • the 3D cleaning tool preferably comprises an end-effector which is configured to contact the outer surface of the object to be cleaned in order to perform the cleaning task.
  • the end-effector may comprise, for example, a vacuum cleaner head, a wipe, a brush, and the like.
  • a 3D cleaning tool preferably also comprises a motion system which is configured to position the end-effector in the correct position and/or orientation.
  • the motion system may comprise one or more robotic arms which are connected at an elbow.
  • the robotic arms may be rotatable about one or more axes relative to each other.
  • the end-effector may be rotatable about one or more axes relative to a distal robotic arm.
  • a proximal robotic arm may be rotatable about one or more axes relative to a base portion of the 3D cleaning tool.
  • the 3D cleaning tool may further comprise a control system, optionally located in the base portion, which may be configured to generate and/or transmit instructions to the motion system, the instructions configured to cause the motion system, and optionally the end-effector to move relative to each other in order to position the end-effector in a desired position and/or orientation. More details relating to the control system will be set out later in this application. It should be stressed that computer-implemented methods according to the first aspect of the invention may be used in combination with other 3D cleaning tools which do not take the form described in this paragraph, without departing in any way from the scope of the invention.
  • the term “outer surface” as used in this application refers to any exposed surface which an external 3D cleaning tool is able to access.
  • the outer surface may comprise a plurality of constituent surfaces, some of which may be flat, and some of which may be curved.
  • the outer surface S may comprise a plurality of constituent smooth continuous surfaces, to be identified using the computer-implemented invention of the present aspect of the invention.
  • the object to be cleaned may comprise an item of furniture (e.g. a chair, a table, a sofa), or a fixture of a house (e.g. a floor, a staircase), but the invention is not limited to furniture or fixtures.
  • the 3D electronic representation may be received from an external image capture device, such as a 2D camera or a 3D camera. Or, the 3D electronic representation may be received from an external computing device which has generated the representation in response to receiving the raw data from the 2D camera or the 3D camera.
  • the 3D electronic representation may be a computer-aided design (CAD) representation, which has already been prepared.
  • CAD computer-aided design
  • the 3D electronic representation may comprise a 3D point cloud representation.
  • a “3D point cloud representation” comprises a plurality of points in 3D space, each having a coordinate indicating its position in 3D space. The points correspond to locations on the outer surface of the object to be cleaned.
  • the invention requires that one or more continuous surface segments S i are identified using the surface segmentation algorithm, but it should be noted that not necessarily all of the points on the outer surface of the object to be cleaned fall within one of the continuous surface segments S i .
  • a continuous surface segment S i For example, areas of high curvature such as the crevices between sofa cushions, or the sharp edge of a square cross-section table leg might not be considered to fall within a continuous surface segment S i .
  • the treatment of these points on the outer surface S of the object to be cleaned is explained in detail elsewhere in this application.
  • there may be a threshold number of points falling within that surface for example 100 to 10,000 points, preferably around 3,000. In this way, small surfaces which may either be too small for an end effector to clean, or surfaces which result from anomalous points, will not be included in the continuous surface segments identified using the computer-implemented method of the first aspect of the invention.
  • the invention also requires that the curvature of each of the continuous surface segments S i is below a predetermined curvature threshold.
  • a predetermined curvature threshold i.e. the curvature in the neighbourhood of a give point
  • the local curvature i.e. the curvature in the neighbourhood of a give point
  • each point e.g. from the point cloud representation
  • this may be determined.
  • continuous within the meaning of the present application should be understood to mean that there are no breaks in the surface segment, neither are there any sharp changes in curvature. Mathematical definitions of “continuous” will be set out later in the application.
  • One algorithm which may be used to identify the continuous surface segments S i is a region growing algorithm.
  • the region growing algorithm works by identifying a seed point, and then adding neighbouring points to the region if they meet smoothness and curvature criteria.
  • the region growing algorithm may comprise: identifying a first seed point P 1 and selecting a set of neighbouring points to the first seed point P 1 , such that the resulting surface sub- segment, comprising the first seed point P 1 and its neighbouring points has a curvature below the predetermined curvature threshold; and iteratively repeating the process, with each neighbouring point in the resulting surface sub- segment becoming a seed point.
  • the region growing algorithm may include the following steps: (a) selecting a point P 1 having the lowest curvature C 1 as a first initial seed point, the point P 1 having one or more neighbouring points; (b) identifying the subset of neighbouring points which meet: a predetermined smoothness criterion and a predetermined curvature criterion; (c) adding the identified subset of neighbouring points to a cluster of points forming a first continuous surface segment S 1 ; (d) repeating steps (b) and (c) using each of the identified subset of neighbouring points as new seed points, until no identified neighbouring points meet both the smoothness and curvature criteria.
  • the smoothness and curvature criteria represent different things.
  • the computer-implemented method may include a step of obtaining the curvature of all of the points in the 3D electronic representation of the outer surface S of the object to be cleaned.
  • Obtaining the curvature of a given point may comprise estimating or calculating the normal to the surface at the point.
  • the curvature may then be calculated based on at least the estimated or calculated normal to the point.
  • it may be possible to calculate the curvature by using the residuals from the eigenvalue local plane fitting problem. This is advantageous since it is already computed when estimating or calculating the normals to the surface.
  • the identified subset may include all of the neighbouring points.
  • the algorithm identifies all of the points which neighbour each other, and which meet the smoothness and curvature criteria, i.e. one of the continuous surface segments S i .
  • the neighbouring points which do not meet the criteria effectively represent the points immediately surrounding the continuous surface segment S i around its edge.
  • step (d) it should be understood that only the identified neighbouring points from step (b) are treated as the new seed points: the original seed point is discounted after its neighbours have been identified.
  • the algorithm may further include (preferably on each iteration) a step of adding the identified neighbours which meet the smoothness and curvature criteria to a list, cluster, or set of points which are included in the continuous surface segment S i in question. 1 https://pcl.readthedocs.io/en/latest/normal_estimation.html
  • the process may further comprise determining the angles between the normal of the seed point and the respective normals of each of its neighbouring points. Then, the subset of those points which meet the predetermined smoothness criterion may be added to a current region (i.e. a set of points which form part of the continuous surface segment being considered).
  • the subset of neighbouring points whose curvature meets the predetermined curvature threshold may be determined, and the points of the subset may be added to the set of seed points.
  • the current seed may be removed from the set of seeds. If the set of seeds is empty, then the algorithm has then found the limits of the current region, and the process returns to the step of determining the point with the lowest curvature (of the remaining points). If not, then, for a next seed point of the set of seed points, the process continues from the step of identifying the subset of neighbouring points of that seed point. The process is repeated until no seed points remain in the set of seed points, at which point the process returns to the step of identifying determining the point with the lowest curvature.
  • the smoothness criterion may be that the seed point and the neighbouring point are not separated by an edge.
  • An edge may be concave (in which case it may be referred to as a “crevice”), or convex.
  • an edge is a sharp line between two surfaces.
  • an edge may be defined based on the relative angle between the normals of two respective points. If there is relative angle between the normals of two points, it will be appreciated that there is a sharp change in angle of the surface containing those two points. This sharp change in angle indicates that the surface between those two points is not smooth, within the meaning of the present invention.
  • Identifying the neighbouring points which meet the smoothness criterion may comprise: determining the angle ⁇ n between the normal N 1 of the first seed point and each of its neighbouring points P n ; selecting the set of neighbouring points for which the angle ⁇ n is below a predetermined threshold ⁇ TH .
  • the angle ⁇ TH is preferably between 0.0 and 1.0, preferably around 0.05 radians.
  • the predetermined smoothness threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or surface-contacting end- effector thereof; the nature of the cleaning task to be performed; and the nature of the outer surface to be cleaned.
  • curvature criterion refers to the range of movement of the various components of the 3D cleaning tool.
  • the curvature criterion should also be met.
  • the curvature criterion may be that the local curvature at the neighbouring point in question is below the predetermined curvature threshold.
  • local curvature should be understood to mean the curvature in the neighbourhood of the point.
  • neighborbourhood may be defined as the number of points within a predetermined radium, or the N nearest neighbours, where N is a predetermined number.
  • Identifying the points which meet the curvature criterion may comprise: obtaining the local curvature of the identified neighbours and selecting the set of neighbouring points for which the local curvature is below the predetermined curvature threshold.
  • curvature we refer to any parameter which may be used to parametrize the extent to which the outer surface S is curved a given point. Curvature may be represented as the inverse of the radius of curvature, which is the radius of a sphere which best approximates the shape of the surface S at the point.
  • the predetermined curvature threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or surface-contacting end- effector thereof; the nature of the cleaning task to be performed; and the nature of the outer surface to be cleaned.
  • the curvature threshold may be between 0.0 and 2.0, preferably around 0.2.
  • the step of identifying the neighbouring points which meet the smoothness and curvature criteria is effectively a two-part step and may comprise: identifying the subset of neighbouring points which meet the smoothness criterion; and identifying the subset of that subset of neighbouring points which also meet the curvature criterion.
  • the step may comprise: identifying the subset of neighbouring points which meet the curvature criterion; and identifying the subset of that subset of neighbouring points which also meet the smoothness criterion.
  • the identification in each of these cases may proceed as outlined above.
  • the subset which meet the smoothness criterion is identified first, since this requires the calculation of the normals to the neighbouring points, from which the local curvatures can then be derived, in order to identify the points meeting the curvature criterion, however it is equally valid to perform the tasks in either order, since the normals may be calculated upon retrieval of the full model, so will be calculated anyway, and also because the comparison for each point is a commutative check.
  • the computer- implemented method further comprises: of the remaining points of the plurality points which have not yet been added to a cluster representing a continuous smooth surface S i , selecting a point P 2 having the lowest curvature C 2 as a second initial seed point; and repeating steps (b) to (d) for the second initial seed point P 2 , thereby generating a cluster of points forming a second continuous surface segment S 2 .
  • the computer- implemented method may further comprise: of the remaining points of the plurality points which have not yet been added to a cluster representing a continuous smooth surface S i , selecting a point P i having the lowest curvature C i as an i th initial seed point; and repeating steps (b) to (d) for the i th initial seed point P i , thereby generating a cluster of points forming a i th continuous surface segment C i .
  • the output of the region growing algorithm is thus a plurality of clusters of points, each cluster comprising points (more specifically, comprising data identifying points) which are located within a respective continuous surface segment S i .
  • these clusters of points may be used to generate a tool path for the end-effector of the 3D cleaning tool to follow in order to clean the object as required.
  • the above description relates to the use of a region growing algorithm for identifying the various continuous surface segments S i .
  • a random sample consensus method algorithm herein, “RANSAC algorithm”
  • a RANSAC algorithm approximates a surface using e.g.
  • RANSAC algorithms are advantageous because they are accurate and quick to compute for objects such as tables and chairs which include easily parameterizable surfaces (e.g. planar surfaces). RANSAC is not applicable only to planar surfaces, it may be applied to any easily parameterizable surface, such as a parabolic, cylindrical or spherical surface too, or a portion thereof.
  • the RANSAC algorithm may be configured to identify a first plurality of parameters of a model which represents a first continuous surface segment S i and the computer-implemented method may further comprise identifying a first subset of points which are located within the first continuous segment S 1 and adding them to a first cluster of points which form the first continuous surface segment S 1 .
  • the RANSAC algorithm may include the steps of: (a)selecting N points from the plurality of points, wherein N is at least the number of points required to estimate the parameters of a parametric model representing the continuous surface segment S i ; (b) determining the values of the parameters for a continuous surface segment S i including the N selected points; (c) determining how many points of the plurality of points fit the model within a predetermined tolerance level; and (d) repeat steps (a) to (c) a predetermined number of times, and identify the set of parameters which maximize the value determined in step (c).
  • the computer-implemented method (preferably the RANSAC algorithm itself) may further include the step of identifying a parametric model which is appropriate to the object to be cleaned.
  • “tolerance” may refer to maximum (e.g. a maximum perpendicular) distance from the surface on which the parametric model is based, e.g.
  • Step (c) defined above requires that the number of points of the plurality of points fit within the predetermined tolerance be determined. This may include the steps of: identifying all of the points which are within the predetermined tolerance level of the surface defined by the parametric model; and counting the number of identified points. After the set of parameters which maximizes the number of points within a predetermined tolerance of the surface defined by the parametric model has been identified, additional steps may be carried out in order further to improve the identification of the surface of the object to be cleaned.
  • the RANSAC algorithm may further comprise: re-estimating the model parameters using all of the points which are located within the continuous surface segment S i defined by the model having the set of parameters identified in step (d) above, to generate a final set of model parameters.
  • the final set of model parameters so generated are likely to give a more accurate representation of the surface in question, because a larger number of initial points N are used.
  • the computer-implemented method may further comprise: applying the RANSAC algorithm to the remaining points which have not been added to the first cluster of points, to identify a second set of parameters of a model which represents a second continuous surface S 2 .
  • the identification of the second continuous surface S 2 may take place in the same manner as the identification of the first continuous surface segment S 1 .
  • the computer- implemented method may further comprise: applying the RANSAC algorithm to the remaining points which have not yet been added to a cluster of points, to identify an i th set of parameters of a model which represents an i th continuous surface segment S i .
  • This general identification step may proceed in the same manner as before.
  • the steps set out above relate to a regular RANSAC algorithm. However, in some cases, a connected RANSAC algorithm may be used. This is beneficial when the object (or objects) to be cleaned include two or more non-contiguous surfaces which nevertheless happen to conform to different regions of the same parametric model.
  • the RANSAC algorithm might determine that all of those non-contiguous coplanar regions to be part of the same continuous surface segment S i , whereas in reality, there are actually a plurality of separate continuous surface segments S i .
  • the object to be cleaned includes a table and four chairs, where each of the four chairs has a generally planar seat.
  • the RANSAC algorithm would most likely conclude that all four of the seat surfaces are part of the same surface (since they likely all fall roughly within the same plane, to within a given tolerance). It may be desirable to avoid this, which can be achieved using a connected RANSAC algorithm.
  • a connected RANSAC algorithm identifies only regions within the parametric model in which all of the points are connected. For example, a point will only be considered to form part of the identified region if it is within a predetermined distance of at least one other point in the region.
  • the computer-implemented method may include identifying a first subset of points which are located within a predetermined distance (i.e. tolerance) of the surface defined by the parametric model; and identifying one or more connected regions of points within the first subset of points.
  • Identifying one or more connected regions of points within the first subset of points may comprise using a region growing algorithm (independently of a region growing algorithm which may be used to identify the continuous surface segments S i , as described earlier in this application).
  • the computer-implemented method may further comprise: selecting a first seed point P i from the first subset of points; identifying all neighbouring points which are within a predetermined threshold distance of the first seed point; repeating this process with the identified neighbouring points becoming new first seed points, until there are no neighbouring points within the predetermined threshold distance of the most recent set of neighbouring points. At that point, the first seed point and all identified neighbouring points are considered to represent a first continuous surface segment S 1 , and added to a cluster of points.
  • This process may then be repeated by selecting, from the remaining points in the first subset of points, a second seed point P 2 , and repeating the above process to identify a second continuous surface segment S 2 .
  • a third seed point may then be selected, and so on, until all of the points from the first subset of points have been assigned to a cluster representing a respective continuous surface segment S i .
  • the computer- implemented method may further comprise: fitting a new parametric model from the remaining points, and determining a second subset of points which are located within a predetermined distance (i.e. tolerance) of the surface defined by the parametric model; and identifying one or more connected regions of points within the second subset of points.
  • Identifying the one or more connected regions within the second subset of points may proceed as outlined for the first subset of points.
  • the computer- implemented invention may comprise: identifying, from the remaining points, fitting a new parametric model, determining an i th subset of points which are located within a predetermined distance of a surface defined by the new parametric model; and identifying one or more connected regions of points within the i th subset of points. This process may be repeated until all points in the 3D electronic representation of the object to be cleaned have been assigned to a cluster of points representing a respective continuous surface segment S i .
  • the computer-implemented method may comprise the application of both a RANSAC algorithm and a region growing algorithm.
  • the application of a RANSAC algorithm may comprise the application of both a regular RANSAC algorithm and a connected RANSAC algorithm.
  • the output of the computer-implemented method is preferably one of more clusters of points, each cluster comprising a respective plurality of points from the 3D electronic representation of the outer surface S of the object to be cleaned. The points in a given cluster correspond to one of the identified continuous surface segments S i .
  • a second aspect of the invention provides a method of identifying one or more continuous surface segments of an object to be cleaned by a 3D cleaning tool, the method comprising: generating an electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points P i , each point representative of a point on the outer surface of the object to be cleaned; transmitting the generated electronic representation to a computer or other data processing unit; and performing the computer-implemented method of the first aspect of the invention.
  • generating the electronic representation may comprise obtaining a CAD representation of the object to be cleaned; or obtaining a 3D representation from a 3D camera or a 2D camera.
  • a third aspect of the present invention provides an input generation system for a control system for a 3D cleaning tool, the system comprising: an input generation module configured to generate a representation of an outer surface S of an object to be cleaned, the representation comprising a plurality of points P i , each point representative of a point on the outer surface of the object to be cleaned; a transmitter; and a computer or other data processing unit, wherein: the transmitter is configured to transmit the generated electronic representation to a computer or other data processing unit; and the computer or other data processing unit is configured to perform the computer-implemented method of the first aspect of the invention.
  • the computer-implemented method of the first aspect of the invention may further comprise generating a 3D cleaning tool path based on the one or more identified continuous smooth surface segments.
  • Generating a 3D cleaning tool path based on the one or more identified continuous smooth segments may be implemented in the form of a computer-implemented method of generating a 3D tool path defining how a 3D cleaning tool moves across one or more continuous surface segments S i during a cleaning operation, the computer-implemented method comprising: for a first continuous surface segment S 1 , receiving continuous surface segment data comprising an indication of a first plurality of points in 3D space which are part of the first continuous surface segment S 1 ; generating a representation of the first continuous surface segment S 1 ; and determining a first 3D cleaning tool path which covers the representation of the first continuous surface segment S 1 , the first 3D cleaning tool path comprising a first plurality of path points to be traversed, in operation, by the 3D cleaning tool.
  • the term “continuous” surface may mean that all points of the surface are located within a predetermined threshold distance of at least one other point in the surface. All points in a continuous surface may also meet a predetermined smoothness criterion which ensures that there are no sharp edges between two points within a surface. This is explained in more detail elsewhere in this application. All points on a continuous surface may also meet a predetermined curvature criterion which stipulates that the local curvature at a given point must be below a predetermined curvature threshold. Again, this is explained in more detail elsewhere in this application.
  • the step of determining a first 3D cleaning tool path may comprise or refer to generating a first 3D generating tool path.
  • the computer-implemented method may further comprise a step of generating instructions to send to a 3D cleaning tool.
  • Such instructions when executed by the 3D cleaning tool, preferably cause the 3D cleaning tool, specifically the end effector thereof, to follow the generated or determined first (or other) 3D cleaning tool path across the surface of the object to be cleaned.
  • the 3D cleaning tool path “covers” the representation of the first continuous surface segment.
  • covers should be understood to mean that all or substantially all of the representation of the continuous surface segment S i , when taking into account the dimensions of the end-effector which is intended to perform the cleaning task. Overlap between strokes of the end- effector may also be borne in mind, as described in more detail later.
  • the 3D cleaning tool path generated may comprise one or more (most likely a plurality) of strokes, wherein a stroke refers to one movement of the end-effector.
  • the path may comprise a plurality of strokes which are straight lines (or substantially straight lines), which may be parallel to each other. Between each of the strokes, the path may comprise a turn, in which the end-effector changes direction in order to place it in the correct position for the subsequent stroke.
  • the computer-implemented method includes a step of generating a representation of the first continuous surface segment S i .
  • the representation of the first continuous surface segment S i may be an isoparametric representation.
  • an “isoparametric representation” refers to a representation in which a parametric surface is generated, wherein each point on the parametric surface corresponds to a point on the “real” continuous surface segment S i .
  • the isoparametric representation is an 2D isoparametric surface representation. This means that every point on the continuous surface segment S i can be defined based on two variables u and v. So, where a point may be represented as e.g. P(x,y,z) in real, or Euclidean space, in the parametric representation, it may be represented in the form S(u,v). Ways in which the isoparametric representation may be generated are explained in more detail later in the application.
  • the computer-implemented method preferably further comprises: determining a path which covers the isoparametric representation of the first continuous surface segment S i in the parametric space; and transforming the path in the isoparametric representation of the first continuous surface segment S i into a path in the original (i.e. “real”) first continuous surface segment S i .
  • the 3D tool cleaning path may include, or be in one of the following forms: a plurality of parallel lines traversing the isoparametric representation of the first continuous surface segment S i from one side to the opposite side, either in the same direction or in a serpentine fashion; a plurality of concentric circles; a plurality of concentric shapes, the perimeter of each of which is parallel to the perimeter of the isoparametric representation of the first continuous surface segment S i ; and a spiral.
  • a serpentine fashion refers to a path which passes from one side to the other of the isoparametric representation, then changes direction and returns to the other side in the opposite direction.
  • the cleaning may take place throughout the whole path, without the surface-contacting end- effector having to separate from the surface to return to the other side.
  • parallel may also cover substantially parallel.
  • the path may comprise a plurality of the above geometries, i.e. a combination of two or more of the geometries outlined above, for example to perform more than one cleaning task, or to account for obstacles in the surface.
  • the selection of the path geometry is liable to vary, for example based on one or more of the following: the nature of the cleaning task to be performed; the nature of the surface to be cleaned; and the geometry/kinematics of the 3D cleaning tool, or the surface-contacting end-effector thereof.
  • the object to be cleaned is a table or a sofa
  • strokes which are straight, parallel, and/or continuous may be used, perhaps going from one extreme of the surface to the other (the isoplanar method described below is useful there).
  • a wiping task might require a circular concentric strokes or corner concentric motions (the isoparametric approach is useful in these cases).
  • Different kinds of cleaning tools might also be chosen for each task.
  • the number of points on a polynomial spline which is used to fit the path. A minimum path length, wherein paths which are shorter than the minimum path length are discarded.
  • An extrapolation limit which defines quantitatively whether a stroke finishes earlier or later than its calculated endpoint.
  • another representation may be used, namely an isoplanar representation, in which the first continuous surface segment S i is represented by a mesh comprising a plurality of contiguous geometric shapes; and determining the path which covers the first continuous surface segment S i comprises determining a path which covers the isoplanar representation of the first continuous surface segmentation S i .
  • the mesh comprises a plurality of triangles.
  • Determining the path which covers the isoplanar representation of the first continuous surface segment may comprise: intersecting the isoplanar representation of the first continuous surface segment S 1 with a plurality of cutting planes, the intersections of the cutting planes and the isoplanar representation of the first continuous surface segment S 1 forming the path to be traversed by the 3D cleaning tool.
  • the path is serpentine, and preferably the cutting planes are parallel to each other.
  • other geometries such as those described earlier for the isoparametric representation may also be used for the isoplanar representation, with the difference that, in the isoplanar case, there is no need to transform the paths back into the real, Euclidean space.
  • the “active area” is the part of the end-effector which actually performs the cleaning function when in contact with the surface, e.g. the aperture of a vacuuming tool, or the wet area of a wiping tool. This may be achieved by a consideration of the overlap between adjacent strokes of the end-effector tool when the 3D cleaning tool path is actually executed by the 3D cleaning tool.
  • a “stroke” refers to a path of the end-effector tool, in use.
  • Each stroke has a given width (i.e. an extent in a direction which is parallel or substantially parallel to the direction of movement of the end-effector), which is predetermined based on the geometry of the end-effector (specifically the active area thereof).
  • the computer-implemented method may further comprise receiving: a predetermined overlap threshold defining a minimum amount of overlap between adjacent parallel strokes of the 3D cleaning tool path to be generated; or a predetermined stroke spacing defining a maximum spacing between adjacent parallel strokes of the 3D cleaning tool path to be generated; and the 3D cleaning tool path is determined such that either: the overlap between adjacent parallel strokes of the 3D cleaning tool path is greater than the predetermined overlap threshold; or a maximum path between adjacent parallel strokes of the 3D cleaning tool is less than the predetermined stroke spacing threshold. More generally, the path is preferably determined such that every point in the first continuous surface segment S i (or, equivalently, the representation thereof) is covered by the active area of the end-effector of the 3D cleaning tool when the path is executed.
  • the 3D cleaning tool is a robotic tool which has a predetermined kinematic profile.
  • the term “kinematic profile” is a profile which contains data indicative of the allowable range of movement of the various components of the 3D cleaning tool. It is possible to determine, based on the kinematic profile of the 3D cleaning tool, whether the end- effector is able to adopt a particular position, in a particular orientation. Such a kinematic profile can be used as a filter on data representing points in space, identifying which points can be reached by the end-effector, and which points can’t.
  • the computer-implemented method may further comprise initially filtering the continuous surface segment data, wherein filtering comprises: determining, based on data indicating the geometry and/or kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the plurality of points; and removing the points which cannot be accessed from the continuous surface segment data, to generate filtered continuous surface segment data; and generating the 3D representation is based on the filtered continuous surface segment data.
  • the computer-implemented method may further comprise filtering the first plurality of path points, wherein filtering comprises: determining, based on data indicating the geometry and kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the first plurality of path points; and removing the points which cannot be accessed from the continuous surface segment data, to generate a filtered first plurality of path points. In some cases, this may be preferable to the previous approach since fewer points may be considered.
  • the data indicating the geometry and/or kinematics of the 3D cleaning tool may be in unified robot description format (URDF).
  • a given object to be cleaned may comprise a plurality of continuous surface segments S i .
  • the computer- implemented method may further comprise: for a second continuous surface segment S 2 , receiving continuous surface segment data comprising an indication of a second plurality of points in 3D space which are part of the second continuous surface segment S 2 ; generating a 3D representation of the second continuous surface segment S 2 ; and determining a second 3D cleaning tool path which covers the 3D representation of the second continuous surface segment S 2 , the second 3D cleaning tool path comprising a second plurality of path points to be traversed, in operation, by the 3D cleaning tool.
  • the computer-implemented method may comprise: for an i th surface segment S i , receiving continuous surface segment data comprising an indication of an i th plurality of points in 3D space which are part of the i th continuous surface segment S i ; generating a 3D representation of the i th continuous surface segment S i ; and determining an i th 3D cleaning tool path which covers the 3D representation of the i th continuous surface segment S i , the i th 3D cleaning tool path comprising a i th plurality of path points to be traversed, in operation, by the 3D cleaning tool.
  • the computer-implemented method may comprise a step of receiving continuous surface segment data which comprises a respective plurality of points in 3D space for each of the continuous surface segments S i .
  • the computer-implemented method may further comprise: filtering the second plurality of path points, wherein filtering comprises: determining, based on data indicating the geometry and kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the second plurality of path points; and removing the points which cannot be accessed from the continuous surface segment data, to generate a filtered second plurality of path points.
  • the computer-implemented invention may comprise filtering the i th plurality of path points, wherein filtering comprises: determining, based on data indicating the geometry and kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the i th plurality of path points; and removing the points which cannot be accessed from the continuous surface segment data, to generate a filtered i th plurality of path points. All of the optional features set out above in respect of filtering the first plurality of path points apply equally well to filtering the second or i th pluralities of path points.
  • the invention further comprises generating a set of instructions to be sent to the 3D cleaning tool, the set of instructions based on one or more of the following: the first plurality of path pints, the filtered first plurality of path points, the second plurality of path points, the filtered second plurality of path points, the i th plurality of path points, and the filtered i th plurality of path points.
  • the instructions are preferably such that when they are received by the 3D cleaning tool, or, for example, a data processing unit or module thereof, they cause the 3D cleaning tool (specifically, the end-effector thereof) to execute the generated 3D cleaning tool path.
  • the invention may further comprise a transmission step, whereby the instructions are sent to the 3D cleaning tool, or a component thereof.
  • the invention may further include a method of controlling a 3D cleaning tool to clean, using a surface- contacting end-effector, one or more continuous surface segments S i of an object to be cleaned, the end-effector adapted to clean the one or more continuous surface segments S i , the method comprising: performing the computer-implemented method of the first aspect of the invention; generating a set of instructions to be sent to the 3D cleaning tool, the set of instructions based on one or more of the following: the first plurality of path points, the filtered first plurality of path points, the second plurality of path points, the filtered second plurality of path points, the i th plurality of path points, and the filtered i th plurality of path points; and transmitting the generated set of instructions to the 3D cleaning tool, the set of instructions operable
  • systems of some aspects of the present invention may further comprise a control system for a 3D cleaning tool, the control system configured to control a surface-contacting end-effector of the 3D cleaning tool to clean one or more continuous surface segments S i of an object to be cleaned, the control system comprising: a computer or other data processing unit configured to perform the computer-implemented method of the first aspect of the invention, and to generate a set of instructions to be sent to the 3D cleaning tool, the set of instructions based on one or more of the following: the first plurality of path points, the filtered first plurality of path points, the second plurality of path points, the filtered second plurality of path points, the i th plurality of path points, and the filtered i th plurality of path points; and a transmitter configured to send the generated set of instructions to the 3D cleaning tool, the set of instructions operable to cause the 3D cleaning tool to clean the one or more continuous surface segments using the end-effector.
  • a control system for a 3D cleaning tool the control system configured to control
  • Generation of the set of instructions may further comprise fitting a spline to the path.
  • a further aspect of the invention may provide a 3D cleaning tool comprising the input generation system and the control system.
  • the computer-implemented method of the first aspect of the invention may further comprise identifying one or more crevices of the object to be cleaned by the 3D cleaning tool.
  • this may be achieved by further including a computer-implemented method of identifying one or more crevices of an object to be cleaned by a 3D cleaning tool: receiving a 3D electronic representation of an outer surface of the object to be cleaned, the representation comprising a plurality of points P i, each point representative of a point on the outer surface of the object; and identifying the subset of points within the plurality of points having concave local curvature which exceeds a predetermined local curvature threshold.
  • the 3D electronic representation may be in the form of a point cloud.
  • the term “local curvature” should be understood to mean the curvature in the neighbourhood of the point.
  • the “neighbourhood” may be the region which is delineated by an imaginary line connecting all of the closest neighbours (e.g. neighbours which are located within a given distance, or a predetermined number of nearest neighbours).
  • the identified subset of points are those where the concave local curvature “exceeds” a predetermined local curvature threshold.
  • the term “exceeds” may be interpreted to mean either “is greater than”, or “is greater than or equal to”. It may be necessary to differentiate between a crevice and an “edge”.
  • edge is used to refer to a sharp convex region, such as a ridge between two smooth surfaces.
  • an edge is the convex equivalent of a (concave) crevice.
  • Identifying the subset of points may comprise, for each point within the plurality of points: determining the respective local curvature C i of each point within the plurality of points; comparing the respective local curvature with the predetermined local curvature threshold; and selecting those points for which the respective local curvature exceeds the predetermined local curvature threshold.
  • curvature we refer to any parameter which may be used to parametrize the extent to which the outer surface S is curved a given point. In some cases, curvature may be represented as the inverse of the radius of curvature, which is the radius of a sphere which best approximates the shape of the surface S at the point.
  • curvature is interpreted as “radius of curvature”, then in order to identify the crevices it is necessary to identify those points for which the radius of curvature is below a threshold value (since a more curved surface has a smaller radius of curvature). Preferred implementations of the computer-implemented method of the first aspect of the present invention do not use the radius of curvature, however, as we discuss in detail below. Determining the respective curvature of any given point may comprise estimating or calculating the normal to the surface at the point. The curvature may then be calculated based on at least the estimated or calculated normal to the point.
  • a measure of the curvature may be calculated by using the residuals from the eigenvalue local plane fitting problem, which gives a dimensionless curvature value. More detail about this calculation process is set out later in this application. Calculating curvature in this way is advantageous since it may be done at practically no additional computational cost when calculating the normals.
  • the measure of curvature is unsigned when calculated in this manner, and it may therefore be necessary to perform additional steps in order to determine e.g. whether a given point is ridge or a crevice. Regions having a high curvature may be either crevices or edges. In order to identify crevices, it is necessary to identify only the concave points.
  • the computer-implemented method may comprise: determining whether the average difference between that nearest neighbour and the tangent plane defined by the normal n i to the point is positive or negative, and if the average difference is positive, determining that the local curvature is concave, and if the difference is negative, determining that the local curvature is convex.
  • the computer-implemented method may further comprise: adding the point to the plurality of points which are determined to be in a crevice.
  • the computer- implemented method may comprise discarding the point.
  • the predetermined local curvature threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or surface-contacting end- effector thereof; the nature of the cleaning task to be performed; and the nature of the outer surface to be cleaned.
  • Example values for the curvature threshold may range from 0.0 to 0.5 m, preferably around 0.01 m.
  • distance units are used, as they correspond to an average distance from the tangent plane to a point. Points having a curvature exceeding the curvature threshold are considered part of a crevice. It should be stressed here that this means that the signed value of the curvature should be higher than this value, for an area of high curvature to be considered a crevice.
  • the local curvature must be concave, i.e. the signed value of the curvature should be no more than zero in order to meet the requirements of a crevice.
  • aspects of the present invention may further comprise a method of identifying one or more crevices of an object to be cleaned by a 3D cleaning tool, the method comprising: generating an electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points P i , each point representative of a point on the outer surface of the object to be cleaned; transmitting the generated electronic representation to a computer or other data processing device; and performing the computer- implemented method discussed above.
  • Some aspects of the present invention provide an input generation system. However, this input generation system may be modified in order to provide an input relating to crevices too.
  • the input generation system for a control system for a 3D cleaning tool may further comprise: an input generation module configured to generate a representation of an outer surface S of an object to be cleaned, the representation comprising a plurality of points P i , each point representative of a point on the outer surface of the object to be cleaned; a transmitter; and a computer or other data processing unit, wherein: the transmitter is configured to transmit the generated electronic representation to a computer or other data processing unit; and the computer or other data processing unit is configured to perform the computer- implemented method of the first aspect of the invention.
  • the method further comprises the identification of crevices
  • the generation of a 3D cleaning tool path will also need to take account of the crevices.
  • the computer-implemented method of the first aspect of the invention may further comprise determining which points of the outer surface correspond to crevices that may need to be cleaned.
  • This may be achieved by incorporating a computer-implemented method of generating a 3D tool path defining how a 3D cleaning tool moves along one or more crevices during a cleaning operation, the computer- implemented invention comprising: receiving data comprising an indication of a plurality of crevice points, which are points in 3D space which are located within a crevice; determining a path which traverses the one or more crevices by converting the points located in a crevice into a respective plurality of nodes, the nodes defining the 3D cleaning tool path.
  • the overall output of the method of such a computer- implemented method is a path comprising a plurality of waypoints.
  • waypoint should be understood to refer to a point along the path which the end-effector (preferably an active area thereof) passes through, in use.
  • a “node” here refers to a point in 3D space, and may interpreted similarly to, or identically to, “waypoint”. Determining a path may use graphical techniques. For example, in some cases, determining a path may comprise generating a graph based on the plurality of crevice points.
  • a “graph” is a visualizable representation of the relationship between the locations of points in 3D space.
  • an actual visual representation may be generated, and e.g. displayed to a user.
  • the visual representation may not actually be generated.
  • the step of generating a graph may comprise generating a convenient coordinate system for representing the plurality of crevice points.
  • Generating the graph may comprise downsampling the data comprising the indication of the plurality of crevice points at a predetermined downsampling resolution, to generate a plurality of nodes, each node having an associated surface normal and position.
  • the “surface normal” represents a vector which is perpendicular to a tangent plane to the surface at the node.
  • the normal and position may be referred to as “node attributes”.
  • downsampling refers to a process of sampling a subset of the plurality of crevice nodes at a resolution which is less than the original resolution. This necessarily results in fewer nodes.
  • Pointcloud downsampling refers in general to the process of reducing the number of points in a pointcloud by selecting or creating a subset of the original set of points while keeping as much information as possible.
  • the method which may be applied here is often referred to voxel grid downsampling, whereby a 3D mesh grid is created. For each cubic volume in the grid (a voxel), all the enclosed points are averaged, and the output is the resulting average point (also called centroid).
  • the downsampling resolution represents the size of the edge of each of the voxels, or in other words, the size step at which the grid is created.
  • One way in which the downsampling process may take place is to superimpose a 3D grid, or lattice onto the plurality of crevice points. Then, for each cell defined by the grid or lattice, which may contain one or more crevice points therewithin, generating a point which represents the average location of all of the crevice points within that cell. In such cases, the size of the cell may be referred to as the downsampling resolution.
  • voxel size This may also be referred to as a “voxel size”.
  • the voxel size may be around 1cm.
  • the downsampling resolution if the resolution is too coarse, details on the path may be missed and displacement of the downsampling grid may heavily influence the results. However, if the resolution is too fine, the number of nodes and connections may create parallel paths along a single crevice edge, which means that it can take a long time to compute a complete traversal which covers all branches.
  • generating the graph may further comprise generating a plurality of edges between neighbouring nodes.
  • neighborebouring nodes may be defined as notes which are located within a first predetermined distance threshold from each other.
  • the first predetermined distance is preferably approximately 1.5 times as large as the downsampling resolution. This ratio has been shown to give rise to a balanced amount of edges between points.
  • Each edge preferably has a normal difference, and a relative displacement associated therewith.
  • the “relative displacement” represents the displacement (i.e. the distance and direction) between the two nodes in Euclidean space, and may be represented by the vector resulting from subtracting the position of one node from the position of the other node.
  • each edge may have a normal difference associated therewith.
  • the resulting nodes or the original crevice points
  • the normals of those points may not represent the true desired direction of the 3D cleaning tool at that point.
  • the computer-implemented method of the first aspect of the invention may further comprise a step of graph reduction, in which connected nodes are iteratively averaged out, merging those which end up nearby each other.
  • the output of the graph reduction process is a graph skeleton which uniquely represents each of the crevices in the outer surface of the object to be cleaned, and any branches thereof.
  • the aim of graph reduction is to eliminate any small branches of crevices which may result from overly fine sampling, and which do not represent the true geometry of the object to be cleaned.
  • graph reduction may comprise: (a) for each node, computing an average position and average normal based on the attributes of that node and all nodes connected to it, and generating a respective average node with those attributes, the generated plurality of average nodes replacing the initial plurality of nodes (here, the averaging step does not remove the node, it simply replaces its attributes with the average values of the connected nodes; this is in contrast to the merging step which follows, in which the number of nodes is reduced); (b) merging pairs of average nodes which are located within a second predetermined distance threshold from each other, to generate a modified plurality of average nodes (each node of the modified plurality of nodes having the average of the attributes of the two nodes which merged to form it; the threshold may be, for example about 1 cm, or preferably, equal to the voxel size or downsampling resolution); and (c) repeating, a predetermined number of times (for example,
  • the second predetermined distance threshold may be approximately the same as the downsampling resolution, in order to give rise to an overall output resolution which is the same as the initial one.
  • determining a path may further comprise applying a complete traversal algorithm to an input, where the input is either: the plurality of crevice points; the plurality of nodes; the generated graph; or the reduced graph.
  • the output of a complete traversal algorithm is a path which traverses all of the points or nodes of the input, preferably no more than once (though in some cases, this may not be possible).
  • the complete traversal algorithm may comprise identifying and separating the input into one or more connected components.
  • a “connected component” is a plurality of points, all of which are connected, in e.g. a chain or branching structure.
  • the complete traversal algorithm may comprise, for the input, or for each connected component of the input: (a)identifying the nodes which are the furthest away from each other and identifying the shortest path between those nodes, and saving the identified path; and (b) removing the nodes on the identified shortest path; (c) for the remaining nodes after the nodes have been removed in step (b), repeating steps (a) and (b) until all nodes in the input, or connected component of the input have been assigned to a shortest path.
  • determining a path may comprise: identifying one or more branches within the plurality of crevice points, wherein identifying the one or more branches comprises: (a) selecting a crevice point of the plurality of crevice points, the selected crevice point forming a seed point; (b) identifying all of the crevice points of the plurality of crevice points which neighbour the seed point within a predetermined threshold distance, and selecting these points as new seed points; (c) repeating step (b) for the newly-selected seed points, until there are no crevice points which neighbour newly-generated seed points, at which point the branch is complete; and (d) repeating steps (a) to (c) until all crevice points of the plurality of crevice points have been assigned to a branch; (e) determining a path which traverses all of the identified branches.
  • Determining the path may comprise the use of a complete traversal algorithm as before.
  • the output of the complete traversal algorithms or other path determination processes is a list of paths, where each path corresponds to an ordered list of paths, such that all of the branches in the crevice area are visited.
  • this threshold number is 5 (i.e. paths with fewer than 5 points are not stored as a path). Since the nodes in the graph correspond to points in a point cloud, i.e. the crevice points, the attributes of those points can be used to find waypoints that can be sent to the 3D cleaning tool to move the crevice tool along those waypoints.
  • the computer-implemented method may further comprise: generating, based on the determined path, a set of instructions to be sent to the 3D cleaning tool.
  • Generating the set of instructions may comprise fitting a spline along the points on the determined path. This will be discussed in more detail later in the application.
  • the invention may further comprise a control system for a 3D cleaning tool, the control system configured to control a surface-contacting end-effector of the 3D cleaning tool to clean one or more crevices of an object to be cleaned, the control system comprising: a computer or other data processing unit configured to perform the computer-implemented method of the first aspect of the invention, and to generate a set of instructions based on the determined path; and a transmitter configured to send the generated set of instructions to the 3D cleaning tool, the set of instructions operable to cause the 3D cleaning tool to clean the one or more crevices using the end-effector according to the determined path.
  • a control system for a 3D cleaning tool the control system configured to control a surface-contacting end-effector of the 3D cleaning tool to clean one or more crevices of an object to be cleaned
  • the control system comprising: a computer or other data processing unit configured to perform the computer-implemented method of the first aspect of the invention, and to generate a set of instructions based on the determined path; and
  • - Fig. 1 is a high-level system diagram showing various components associated with the present invention.
  • - Fig. 2 is a schematic of an input generation system and a control system for a 3D cleaning tool.
  • - Figs. 3A and 3B are flowcharts illustrating processes which may be performed by the system of Fig. 1.
  • - Fig. 4 is an illustration of a representation of a chair showing continuous surface segments and crevices therebetween. The representation is shown on a computer screen.
  • - Fig. 1 is a high-level system diagram showing various components associated with the present invention.
  • - Fig. 2 is a schematic of an input generation system and a control system for a 3D cleaning tool.
  • - Figs. 3A and 3B are flowcharts illustrating processes which may be performed by the system of Fig. 1.
  • - Fig. 4 is an illustration of a representation of a chair showing continuous surface segments and crevices therebetween. The representation is shown on a computer
  • FIG. 5 is a flowchart showing high-level steps involved in the identification of continuous surface segments.
  • - Fig. 6 is a flowchart illustrating the steps of an exemplary RANSAC algorithm.
  • - Fig. 7A shows an example of 3D electronic representation of the outer surface of an object to be cleaned, in this case a table and chairs.
  • - Fig. 9 is a flowchart illustrating the high-level steps involved in generating a 3D cleaning tool path based on the detected continuous surface segments.
  • FIG. 10 is a flowchart illustrating a series of steps which may be used to generate a 3D cleaning tool path based on an isoparametric representation of continuous surface segments of an object to be cleaned.
  • - Fig. 11 is a schematic illustrating an SOM approach to generating a grid representation.
  • - Fig. 12 is a schematic illustrating various vectors which may be considered when generating isoparametric representations of continuous surface segments of an object to be cleaned.
  • - Figs. 13A to 13C show examples of 3D cleaning tool paths which may be generated based on an isoparametric representation of continuous surface segments of an object to be cleaned.
  • - Fig. 11 is illustrating an SOM approach to generating a grid representation.
  • - Fig. 12 is a schematic illustrating various vectors which may be considered when generating isoparametric representations of continuous surface segments of an object to be cleaned.
  • - Figs. 13A to 13C show examples of 3D cleaning tool paths which may be generated based on an
  • FIG. 14A is a flowchart illustrating a series of steps which may be used to generate a 3D cleaning tool path based on an isoplanar representation of continuous surface segments of an object to be cleaned.
  • - Fig. 14B is a schematic illustrating the use of intersecting cutting planes of an isoplanar representation of a continuous surface segment of an object to be cleaned, in order to generate a 3D cleaning tool path.
  • - Fig. 15A is a flowchart illustrating the high-level steps involved in detecting crevices in a 3D electronic representation of an outer surface of an object to be cleaned.
  • - Fig. 15B shows an example of an output of the flowchart of Fig. 15A.
  • FIG. 16 is a flowchart illustrating the high-level steps involved in generating a 3D cleaning tool path based on the detected crevices.
  • - Fig. 17 is a flowchart illustrating an exemplary graph generation process.
  • - Fig. 18 is a flowchart illustrating am exemplary graph reduction process.
  • - Figs. 19A to 19F show the outputs of various iterations of the graph reduction process of Fig. 18.
  • - Fig. 20 is a flowchart illustrating an exemplary complete traversal algorithm.
  • - Figs. 21A to 21C show the outputs of various steps in the flowchart of Fig. 20.
  • - Fig. 22 is a flowchart illustrating an exemplary instruction generation process.
  • the present disclosure relates to control of a 3D cleaning tool 102. Specifically, the disclosure relates to methods (including computer-implemented methods) for identifying various specified regions of an object to be cleaned (namely continuous surface segments and crevices), and processing data related to the identified regions in order to generate paths to be followed by a surface-contacting end- effector 110 of the 3D cleaning tool 102, in order to perform some cleaning task, such as vacuuming, brushing, wiping, or the like.
  • System 100 may include a 3D cleaning tool 102, a control system 104, an input generation system 105, a 3D camera 106, and an external computing device 108.
  • the input generation system 105 may be configured to receive inputs from the 3D camera 106 and the external computing device 108, either via a direct (e.g. wired) connection, or, for example, via a network (not shown), which may be a wired or wireless network such as a Wi-Fi network or a cellular network.
  • the 3D cleaning tool comprises an end-effector 110, and may be configured to receive inputs from the control system 104, again, either directly or via a network (also not shown).
  • the network may be the same network as or a different network from the network via which the input generation system 105 is configured to receive input from the 3D camera 106 and/or the external computing device 108.
  • the input generation system 105 and the control system 104 may be part of the same component.
  • the input generation system 105 may be part of the control system 104.
  • the two may be connected either directly, or via a network.
  • the connection may take the same form as the connections previously described, and may be the same or different connections and/or networks.
  • the input generation system 105 and the control system 104 are shown as separate modules. This should not be regarded as limiting.
  • the input generation system 105 may contain the modules which are configured to receive the 3D representation of the object to be cleaned from external sources such as the 3D camera 106 and the external computing device 108, and to identify the continuous surface segments and/or the crevices in the outer surface of an object to be cleaned.
  • the control system 104 may contain the modules which are configured to generate an appropriate tool path, and/or instructions for the 3D cleaning tool 102, and to transmit those instructions to the 3D cleaning tool 102.
  • Fig. 2 is a schematic diagram of an input generation system 105 and control system 104.
  • Input generation system 105 may comprise an interface module 1050 which may be configured to receive inputs from the 3D camera 106 and/or the external computing device 108.
  • Input generation system 105 may further comprise a processor 1052 and a memory 1054, which may include a buffer 1056.
  • Processor 1052 may comprise a surface segmentation module 1058 and a crevice detection module 1060.
  • the surface segmentation module 1058 may comprise a RANSAC module 1062 and a region growing algorithm (RGA) module 1064.
  • the input generation module may further include a control system interface module 1066.
  • Control system 104 may comprise an input generation system interface module 1068 via which it may be able to communicate with the input generation system 105.
  • Control system 104 like the input generation system 105, may also comprise a processor 1070 and a memory 1072, which may include a buffer 1074.
  • Processor 1070 may comprise a tool path generation (continuous surface segment, CSS) module 1076 and a tool path generation (crevice) module 1078.
  • the tool path generation module (CSS) 1076 may comprise a representation generation module 1080, which may comprise an isoplanar representation generation module 1082 and an isoparametric representation generation module 1084, and a filtering module 1086.
  • the tool path generation module (crevice) 1078 may include a graph generation module 1088, a graph reduction module 1090, and a traversal algorithm module 1092.
  • the control system 104 may further comprise an instruction generation module 1094, and a 3D cleaning tool interface module 1096, via which it may communicate with the 3D cleaning tool 102.
  • the input generation system 105 and/or the control system 104 may form part of the 3D cleaning tool. Alternatively, they may be separate components, such as computing devices e.g. a laptop, desktop PC, tablet, or smartphone.
  • Figs. 3A and 3B show flowcharts which illustrate high-level methods which may be performed by various aspects of this disclosure. In Fig.
  • step S1 performed by the surface segmentation module 1058 of the input generation module 105 is the identification of one or more continuous surface segments
  • step S2 performed by the tool path generation module (CSS) 1076 is the generation of a 3D cleaning tool path based on the identified continuous surface segments
  • step S3 performed by the crevice detection module 1060 of the input generation system 105 is the identification of one or more crevices
  • step S4 performed by the tool path generation module (crevice) 1078 is the generation of a 3D cleaning tool path based on the identified crevices.
  • an object to be cleaned e.g. an item of furniture may well include both continuous surface segments and crevices.
  • various aspects of the disclosure may include the methods of both Fig. 3A and Fig. 3B.
  • the methods of each of Figs. 3A and Fig. 3B may take place simultaneously, and in parallel with each other (e.g. independently of each other).
  • a further step may be included in which the resulting 3D cleaning tool paths are combined.
  • steps S1 and S3 may be performed first (in either order), and then the outputs of those steps may be sent simultaneously to the control system 104, and then steps S2 and S4 may take place afterwards (again, in either order).
  • steps S1 and S2 may take place, then steps S3 and S4 may follow.
  • steps S3 and S4 may take place first, followed by steps S1 and S2.
  • steps S1 to S4 may be performed in any order as long as step S1 takes place before S2, and step S3 takes place step S4.
  • the purpose of the surface segmentation process shown in Fig. 5 in the form of a flowchart is to identify the four separate continuous smooth surface segments (1), (2), (3), and (4), and preferably to indicate which of the plurality of points in the 3D electronic representation belong to each respective surface segment.
  • the 3D electronic representation of the outer surface of the object to be cleaned may be received by the input generation system 105 (e.g. from the 3D camera 106 or the external computing device 108, via the interface module 1050). Implicitly, it will be appreciated that prior to the method of Fig.
  • the 3D camera 106 may obtain a 3D representation of the object to be cleaned.
  • the 3D representation may exist in the form of a CAD representation on the external computing device 108, which simply transmits that representation to the input generation system 105.
  • a surface segmentation algorithm may be applied to the electronic representation of the outer surface S of the object to be cleaned, preferably by the surface segmentation module 1058 of processor 1052 of the input generation system 105.
  • the output of the surface segmentation algorithm is preferably an indication of one or more continuous surface segments S i . Two exemplary surface segmentation algorithms are now described. Fig.
  • N points are selected from the plurality of points, where N is at least the number of points required to estimate the parameters of a parametric model representing the continuous surface segment S i .
  • step S121 the model is fit to the N selected points, in order to find the values of a, b, c, and d.
  • step S122 the number of points (inliers) from the set of all of the input points (i.e. the points in the 3D representation) fit that model within a predetermined tolerance is determined. In other words, the number of points which are within a predetermined distance of the plane defined by the model is determined.
  • step S123 is then repeated X times, where X is predetermined value.
  • step S124 the model parameters which gave rise to the largest result in step S122 are selected.
  • step S125 the model parameters are re-determined based on all of the inliers of the model identified in step S124, to generate a final continuous surface segment S i .
  • step S126 it is determined whether there are any other points which are not accounted for. If not, the process ends, otherwise, the inliers are removed from consideration in step S127, and the process returns to step S120. This continues until all points in the 3D electronic representation are accounted for, at which point the algorithm is terminated.
  • a connected RANSAC algorithm may be used. This differs from the “normal” RANSAC algorithm in that it includes a further connected components step is performed on all the planar inliers found by RANSAC, and only the biggest connected cluster is retained.
  • Figs. 7A to 7C illustrate the effects of the RANSAC and connected RANSAC algorithms.
  • Fig. 7A shows an illustration of a 3D point cloud of a table and chairs.
  • Fig. 7B shows a schematic diagram of a surface segmentation which may be obtained using a RANSAC algorithm, where different types of shading illustrate different identified planar surfaces. From this, it will be noted that the planar surfaces identified from the tabletop and base of the tabletop extend to the middle of the backs of the chairs.
  • the RANSAC algorithm identifies all of the seats of the chairs as belonging to the same surface, as well as a layer in the table legs.
  • the RANSAC algorithm identifies a portion of the bottom right-hand table leg as belonging to the same surfaces as the backs of the chairs on the right-hand side.
  • Fig. 7C shows the same setup when a connected RANSAC algorithm is used, and shows that each of the seats of the chairs are identified as separate surfaces, and the plane of the table does not intersect the chair backs.
  • the RANSAC algorithm works perfectly well, this example is only illustrative of a situation in which a connected RANSAC algorithm is preferable, i.e. when there are a plurality of non-contiguous but coplanar smooth surfaces within on object or objects to be cleaned.
  • RGA region growing algorithm
  • a region growing algorithm is effective when segmenting surfaces of items where a planar (or other easily parameterizable) model cannot be used. So, while a RANSAC algorithm would be useful for e.g. tables, counters, shelves, stairs, it is preferable to use an RGA for furniture types such as sofas, chairs, beds, and bathtubs. These lists are, of course, not exhaustive. At a high-level, an RGA works iteratively by adding contiguous points to a surface as long as they are (a) not edges, and (b) have a low relative curvature.
  • the output of the algorithm is preferably a set of clusters, where each cluster contains a set of points that are considered to belong to the same smooth surface.
  • the 3D camera 106 may obtain a 3D representation of the object to be cleaned.
  • the 3D representation may exist in the form of a CAD representation on the external computing device 108, which simply transmits that representation to the input generation system 105.
  • the RGA may comprise computing the curvature value C i of all points in the point cloud.
  • curvature may be defined in 3D as the degree to which a surface deviates from a plane. Mathematically, it is measured as the reciprocal of the radius of curvature at the point where it is measured. In the present context, calculating the exact reciprocal of the radius of curvature is unnecessary, and computationally expensive, since the considered surfaces have varying curvatures at each point. So, instead, the curvature at any point P i may be computed via proxies that measure the deviation of the points surrounding P i from the plane given by the normal at that point n i .
  • Normals may be estimated by estimating the normal of a plane tangent to the surface, which takes the form of a least-square plane fitting estimation problem.
  • the 3D covariance matrix at each point will naturally have the largest eigenvectors in the directions tangent to the surface at that point (since it is where the points are). therefore, the normal at that point will be the third eigenvector, which is perpendicular (normal) to the previous two.
  • the normals and curvatures are therefore computed via the eigenvectors and eigenvalues of a covariance matrix C created from the k nearest neighbours of the query point P i as follows:
  • C represents the 3C centroid of the nearest neighbours around the point in question, is the j th eigenvalue and is t h the j eigenvector.
  • step S132 the points are sorted by their curvature value, and the point with the minimum curvature value is selected as the seed point for the first region.
  • step S134 a smoothness check is performed, in which the angle between the normal of the seed point and that of each of its neighbouring points is determined. The neighbouring points for which the angle is below a predetermined smoothness threshold ⁇ th are then added to the region.
  • step S136 the neighbouring points whose curvature C i is below a predetermined curvature threshold are selected. This step removes edges (i.e. regions with high local curvature) from the set of seeds, preventing the surface from growing past them.
  • step S138 the current (i.e. initial) seed is removed from the set of seeds. It is then determined whether the seed set is empty, in decision step S140. If the seed set is empty, this means that the algorithm has found the limits of the current region, so the method returns to step S132.
  • the method returns to step S134 for each seed point, using the new seeds as a starting point. The process continues until all points in the point cloud are accounted for.
  • the points which are included in a given region, i.e. continuous surface segment S i are selected based on various thresholds and tolerances. These thresholds and tolerances may be varied to suit a particular item of furniture, cleaning task, 3D cleaning tool 102 (or end- effector 110) geometry.
  • the thresholds may be variable, and can be changed by a user, e.g. via a user interface of the input generation system 105 (not shown), or of an external computing device such as external computing device 108 as shown in Fig. 1.
  • an external computing device such as external computing device 108 as shown in Fig. 1.
  • step S14 in which data representing the identified continuous surface segments S i may be transmitted to the control system 104, preferably the tool path generation module (CSS) 1076 thereof.
  • the method moves to step S2 of Fig. 3A, in which a 3D cleaning tool path is generated by the tool path generation module (CSS) 1076 of the processor 1070 of the control system 104.
  • a high-level flowchart illustrating the tool path generation process is set out in Fig. 9.
  • the tool path generation process starts after the continuous surface segment data has been received by the control system 104, specifically the tool path generation (CSS) module 1076 thereof.
  • a representation of a continuous surface segment S i is generated.
  • a 3D cleaning tool path is generated, which covers the representation of the continuous surface segment S i .
  • the generated path may comprise a plurality of path points which are to be traversed by the 3D cleaning tool path, in operation.
  • a representation of the surface is generated by the representation generation module 1080.
  • the data which is generated by e.g. the surface segmentation module 1058 of the processor 1052 of the input generation system 105 essentially includes several clusters of points (or data identifying those points), each cluster representing a different, respective continuous surface segment S i .
  • the purpose of generating a representation of the continuous surface segments S i is to generate a model of the surface in 3D space, such that a tool path may be plotted geometrically on that model, rather than the continuous surface segment S i being represented only by a somewhat abstract list of points. This meaning will become clearer when we consider examples of representations which may be generated by the representation generation module 1080.
  • the isoparametric representation generation module 1084 generates an isoparametric representation of the continuous surface segment S i .
  • the continuous surface segment S i is described in a parametric space.
  • the isoparametric representation may be in the form of a polynomial parametric surface, such as a non-uniform rational B-spline, or NURBS, which is a well-defined representation of 3D geometry which can accurately describe any shape.
  • a representation in the form of a NURBS may be generated by the isoparametric representation generation module 1084.
  • the topology of the continuous surface segment S i may be learned, e.g. using a self-organizing map (SOM) approach.
  • SOM self-organizing map
  • a self-organized map is an unsupervised machine learning technique which may be used to produce a low-dimensional (typically two-dimensional) representation of a higher- dimensional data set while preserving the topological structure of the data.
  • a SOM is a network of weights: The weights are topologically laid out as nodes in a lattice grid as shown in Fig. 11. Each weight represents the position of the node in the input (i.e.
  • the purpose of the SOM is to find the underlying topologies of an input space (in this case, the plurality of points making up the continuous surface segment S i ) with a reduced set of m nodes.
  • the lattice will assume the underlying shape of the continuous surface segment S i while preserving the property that neighbouring points in the continuous surface segment S i will remain close in the lattice.
  • the result is thus a discrete grid which assumes the shape of the underlying surface segment in an orderly fashion (i.e. it remains a grid).
  • a NURBS is fit to the grid representation, preferably by a maximum likelihood method, as described in the NURBS book 3 .
  • the output of this process is preferably a surface given by: 2 T. Kohonen, "The self-organizing map,” in Proceedings of the IEEE, vol.78, no.9, pp.1464-1480, Sept. 1990, doi: 10.1109/5.58325.
  • the surface S(u,v) is a parametric surface representation of the continuous surface segment, and one may trace paths in that space 0 ⁇ u,v ⁇ 1.
  • a 3D cleaning tool path may be generated in that parametric space, in step S206. Examples of 3D cleaning tool paths which may be generated from isoparametric representations which may be generated in the manner outlined above are shown in Figs.
  • ⁇ B represents the direction of the displacement in the real space.
  • represents the normal to the plane at a point ⁇ representing the differential displacement vector in the metric space as is changed.
  • representing the unit vector in metric space representing the direction in which one would travel as is changed by an amount
  • representing the unit vector in metric space representing the direction in which one would travel as is changed by an amount
  • differential geometry and linear algebra it is therefore possible to obtain a linear approximation to convert to the corresponding displacement in parametric space, This way, it is possible to map the desired displacement in metric space which is the spacing between strokes (dependent on tool geometry, etc.) to corresponding to this displacement, since we are tracing the path in metric space. Based on these, it is possible to calculate as follows:
  • the angle brackets in the denominator represent an inner product operation.
  • 13A shows an example in which the surface is to be cleaned using parallel strokes in parametric space.
  • ⁇ Input NURBS, or any other, parametric surface representing the continuous surface segment S i to be cleaned.
  • Let and ⁇ Let be a chosen constant spacing between the parallel strokes in real space, e.g. 0.05m. As discussed elsewhere in the application, this parameter may be dependent on cleaning tool geometry and the desired degree of overlap between adjacent cleaning strokes.
  • Let initially empty, be a list of cleaning strokes, where each stroke is given by a list of waypoints. While ⁇ 1, do: 1.
  • step S206 of Fig. 10 the generated 3D leaning tool path (preferably in the form of a list of waypoints) may optionally then be transmitted to the filtering module 1086, step S208.
  • the operation of the filtering module 1086 will be described later, after an explanation of the generation of a 3D tool cleaning path using the isoplanar representation. We now describe the processes involved in generating an isoplanar representation of a continuous surface segment S i .
  • the isoplanar method generates 3D cleaning tool paths by intersecting a plane with a given mesh model generated from an input continuous surface segment S i .
  • This process may be performed by the isoplanar representation generation module 1082 of the representation generation module 1080 of the tool path generation module (CSS) 1076 of the processor 1070 of the control system 104.
  • the steps are illustrated in Fig. 14A.
  • a triangular mesh is generated, and in step S304, a 3D tool path is generated based on the triangular mesh.
  • BPA ball-pivoting algorithm
  • the principle of the BPA is very simple: Three points form a triangle if a ball of a user-specified radius p touches them without containing any other point. Starting with a seed triangle, the ball pivots around an edge (i.e., it revolves around the edge while keeping in contact with the edge's endpoints) until it touches another point, forming another triangle. The process continues until all reachable edges have been tried, and then starts from another seed triangle, until all points have been considered. The process can then be repeated with a ball of larger radius to handle uneven sampling densities.
  • a detailed explanation of the ball- pivoting algorithm can be found in Bernardini et al. (1999) 4 .
  • step S304 the 3D cleaning tool path is generated.
  • FIG. 14B An example algorithm by which the path may generated is as follows.
  • a schematic diagram showing the plurality of cutting planes is provided in Fig. 14B
  • ⁇ Input triangular mesh model representing the surface to be cleaned.
  • Let be the maximum and minimum limits of the surface.
  • Let be the position and orientation of the 3D cleaning tool 102 (specifically, the end-effector 110 thereof) with respect to a fixed coordinate frame (the continuous surface segment S i to be cleaned), representing by a 3 ⁇ 4 matrix, where all elements are column vectors of three dimensions.
  • Based on the current orientation of the robot contained in the equation of a cutting plane is defined as where a basic choice for can be either of the horizontal vectors given by the 3D cleaning tool’s pose This choice may be dependent on cleaning task, and desired direction of cleaning strokes as well as the choice of reference frame. Setting defines the plane at the starting edge of the surface, which can then gradually be moved towards 4 F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva and G. Taubin, "The ball-pivoting algorithm for surface reconstruction," in IEEE Transactions on Visualization and Computer Graphics, vol.5, no.4, pp.349-359, Oct.-Dec.1999, doi: 10.1109/2945.817351.
  • Let initially empty, be a list of cleaning strokes, where each stroke is given by a list of waypoints.
  • Let be a chosen constant spacing between parallel strokes, e.g. 0.05m. This parameter, as discussed elsewhere, may be dependent on e.g. the geometry of the 3D cleaning tool, and the desired degree of overlap between adjacent cleaning strokes.
  • the algorithm may then proceed as follows: 1. Let 2. While do: a. Let be a stroke generated from intersecting the mesh model with the plane equation given by b. Move the plane forward: i. Update the plane position c.
  • the final output of the algorithm may then be a list of parallel strokes which are spaced by approximately
  • the generated 3D cleaning tool path may optionally then be transmitted to the filtering module 1086, in step S306.
  • the filtering tool may be configured to determine based on data indicating the geometry and kinematics of the 3D cleaning tool 102, e.g. in the form of unified robot description format data, whether the end-effector 110 of the 3D cleaning tool 102 is able to access all of the generated waypoints of each 3D cleaning tool path. If not, the inaccessible points are removed from the plurality of waypoints.
  • the 3D cleaning tool path may be rejected entirely.
  • the threshold may take the form of a simple number of waypoints, or may be in terms of an accessible area of the continuous surface segment S i in question.
  • the output of the filtering module 1086 may be a filtered plurality of points forming the waypoints of the generated 3D cleaning tool path, and may be transmitted from the control system 104 to the 3D cleaning tool 102, for execution.
  • the instruction generating module 1094 may generate some executable instructions based on the filtered plurality of points, and these instructions may be sent to the 3D cleaning tool 102 for execution.
  • Crevice detection may be regarded as the complementary problem to the detection of smooth continuous surfaces. While smooth surfaces are sets of contiguous points with low local curvature, crevices can be characterized as the sets of contiguous points with a high, concave local curvature. We therefore now consider the detection of crevices, and the subsequent generation of 3D cleaning tool paths for cleaning those crevices. Crevice detection, i.e. step S3 of Fig.
  • step S4 of Fig. 3B may be performed by the crevice detection module 1060 of the processor 1052 of the input generation system 105. Then, in step S4 of Fig. 3B, tool path generation based on the detected crevices, may be performed by the tool path generation module (crevice) 1078 of the processor 1070 of the control system 104.
  • the crevices are the narrow trench regions between e.g. surfaces (1) and (2), (1) and (3), (1) and (4), (2) and (3), and (2) and (4).
  • the purpose of the crevice detection process shown in Fig. 15 in the form of a flowchart is to identify the crevices described above, and preferably to indicate which of the plurality of points in the 3D electronic representation are located in each respective crevice.
  • the 3D electronic representation of the outer surface of the object to be cleaned may be received by the input generation system 105 (e.g. from the 3D camera 106 or the external computing device 108, via the interface module 1050). Implicitly, it will be appreciated that prior to the method of Fig.
  • the 3D camera 106 may obtain a 3D representation of the object to be cleaned.
  • the 3D representation may exist in the form of a CAD representation on the external computing device 108, which simply transmits that representation to the input generation system 105.
  • the curvature of each of the points in the 3D representation is determined in step S32. This may be done by considering the eigenvalues of the covariance matrix, as explained earlier in this application. For conciseness, the explanation will not be repeated here.
  • step S34 those points having a sufficiently high curvature are identified, e.g. by comparison with a threshold.
  • the concave points are identified.
  • steps S34 and S36 may be reversed.
  • the eigenvalue method of estimating curvature does not yield any information as to whether the surface is concave or convex, and accordingly, an additional step may be required.
  • the average distance from its k neighbours to the tangent plane perpendicular to n i at the point is obtained. If this distance is positive, it means that the neighbouring points are above the plane, and thus the curvature at the point P i is concave. And, if the distance is negative, it means that the neighbouring points are below the plane, and thus the curvature at P i is convex.
  • a sign can be added to the curvature values in order to designate concavity or convexity, e.g. + for concave, or – for convex (or vice versa). Then, from that list of signed curvatures, the concave points which have a curvature greater than the predetermined local curvature threshold may be selected. Now that steps S34 and S36 have been completed, the process moves to step S38, in which data representing the identified crevices may be transmitted to the control system 104, preferably the tool path generation module (crevice) 1078 thereof.
  • this step S38 may not take place.
  • An example of an output of the crevice detection process is shown in Fig. 15B in which the light regions represent the detected crevices of an armchair.
  • the method moves to step S4 of Fig. 3B, namely the generation of a 3D cleaning tool path based on the detected crevices.
  • This step is performed by the tool path generation module (crevice) 1078, which may comprise: a graph generation module 1088, a graph reduction module 1090, and a traversal algorithm module 1092. As is indicated from the names of the modules, and shown in Fig.
  • the tool path generation process may include, in step S40 generation of a graph by the graph generation module 1088, in step S42 reduction of the graph by the graph reduction module 1090, and in step S44 applying a complete traversal algorithm to the reduced graph by the traversal algorithm module 1092.
  • the output of this process is, as with the 3D cleaning tool paths generated in respect of the continuous surface segments, a path in the form of e.g. a series of nodes.
  • the instruction generation module 1094 may, based on the path, generate instructions, perhaps in the form of a plurality of waypoints to be visited by the end-effector 110 of the 3D cleaning tool 102, in use.
  • the path may include a series of segments, each in turn made up from a six-dimensional vector specifying the position and orientation of the end-effector 110 of the 3D cleaning tool 102, akin to the paths generated in respect of the continuous surface segments, explained in detail above.
  • crevices include elongated stretches which are long in one dimension (i.e. along the crevice), but short in the other two dimensions (i.e. the boundary surfaces).
  • the points in the crevice have, by definition, high concave curvature.
  • a single detected crevice may comprise a plurality of branches.
  • Fig. 17 shows a graph generation process, in which a graph is generated from the 3D electronic representation of the outer surface of the object to be cleaned, which may be in the form of a point cloud. Nodes in the graph correspond to points and normals, and the position and normals of the points are stored as node attributes.
  • the edges between the nodes store information on the relationship between the points which they connect.
  • the point cloud may be downsampled at a predetermined downsampling resolution, which may be expressed as a voxel size. Every resulting point from the downsampling may then be considered a node of the graph.
  • edges are generated between pairs of nodes which are located within a threshold distance of each other.
  • edge attributes are calculated, namely a Euclidean distance between the two nodes, and a normal difference.
  • step S42 aims to address some of these issues.
  • step S502 for each pair of calculated nodes, an average node is calculated, which is located halfway along the edge connecting those nodes. Then, in step S504, pairs of average nodes which are within a predetermined threshold distance (preferably the same as the voxel size or downsampling resolution) of each other are merged with each other. This process acts to reduce the number of nodes.
  • the process shown in Fig. 18 may be repeated a predetermined number of times, in order to obtain a reduced graph. In this way, it is possible to start with a graph generated using a very fine downsampling resolution, and end up with a reduced graph which represents every crevice branch uniquely. Figs.
  • step S44 of Fig. 16 takes, place: the application of a complete traversal algorithm to the reduced graph generated in step S42.
  • the purpose of the complete traversal algorithm is to identify one or more paths along the various branches of the detected crevices so that all of the determined points on the graph are covered or traversed by the end-effector 110 of the 3D cleaning tool 102.
  • various forms of complete traversal algorithm may be used, and the algorithm discussed now with reference to Fig. 20 is only one preferable example.
  • the input to the algorithm is either the generated graph, or the reduced graph. In some cases, the input may be the raw data in the form of the 3D point cloud, or other electronic representation of the points which are located within the crevices of the object to be cleaned.
  • the algorithm works by identifying the longest shortest path (i.e. the shortest path spanning two extremes of the graph), storing (e.g. to a buffer 1074) that path, and removing those nodes from consideration. The process then repeats itself until all nodes have been covered, or the remaining paths are too short (i.e. below a predetermined length threshold, or number of nodes).
  • a minimum spanning tree is computed, in order to remove cycles from the graph. Intuitively, this just refers to cutting any loops that may exist in the original graph.
  • a cycle in a graph refers to a situation where, starting from node A, and following the edges, it is possible to return to the same node without going twice over any edge. Removing the cycles means that now, departing from node A, it is only possible to return to node A again by backtracking, and hence going at least twice over some of the edges. Then, in order to remove small side branches, or other small branches, in step S442, the tree is pruned. This may include removing all points from the graph which are located in branches which are below a threshold length (either in terms of actual length, or number of nodes). If Fig. 21A shows the original graph, the output of steps S440 and S442 are shown in Fig. 21B, which shows the pruned minimum spanning tree.
  • step S444 The crevice areas on a piece of furniture (or other object to be cleaned) may not be all be connected in one large tree – there may be several disconnected trees.
  • connected components are identified in step S444, and the identified connected components are separated.
  • one of the connected components may be selected (e.g. the largest, or selection may be random).
  • the next step may be to identify the longest shortest path between all nodes in the selected connected component. In order to do so, in step S446 the two nodes which are located the furthest away from each other may be identified, and in step S448, the shortest path (i.e. through the nodes of the graph) between those two points may be identified, a result of which may be shown in Fig. 21C.
  • this path may be added to e.g. a list of paths which may be stored in the buffer 1074, or otherwise stored, at least temporarily.
  • the nodes on the graph from the identified path may be removed from consideration.
  • step S454 it may be determined whether all of the nodes in the connected component (with the possible exception of outliers) have been included in a path. If not, the method may return to step S446. If so, the method may proceed to step S456 in which it may be determined whether all of the connected components have been processed. If not, the process may return to step S445, and a set of paths may be generated for a next connected component.
  • step S456 If the result of the determination in step S456 is positive, a path has been generated which traverses all points in the graph (or raw data), and the algorithm ends.
  • the result of the algorithm is a list of paths, where each path corresponds to an ordered list of nodes, such that all of the branches corresponding to the crevices of the object to be cleaned are visited.
  • Nodes in the graph correspond (at least approximately) to points in the point cloud, so their attributes (i.e. the node attributes) can be used to find, in step S46 of Fig. 16, waypoints that can be sent to the 3D cleaning tool 102, in order to cause the end-effector 110 thereof (in this case preferably in the form of a crevice cleaning tool) to traverse those waypoints.
  • the following process of generating waypoints may also be applied in the case of the path generated to cover the continuous surface segments, as explained previously.
  • the steps may be performed by the instruction generation module 1094 of the control system 104, and are illustrated in Fig. 22.
  • the instruction generation module 1094 may receive the paths generated by the tool path generation module (CSS) 1076 or tool path generation module (crevice) 1078.
  • CCS tool path generation module
  • crevice tool path generation module
  • x, y, and z may be used to refer to the XYZ vectors which define a frame of reference.
  • a frame of reference defines a 6 degree of freedom “pose” (position and orientation) with respect to any other frame of reference.
  • the end-effector of the 3D cleaning tool has a frame of reference that describes its pose with respect to the reference frame in the base of the robot.
  • the XYZ vectors defined here create a list of frames of reference (one per waypoint) which are the poses the end-effector aims to accomplish in order to complete the cleaning task. At this point, for a given object to be cleaned, such as the chair in Fig. 4, the continuous surface segments and/or the crevices therebetween may be identified.
  • instructions for the 3D cleaning tool 102 may be generated, in order to enable the end-effector 110 thereof to perform the desired cleaning task.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Manufacturing & Machinery (AREA)
  • Evolutionary Computation (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Human Computer Interaction (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Remote Sensing (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Computing Systems (AREA)
  • Electromagnetism (AREA)
  • Algebra (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Processing Or Creating Images (AREA)
  • Numerical Control (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

A computer-implemented method of identifying one or more continuous smooth surface segments of an object to be cleaned by a 3D cleaning tool is provided, the computer-implemented method comprising : receiving a 3D electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points P 1, each point representative of a point on the outer surface of the object to be cleaned; applying a surface segmentation algorithm to the electronic representation of the surface S, thereby identifying one or more continuous surface segments S 1 of the surface S, wherein : each of the continuous surface segments S 1 comprises a subset of the plurality of points, and the curvature of each of the continuous surface segments S 1 is below a predetermined curvature threshold. Corresponding methods and an input generation system are also provided.

Description

CONTINUOUS SMOOTH SURFACE SEGMENTATION TECHNICAL FIELD OF THE INVENTION The present invention relates to a computer-implemented method of identifying one or more continuous smooth surface segments of an object to be cleaned by a 3D cleaning tool. The invention also covers a related method and an input generation system for a control system for a 3D cleaning tool. BACKGROUND TO THE INVENTION 3D cleaning tools are devices such as robotic devices which may be placed adjacent to an object to be cleaned, such as an item of furniture, or a fixture of a house such as a staircase or floor, and set to clean the item automatically. In order to clean the object effectively, the 3D cleaning tool must first be able to build an electronic picture of the object to be cleaned, so that it can subsequently generate instructions to control the operation of the device. In order to ensure that the 3D cleaning tool operates effectively, it is necessary to ensure that it is able to build an accurate, useful electronic picture or digital representation of the object to be cleaned, and then that it is able to generate accurate instructions which control the operation of the device. The majority of the surfaces of furniture such as tables, chairs, and sofas are in the form of continuous, smooth, surfaces. These may be cleaned e.g. by brushing, sweeping, vacuuming, or wiping. In order to plan such tasks, it is first necessary accurately to identify those surfaces. The present invention relates to processes (and other related aspects) for identifying these surfaces. SUMMARY OF THE INVENTION At a high level, the present invention relates to methods for identifying one or more surfaces of an object to be cleaned by applying a surface segmentation algorithm to an electronic representation of an outer surface of the object. According to the invention, the segments are identified using a predetermined curvature threshold. Specifically, a first aspect of the present invention provides a computer-implemented method of identifying one or more continuous smooth surface segments of an object to be cleaned by a 3D cleaning tool, the computer-implemented method comprising: receiving a 3D electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; applying a surface segmentation algorithm to the electronic representation of the surface S, thereby identifying one or more continuous surface segments Si of the surface S, wherein: each of the continuous surface segments Si comprises a subset of the plurality of points, and the curvature of each of the continuous surface segments Si is below a predetermined curvature threshold. The use of a surface segmentation algorithm ensures that an accurate picture of the surfaces of the object to be cleaned can be build, which subsequently enables the 3D cleaning tool to clean the object thoroughly and accurately. Examples of surface segmentation algorithms which may be used in implementations of the present invention are set out and explained later on in this application. The term “3D cleaning tool” as used in this application should be understood to cover any device which is adapted to clean an outer surface of a 3D object. The 3D cleaning tool preferably comprises an end-effector which is configured to contact the outer surface of the object to be cleaned in order to perform the cleaning task. The end-effector may comprise, for example, a vacuum cleaner head, a wipe, a brush, and the like. A 3D cleaning tool preferably also comprises a motion system which is configured to position the end-effector in the correct position and/or orientation. The motion system may comprise one or more robotic arms which are connected at an elbow. The robotic arms may be rotatable about one or more axes relative to each other. The end-effector may be rotatable about one or more axes relative to a distal robotic arm. A proximal robotic arm may be rotatable about one or more axes relative to a base portion of the 3D cleaning tool. The 3D cleaning tool may further comprise a control system, optionally located in the base portion, which may be configured to generate and/or transmit instructions to the motion system, the instructions configured to cause the motion system, and optionally the end-effector to move relative to each other in order to position the end-effector in a desired position and/or orientation. More details relating to the control system will be set out later in this application. It should be stressed that computer-implemented methods according to the first aspect of the invention may be used in combination with other 3D cleaning tools which do not take the form described in this paragraph, without departing in any way from the scope of the invention. The term “outer surface” as used in this application refers to any exposed surface which an external 3D cleaning tool is able to access. The outer surface may comprise a plurality of constituent surfaces, some of which may be flat, and some of which may be curved. The outer surface S may comprise a plurality of constituent smooth continuous surfaces, to be identified using the computer-implemented invention of the present aspect of the invention. The object to be cleaned may comprise an item of furniture (e.g. a chair, a table, a sofa), or a fixture of a house (e.g. a floor, a staircase), but the invention is not limited to furniture or fixtures. The 3D electronic representation may be received from an external image capture device, such as a 2D camera or a 3D camera. Or, the 3D electronic representation may be received from an external computing device which has generated the representation in response to receiving the raw data from the 2D camera or the 3D camera. Alternatively, the 3D electronic representation may be a computer-aided design (CAD) representation, which has already been prepared. Such a CAD representation may be a design based on which the object was manufactured. In preferred implementations, the 3D electronic representation may comprise a 3D point cloud representation. Herein, a “3D point cloud representation” comprises a plurality of points in 3D space, each having a coordinate indicating its position in 3D space. The points correspond to locations on the outer surface of the object to be cleaned. The invention requires that one or more continuous surface segments Si are identified using the surface segmentation algorithm, but it should be noted that not necessarily all of the points on the outer surface of the object to be cleaned fall within one of the continuous surface segments Si. For example, areas of high curvature such as the crevices between sofa cushions, or the sharp edge of a square cross-section table leg might not be considered to fall within a continuous surface segment Si. The treatment of these points on the outer surface S of the object to be cleaned is explained in detail elsewhere in this application. In order to be classified as a continuous surface segment Si, there may be a threshold number of points falling within that surface, for example 100 to 10,000 points, preferably around 3,000. In this way, small surfaces which may either be too small for an end effector to clean, or surfaces which result from anomalous points, will not be included in the continuous surface segments identified using the computer-implemented method of the first aspect of the invention. The invention also requires that the curvature of each of the continuous surface segments Si is below a predetermined curvature threshold. This may be understood to mean that the local curvature (i.e. the curvature in the neighbourhood of a give point) at each point (e.g. from the point cloud representation) in a given continuous surface segment Si is below the predetermined curvature threshold. When discussing the algorithms in more detail, it will become apparent how this may be determined. The term “continuous” within the meaning of the present application should be understood to mean that there are no breaks in the surface segment, neither are there any sharp changes in curvature. Mathematical definitions of “continuous” will be set out later in the application. One algorithm which may be used to identify the continuous surface segments Si is a region growing algorithm. At a high level, the region growing algorithm works by identifying a seed point, and then adding neighbouring points to the region if they meet smoothness and curvature criteria. Specifically, the region growing algorithm may comprise: identifying a first seed point P1 and selecting a set of neighbouring points to the first seed point P1, such that the resulting surface sub- segment, comprising the first seed point P1 and its neighbouring points has a curvature below the predetermined curvature threshold; and iteratively repeating the process, with each neighbouring point in the resulting surface sub- segment becoming a seed point. More specifically, the region growing algorithm may include the following steps: (a) selecting a point P1 having the lowest curvature C1 as a first initial seed point, the point P1 having one or more neighbouring points; (b) identifying the subset of neighbouring points which meet: a predetermined smoothness criterion and a predetermined curvature criterion; (c) adding the identified subset of neighbouring points to a cluster of points forming a first continuous surface segment S1; (d) repeating steps (b) and (c) using each of the identified subset of neighbouring points as new seed points, until no identified neighbouring points meet both the smoothness and curvature criteria. Here, the smoothness and curvature criteria represent different things. The curvature concerns the local curvature at a point itself, i.e. in the neighbourhood of the point. A neighbouring point may have very low curvature, but if the point is e.g. around the corner of a square table leg, the space between the two points clearly has a sharp change of direction in it. The space between the two points therefore cannot be considered smooth. For the purposes of the present invention it is highly desirable that both smoothness and curvature criteria are met. Before step (a), the computer-implemented method (preferably the region growing algorithm itself) may include a step of obtaining the curvature of all of the points in the 3D electronic representation of the outer surface S of the object to be cleaned. Obtaining the curvature of a given point may comprise estimating or calculating the normal to the surface at the point. There are many well-established ways to compute normals, including for example those listed on the Point Cloud Library1. The curvature may then be calculated based on at least the estimated or calculated normal to the point. In some cases, it may be possible to calculate the curvature by using the residuals from the eigenvalue local plane fitting problem. This is advantageous since it is already computed when estimating or calculating the normals to the surface. It should be noted that in step (b) mentioned above, the identified subset may include all of the neighbouring points. By repeating steps (b) and (c) until no identified neighbouring points meet both the smoothness and curvature criteria, the algorithm identifies all of the points which neighbour each other, and which meet the smoothness and curvature criteria, i.e. one of the continuous surface segments Si. The neighbouring points which do not meet the criteria effectively represent the points immediately surrounding the continuous surface segment Si around its edge. In step (d), it should be understood that only the identified neighbouring points from step (b) are treated as the new seed points: the original seed point is discounted after its neighbours have been identified. The algorithm may further include (preferably on each iteration) a step of adding the identified neighbours which meet the smoothness and curvature criteria to a list, cluster, or set of points which are included in the continuous surface segment Si in question. 1 https://pcl.readthedocs.io/en/latest/normal_estimation.html In a slightly alternative process, after the point with the minimum curvature value has been selected as the seed point, the process may further comprise determining the angles between the normal of the seed point and the respective normals of each of its neighbouring points. Then, the subset of those points which meet the predetermined smoothness criterion may be added to a current region (i.e. a set of points which form part of the continuous surface segment being considered). Then, the subset of neighbouring points whose curvature meets the predetermined curvature threshold may be determined, and the points of the subset may be added to the set of seed points. Then, the current seed may be removed from the set of seeds. If the set of seeds is empty, then the algorithm has then found the limits of the current region, and the process returns to the step of determining the point with the lowest curvature (of the remaining points). If not, then, for a next seed point of the set of seed points, the process continues from the step of identifying the subset of neighbouring points of that seed point. The process is repeated until no seed points remain in the set of seed points, at which point the process returns to the step of identifying determining the point with the lowest curvature. The smoothness criterion may be that the seed point and the neighbouring point are not separated by an edge. An edge may be concave (in which case it may be referred to as a “crevice”), or convex. Essentially, an edge is a sharp line between two surfaces. In preferred cases, an edge may be defined based on the relative angle between the normals of two respective points. If there is relative angle between the normals of two points, it will be appreciated that there is a sharp change in angle of the surface containing those two points. This sharp change in angle indicates that the surface between those two points is not smooth, within the meaning of the present invention. Identifying the neighbouring points which meet the smoothness criterion may comprise: determining the angle θn between the normal N1 of the first seed point and each of its neighbouring points Pn; selecting the set of neighbouring points for which the angle θn is below a predetermined threshold θTH. In radians, the angle θTH is preferably between 0.0 and 1.0, preferably around 0.05 radians. In some cases, the predetermined smoothness threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or surface-contacting end- effector thereof; the nature of the cleaning task to be performed; and the nature of the outer surface to be cleaned. Herein, “kinematics” refers to the range of movement of the various components of the 3D cleaning tool. In addition to the smoothness criterion, the curvature criterion should also be met. The curvature criterion may be that the local curvature at the neighbouring point in question is below the predetermined curvature threshold. Herein, the term “local curvature” should be understood to mean the curvature in the neighbourhood of the point. The “neighbourhood” may be defined as the number of points within a predetermined radium, or the N nearest neighbours, where N is a predetermined number. Identifying the points which meet the curvature criterion may comprise: obtaining the local curvature of the identified neighbours and selecting the set of neighbouring points for which the local curvature is below the predetermined curvature threshold. By “curvature”, we refer to any parameter which may be used to parametrize the extent to which the outer surface S is curved a given point. Curvature may be represented as the inverse of the radius of curvature, which is the radius of a sphere which best approximates the shape of the surface S at the point. In some cases, the predetermined curvature threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or surface-contacting end- effector thereof; the nature of the cleaning task to be performed; and the nature of the outer surface to be cleaned. The curvature threshold may be between 0.0 and 2.0, preferably around 0.2. The step of identifying the neighbouring points which meet the smoothness and curvature criteria is effectively a two-part step and may comprise: identifying the subset of neighbouring points which meet the smoothness criterion; and identifying the subset of that subset of neighbouring points which also meet the curvature criterion. Alternatively, the step may comprise: identifying the subset of neighbouring points which meet the curvature criterion; and identifying the subset of that subset of neighbouring points which also meet the smoothness criterion. The identification in each of these cases may proceed as outlined above. Preferably, the subset which meet the smoothness criterion is identified first, since this requires the calculation of the normals to the neighbouring points, from which the local curvatures can then be derived, in order to identify the points meeting the curvature criterion, however it is equally valid to perform the tasks in either order, since the normals may be calculated upon retrieval of the full model, so will be calculated anyway, and also because the comparison for each point is a commutative check. The processes explained above relate only to the identification of a first continuous smooth surface S1 using the region growing algorithm. Of course, for the majority of objects to be cleaned, there will be a plurality of surfaces which require attention. With that in mind, when it is determined that no neighbouring points of a set of new seed points meet both the predetermined smoothness criterion and the predetermined curvature criterion, the computer- implemented method further comprises: of the remaining points of the plurality points which have not yet been added to a cluster representing a continuous smooth surface Si, selecting a point P2 having the lowest curvature C2 as a second initial seed point; and repeating steps (b) to (d) for the second initial seed point P2, thereby generating a cluster of points forming a second continuous surface segment S2. In other words, after a first cluster of points has been identified as belonging to a first continuous surface segment S1, the remaining point with the lowest curvature is selected, and the region growing process is applied again to detect a second continuous surface segment S2. This process continues until all continuous surface segments have been identified. To generalize, when it is determined that no neighbouring points of a set of most recently identified seed points meet both the predetermined smoothness and curvature criteria, the computer- implemented method may further comprise: of the remaining points of the plurality points which have not yet been added to a cluster representing a continuous smooth surface Si, selecting a point Pi having the lowest curvature Ci as an ith initial seed point; and repeating steps (b) to (d) for the ith initial seed point Pi, thereby generating a cluster of points forming a ith continuous surface segment Ci. The output of the region growing algorithm is thus a plurality of clusters of points, each cluster comprising points (more specifically, comprising data identifying points) which are located within a respective continuous surface segment Si. As we will explain later in this application, these clusters of points may be used to generate a tool path for the end-effector of the 3D cleaning tool to follow in order to clean the object as required. The above description relates to the use of a region growing algorithm for identifying the various continuous surface segments Si. Alternatively, or additionally, a random sample consensus method algorithm (herein, “RANSAC algorithm”) may be used. Generally speaking, a RANSAC algorithm approximates a surface using e.g. parametric model, and then aims to identify all points in the 3D electronic representation which fall within a threshold distance of a surface represented by that parametric model. This process is then repeated until all points (within reason) are accounted for, or preferably performed until a model is identified that fits a threshold number of threshold of points within the point cloud. RANSAC algorithms are advantageous because they are accurate and quick to compute for objects such as tables and chairs which include easily parameterizable surfaces (e.g. planar surfaces). RANSAC is not applicable only to planar surfaces, it may be applied to any easily parameterizable surface, such as a parabolic, cylindrical or spherical surface too, or a portion thereof. The RANSAC algorithm may be configured to identify a first plurality of parameters of a model which represents a first continuous surface segment Si and the computer-implemented method may further comprise identifying a first subset of points which are located within the first continuous segment S1 and adding them to a first cluster of points which form the first continuous surface segment S1. In order to do so, the RANSAC algorithm may include the steps of: (a)selecting N points from the plurality of points, wherein N is at least the number of points required to estimate the parameters of a parametric model representing the continuous surface segment Si; (b) determining the values of the parameters for a continuous surface segment Si including the N selected points; (c) determining how many points of the plurality of points fit the model within a predetermined tolerance level; and (d) repeat steps (a) to (c) a predetermined number of times, and identify the set of parameters which maximize the value determined in step (c). The computer-implemented method (preferably the RANSAC algorithm itself) may further include the step of identifying a parametric model which is appropriate to the object to be cleaned. Examples of parametric models which may be used are a plane, which may be represented by the equation: ax + by + cz + d = 0 In which a, b, c, and d are parameters to be estimated. Of course, alternative equations may also be used to represent a plane. As discussed previously, other parametric models which may be used include cylindrical surfaces, spherical surfaces, parabolic surface, paraboloidal surfaces, hyperbolic surfaces, hyperboloidal surfaces, and conical surfaces, the defining equations of which are well-known. The N points are selected randomly or pseudo-randomly. Herein, “tolerance” may refer to maximum (e.g. a maximum perpendicular) distance from the surface on which the parametric model is based, e.g. the maximum perpendicular distance from a plane or parametric surface. The tolerance may be between 0.1 and 5 cm, preferably around 0.4cm. Step (c) defined above requires that the number of points of the plurality of points fit within the predetermined tolerance be determined. This may include the steps of: identifying all of the points which are within the predetermined tolerance level of the surface defined by the parametric model; and counting the number of identified points. After the set of parameters which maximizes the number of points within a predetermined tolerance of the surface defined by the parametric model has been identified, additional steps may be carried out in order further to improve the identification of the surface of the object to be cleaned. For example, the RANSAC algorithm may further comprise: re- estimating the model parameters using all of the points which are located within the continuous surface segment Si defined by the model having the set of parameters identified in step (d) above, to generate a final set of model parameters. The final set of model parameters so generated are likely to give a more accurate representation of the surface in question, because a larger number of initial points N are used. After the RANSAC algorithm has been used to identify a first continuous surface segment Si, the computer-implemented method may further comprise: applying the RANSAC algorithm to the remaining points which have not been added to the first cluster of points, to identify a second set of parameters of a model which represents a second continuous surface S2. The identification of the second continuous surface S2 may take place in the same manner as the identification of the first continuous surface segment S1. For conciseness, those steps are not repeated here. In a general case, the computer- implemented method may further comprise: applying the RANSAC algorithm to the remaining points which have not yet been added to a cluster of points, to identify an ith set of parameters of a model which represents an ith continuous surface segment Si. This general identification step may proceed in the same manner as before. The steps set out above relate to a regular RANSAC algorithm. However, in some cases, a connected RANSAC algorithm may be used. This is beneficial when the object (or objects) to be cleaned include two or more non-contiguous surfaces which nevertheless happen to conform to different regions of the same parametric model. The most common example of this would be when the object to be cleaned includes a plurality of non- contiguous coplanar regions. In this case, the RANSAC algorithm might determine that all of those non-contiguous coplanar regions to be part of the same continuous surface segment Si, whereas in reality, there are actually a plurality of separate continuous surface segments Si. Consider an example in which the object to be cleaned includes a table and four chairs, where each of the four chairs has a generally planar seat. Operating normally, the RANSAC algorithm would most likely conclude that all four of the seat surfaces are part of the same surface (since they likely all fall roughly within the same plane, to within a given tolerance). It may be desirable to avoid this, which can be achieved using a connected RANSAC algorithm. Essentially, a connected RANSAC algorithm identifies only regions within the parametric model in which all of the points are connected. For example, a point will only be considered to form part of the identified region if it is within a predetermined distance of at least one other point in the region. In implementations employing the connected RANSAC algorithm, the computer-implemented method may include identifying a first subset of points which are located within a predetermined distance (i.e. tolerance) of the surface defined by the parametric model; and identifying one or more connected regions of points within the first subset of points. Identifying one or more connected regions of points within the first subset of points may comprise using a region growing algorithm (independently of a region growing algorithm which may be used to identify the continuous surface segments Si, as described earlier in this application). Specifically, the computer-implemented method may further comprise: selecting a first seed point Pi from the first subset of points; identifying all neighbouring points which are within a predetermined threshold distance of the first seed point; repeating this process with the identified neighbouring points becoming new first seed points, until there are no neighbouring points within the predetermined threshold distance of the most recent set of neighbouring points. At that point, the first seed point and all identified neighbouring points are considered to represent a first continuous surface segment S1, and added to a cluster of points. This process may then be repeated by selecting, from the remaining points in the first subset of points, a second seed point P2, and repeating the above process to identify a second continuous surface segment S2. A third seed point may then be selected, and so on, until all of the points from the first subset of points have been assigned to a cluster representing a respective continuous surface segment Si. After one or more continuous surface segments Si have been identified from the first subset of points, the computer- implemented method may further comprise: fitting a new parametric model from the remaining points, and determining a second subset of points which are located within a predetermined distance (i.e. tolerance) of the surface defined by the parametric model; and identifying one or more connected regions of points within the second subset of points. Identifying the one or more connected regions within the second subset of points may proceed as outlined for the first subset of points. Subsequently, or generally, the computer- implemented invention may comprise: identifying, from the remaining points, fitting a new parametric model, determining an ith subset of points which are located within a predetermined distance of a surface defined by the new parametric model; and identifying one or more connected regions of points within the ith subset of points. This process may be repeated until all points in the 3D electronic representation of the object to be cleaned have been assigned to a cluster of points representing a respective continuous surface segment Si. In some cases, if the outer surface of the object to be cleaned includes some regions which may be easily represented by a surface defined by a parametric model, and other surfaces for which this is not the case, the computer-implemented method may comprise the application of both a RANSAC algorithm and a region growing algorithm. In some cases, the application of a RANSAC algorithm may comprise the application of both a regular RANSAC algorithm and a connected RANSAC algorithm. The output of the computer-implemented method is preferably one of more clusters of points, each cluster comprising a respective plurality of points from the 3D electronic representation of the outer surface S of the object to be cleaned. The points in a given cluster correspond to one of the identified continuous surface segments Si. A second aspect of the invention provides a method of identifying one or more continuous surface segments of an object to be cleaned by a 3D cleaning tool, the method comprising: generating an electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; transmitting the generated electronic representation to a computer or other data processing unit; and performing the computer-implemented method of the first aspect of the invention. In embodiments of the second aspect of the invention, generating the electronic representation may comprise obtaining a CAD representation of the object to be cleaned; or obtaining a 3D representation from a 3D camera or a 2D camera. It goes without saying that the optional features set out previously in respect of the first aspect of the invention apply equally well to the second aspect of the invention, unless they are clearly incompatible. A third aspect of the present invention provides an input generation system for a control system for a 3D cleaning tool, the system comprising: an input generation module configured to generate a representation of an outer surface S of an object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; a transmitter; and a computer or other data processing unit, wherein: the transmitter is configured to transmit the generated electronic representation to a computer or other data processing unit; and the computer or other data processing unit is configured to perform the computer-implemented method of the first aspect of the invention. Again, it goes without saying that the optional features set out previously in respect of the first aspect of the invention apply equally well to the third aspect of the invention, unless they are clearly incompatible. The computer-implemented method of the first aspect of the invention may further comprise generating a 3D cleaning tool path based on the one or more identified continuous smooth surface segments. Generating a 3D cleaning tool path based on the one or more identified continuous smooth segments may be implemented in the form of a computer-implemented method of generating a 3D tool path defining how a 3D cleaning tool moves across one or more continuous surface segments Si during a cleaning operation, the computer-implemented method comprising: for a first continuous surface segment S1, receiving continuous surface segment data comprising an indication of a first plurality of points in 3D space which are part of the first continuous surface segment S1; generating a representation of the first continuous surface segment S1; and determining a first 3D cleaning tool path which covers the representation of the first continuous surface segment S1, the first 3D cleaning tool path comprising a first plurality of path points to be traversed, in operation, by the 3D cleaning tool. Herein, the term “continuous” surface may mean that all points of the surface are located within a predetermined threshold distance of at least one other point in the surface. All points in a continuous surface may also meet a predetermined smoothness criterion which ensures that there are no sharp edges between two points within a surface. This is explained in more detail elsewhere in this application. All points on a continuous surface may also meet a predetermined curvature criterion which stipulates that the local curvature at a given point must be below a predetermined curvature threshold. Again, this is explained in more detail elsewhere in this application. The step of determining a first 3D cleaning tool path may comprise or refer to generating a first 3D generating tool path. In some cases, the computer-implemented method may further comprise a step of generating instructions to send to a 3D cleaning tool. Such instructions, when executed by the 3D cleaning tool, preferably cause the 3D cleaning tool, specifically the end effector thereof, to follow the generated or determined first (or other) 3D cleaning tool path across the surface of the object to be cleaned. The 3D cleaning tool path “covers” the representation of the first continuous surface segment. Here, the term “covers” should be understood to mean that all or substantially all of the representation of the continuous surface segment Si, when taking into account the dimensions of the end-effector which is intended to perform the cleaning task. Overlap between strokes of the end- effector may also be borne in mind, as described in more detail later. The 3D cleaning tool path generated may comprise one or more (most likely a plurality) of strokes, wherein a stroke refers to one movement of the end-effector. In some cases, the path may comprise a plurality of strokes which are straight lines (or substantially straight lines), which may be parallel to each other. Between each of the strokes, the path may comprise a turn, in which the end-effector changes direction in order to place it in the correct position for the subsequent stroke. The computer-implemented method includes a step of generating a representation of the first continuous surface segment Si. In some cases, the representation of the first continuous surface segment Si may be an isoparametric representation. Herein, an “isoparametric representation” refers to a representation in which a parametric surface is generated, wherein each point on the parametric surface corresponds to a point on the “real” continuous surface segment Si. In preferred cases, the isoparametric representation is an 2D isoparametric surface representation. This means that every point on the continuous surface segment Si can be defined based on two variables u and v. So, where a point may be represented as e.g. P(x,y,z) in real, or Euclidean space, in the parametric representation, it may be represented in the form S(u,v). Ways in which the isoparametric representation may be generated are explained in more detail later in the application. Herein, “corresponds” should be understood to mean that every point in real, Euclidean space, is mapped onto a point in the parametric space by means of an appropriately selected transformation. By employing a 2D parametric surface representation of the continuous surface segment Si, the task of generating a tool path is simplified. After generating the isoparametric representation, the computer-implemented method preferably further comprises: determining a path which covers the isoparametric representation of the first continuous surface segment Si in the parametric space; and transforming the path in the isoparametric representation of the first continuous surface segment Si into a path in the original (i.e. “real”) first continuous surface segment Si. In the parametric space of the isoparametric representation, the 3D tool cleaning path may include, or be in one of the following forms: a plurality of parallel lines traversing the isoparametric representation of the first continuous surface segment Si from one side to the opposite side, either in the same direction or in a serpentine fashion; a plurality of concentric circles; a plurality of concentric shapes, the perimeter of each of which is parallel to the perimeter of the isoparametric representation of the first continuous surface segment Si; and a spiral. Herein, “a serpentine fashion” refers to a path which passes from one side to the other of the isoparametric representation, then changes direction and returns to the other side in the opposite direction. In this way, the cleaning may take place throughout the whole path, without the surface-contacting end- effector having to separate from the surface to return to the other side. In the present context, parallel may also cover substantially parallel. In some cases, the path may comprise a plurality of the above geometries, i.e. a combination of two or more of the geometries outlined above, for example to perform more than one cleaning task, or to account for obstacles in the surface. The selection of the path geometry is liable to vary, for example based on one or more of the following: the nature of the cleaning task to be performed; the nature of the surface to be cleaned; and the geometry/kinematics of the 3D cleaning tool, or the surface-contacting end-effector thereof. For example, if the object to be cleaned is a table or a sofa, strokes which are straight, parallel, and/or continuous may be used, perhaps going from one extreme of the surface to the other (the isoplanar method described below is useful there). In other cases, a wiping task might require a circular concentric strokes or corner concentric motions (the isoparametric approach is useful in these cases). Different kinds of cleaning tools might also be chosen for each task. There are various other parameters which are associated with the tool path which may also be controlled. A non-exhaustive list of these is provided below: - In cases in which a concentric stroke pattern is used, the number of concentric strokes. - A height offset relative to the surface. - The number of points on a polynomial spline which is used to fit the path. - A minimum path length, wherein paths which are shorter than the minimum path length are discarded. - An extrapolation limit, which defines quantitatively whether a stroke finishes earlier or later than its calculated endpoint. In other cases, another representation may be used, namely an isoplanar representation, in which the first continuous surface segment Si is represented by a mesh comprising a plurality of contiguous geometric shapes; and determining the path which covers the first continuous surface segment Si comprises determining a path which covers the isoplanar representation of the first continuous surface segmentation Si. Preferably, the mesh comprises a plurality of triangles. In contrast to the isoparametric approach, when taking the isoplanar approach, everything takes place in real, Euclidean space – there is no need to transform any coordinates into any parametric shape. This means, for example, that lines which are parallel in the isoplanar representation of the continuous surface segment Si will be parallel in the actual continuous surface segment Si. More generally, any path geometry which is established based on the isoplanar representation will be reflected accurately in the actual continuous surface segment Si since no transformation of coordinates is required. Determining the path which covers the isoplanar representation of the first continuous surface segment may comprise: intersecting the isoplanar representation of the first continuous surface segment S1 with a plurality of cutting planes, the intersections of the cutting planes and the isoplanar representation of the first continuous surface segment S1 forming the path to be traversed by the 3D cleaning tool. Preferably, the path is serpentine, and preferably the cutting planes are parallel to each other. Of course, other geometries such as those described earlier for the isoparametric representation may also be used for the isoplanar representation, with the difference that, in the isoplanar case, there is no need to transform the paths back into the real, Euclidean space. Regardless of whether the isoparametric, isoplanar, or another representation entirely is used, when determining a path across the representation it is necessary to ensure that all portions of the representation of the continuous surface segment Si and therefore the continuous surface segment Si itself will be covered, or traversed by an active area of the surface-contacting end-effector of the 3D cleaning tool. Herein, the “active area” is the part of the end-effector which actually performs the cleaning function when in contact with the surface, e.g. the aperture of a vacuuming tool, or the wet area of a wiping tool. This may be achieved by a consideration of the overlap between adjacent strokes of the end-effector tool when the 3D cleaning tool path is actually executed by the 3D cleaning tool. In the present context, a “stroke” refers to a path of the end-effector tool, in use. Each stroke has a given width (i.e. an extent in a direction which is parallel or substantially parallel to the direction of movement of the end-effector), which is predetermined based on the geometry of the end-effector (specifically the active area thereof). The computer-implemented method may further comprise receiving: a predetermined overlap threshold defining a minimum amount of overlap between adjacent parallel strokes of the 3D cleaning tool path to be generated; or a predetermined stroke spacing defining a maximum spacing between adjacent parallel strokes of the 3D cleaning tool path to be generated; and the 3D cleaning tool path is determined such that either: the overlap between adjacent parallel strokes of the 3D cleaning tool path is greater than the predetermined overlap threshold; or a maximum path between adjacent parallel strokes of the 3D cleaning tool is less than the predetermined stroke spacing threshold. More generally, the path is preferably determined such that every point in the first continuous surface segment Si (or, equivalently, the representation thereof) is covered by the active area of the end-effector of the 3D cleaning tool when the path is executed. There may be some exceptions, in scenarios in which it is physically impossible for the end effector to reach various regions of the continuous surface segment Si. This is discussed below in more detail. Preferably, the 3D cleaning tool is a robotic tool which has a predetermined kinematic profile. Herein, the term “kinematic profile” is a profile which contains data indicative of the allowable range of movement of the various components of the 3D cleaning tool. It is possible to determine, based on the kinematic profile of the 3D cleaning tool, whether the end- effector is able to adopt a particular position, in a particular orientation. Such a kinematic profile can be used as a filter on data representing points in space, identifying which points can be reached by the end-effector, and which points can’t. In this application, we refer to this as a “kinematic filter”. This filtering step may be performed either at the outset, before the step of generating the 3D cleaning tool path, or after the path has been generated. Specifically: · The computer-implemented method may further comprise initially filtering the continuous surface segment data, wherein filtering comprises: determining, based on data indicating the geometry and/or kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the plurality of points; and removing the points which cannot be accessed from the continuous surface segment data, to generate filtered continuous surface segment data; and generating the 3D representation is based on the filtered continuous surface segment data. · Alternatively, or additionally, the computer-implemented method may further comprise filtering the first plurality of path points, wherein filtering comprises: determining, based on data indicating the geometry and kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the first plurality of path points; and removing the points which cannot be accessed from the continuous surface segment data, to generate a filtered first plurality of path points. In some cases, this may be preferable to the previous approach since fewer points may be considered. In the above, the data indicating the geometry and/or kinematics of the 3D cleaning tool may be in unified robot description format (URDF). The above description relates only to the process of determining a first 3D cleaning tool path. However, in reality, a given object to be cleaned may comprise a plurality of continuous surface segments Si. Accordingly, the computer- implemented method may further comprise: for a second continuous surface segment S2, receiving continuous surface segment data comprising an indication of a second plurality of points in 3D space which are part of the second continuous surface segment S2; generating a 3D representation of the second continuous surface segment S2; and determining a second 3D cleaning tool path which covers the 3D representation of the second continuous surface segment S2, the second 3D cleaning tool path comprising a second plurality of path points to be traversed, in operation, by the 3D cleaning tool. All of the optional features set out above in respect of the determination of the first 3D cleaning tool path apply equally well to the determination of the second 3D cleaning tool path. More generally, the computer-implemented method may comprise: for an ith surface segment Si, receiving continuous surface segment data comprising an indication of an ith plurality of points in 3D space which are part of the ith continuous surface segment Si; generating a 3D representation of the ith continuous surface segment Si; and determining an ith 3D cleaning tool path which covers the 3D representation of the ith continuous surface segment Si, the ith 3D cleaning tool path comprising a ith plurality of path points to be traversed, in operation, by the 3D cleaning tool. As before, the optional features set out previously in respect of the determination of the first 3D cleaning tool path apply equally well to the determination of the ith 3D cleaning tool path. In these cases, in which more than one 3D cleaning tool path is generated, the computer-implemented method may comprise a step of receiving continuous surface segment data which comprises a respective plurality of points in 3D space for each of the continuous surface segments Si. The computer-implemented method may further comprise: filtering the second plurality of path points, wherein filtering comprises: determining, based on data indicating the geometry and kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the second plurality of path points; and removing the points which cannot be accessed from the continuous surface segment data, to generate a filtered second plurality of path points. More generally, the computer-implemented invention may comprise filtering the ith plurality of path points, wherein filtering comprises: determining, based on data indicating the geometry and kinematics of the 3D cleaning tool, whether the 3D cleaning tool or a surface-contacting end-effector thereof, is able to access each point in the ith plurality of path points; and removing the points which cannot be accessed from the continuous surface segment data, to generate a filtered ith plurality of path points. All of the optional features set out above in respect of filtering the first plurality of path points apply equally well to filtering the second or ith pluralities of path points. In some cases, after the 3D cleaning tool path has been generated, the invention further comprises generating a set of instructions to be sent to the 3D cleaning tool, the set of instructions based on one or more of the following: the first plurality of path pints, the filtered first plurality of path points, the second plurality of path points, the filtered second plurality of path points, the ith plurality of path points, and the filtered ith plurality of path points. The instructions are preferably such that when they are received by the 3D cleaning tool, or, for example, a data processing unit or module thereof, they cause the 3D cleaning tool (specifically, the end-effector thereof) to execute the generated 3D cleaning tool path. In addition to generating the 3D cleaning tool path, and instructions for the 3D cleaning tool, the invention may further comprise a transmission step, whereby the instructions are sent to the 3D cleaning tool, or a component thereof. Accordingly, the invention may further include a method of controlling a 3D cleaning tool to clean, using a surface- contacting end-effector, one or more continuous surface segments Si of an object to be cleaned, the end-effector adapted to clean the one or more continuous surface segments Si, the method comprising: performing the computer-implemented method of the first aspect of the invention; generating a set of instructions to be sent to the 3D cleaning tool, the set of instructions based on one or more of the following: the first plurality of path points, the filtered first plurality of path points, the second plurality of path points, the filtered second plurality of path points, the ith plurality of path points, and the filtered ith plurality of path points; and transmitting the generated set of instructions to the 3D cleaning tool, the set of instructions operable to cause the 3D cleaning tool to clean the continuous surface segments Si using the end-effector. In addition to the input generation system, systems of some aspects of the present invention may further comprise a control system for a 3D cleaning tool, the control system configured to control a surface-contacting end-effector of the 3D cleaning tool to clean one or more continuous surface segments Si of an object to be cleaned, the control system comprising: a computer or other data processing unit configured to perform the computer-implemented method of the first aspect of the invention, and to generate a set of instructions to be sent to the 3D cleaning tool, the set of instructions based on one or more of the following: the first plurality of path points, the filtered first plurality of path points, the second plurality of path points, the filtered second plurality of path points, the ith plurality of path points, and the filtered ith plurality of path points; and a transmitter configured to send the generated set of instructions to the 3D cleaning tool, the set of instructions operable to cause the 3D cleaning tool to clean the one or more continuous surface segments using the end-effector. Generation of the set of instructions may further comprise fitting a spline to the path. A further aspect of the invention may provide a 3D cleaning tool comprising the input generation system and the control system. In some cases, in addition to identifying continuous surface segments of the outer surface of the object to be cleaned, the computer-implemented method of the first aspect of the invention may further comprise identifying one or more crevices of the object to be cleaned by the 3D cleaning tool. Specifically, this may be achieved by further including a computer-implemented method of identifying one or more crevices of an object to be cleaned by a 3D cleaning tool: receiving a 3D electronic representation of an outer surface of the object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object; and identifying the subset of points within the plurality of points having concave local curvature which exceeds a predetermined local curvature threshold. The 3D electronic representation may be in the form of a point cloud. Herein, the term “local curvature” should be understood to mean the curvature in the neighbourhood of the point. The “neighbourhood” may be the region which is delineated by an imaginary line connecting all of the closest neighbours (e.g. neighbours which are located within a given distance, or a predetermined number of nearest neighbours). The identified subset of points are those where the concave local curvature “exceeds” a predetermined local curvature threshold. In the context of the present application, the term “exceeds” may be interpreted to mean either “is greater than”, or “is greater than or equal to”. It may be necessary to differentiate between a crevice and an “edge”. In the context of the present application, the term “edge” is used to refer to a sharp convex region, such as a ridge between two smooth surfaces. Effectively, an edge is the convex equivalent of a (concave) crevice. Identifying the subset of points may comprise, for each point within the plurality of points: determining the respective local curvature Ci of each point within the plurality of points; comparing the respective local curvature with the predetermined local curvature threshold; and selecting those points for which the respective local curvature exceeds the predetermined local curvature threshold. By “curvature”, we refer to any parameter which may be used to parametrize the extent to which the outer surface S is curved a given point. In some cases, curvature may be represented as the inverse of the radius of curvature, which is the radius of a sphere which best approximates the shape of the surface S at the point. It should be stressed that if the term “curvature” is interpreted as “radius of curvature”, then in order to identify the crevices it is necessary to identify those points for which the radius of curvature is below a threshold value (since a more curved surface has a smaller radius of curvature). Preferred implementations of the computer-implemented method of the first aspect of the present invention do not use the radius of curvature, however, as we discuss in detail below. Determining the respective curvature of any given point may comprise estimating or calculating the normal to the surface at the point. The curvature may then be calculated based on at least the estimated or calculated normal to the point. In preferred cases, it may be possible to calculate a measure of the curvature by using the residuals from the eigenvalue local plane fitting problem, which gives a dimensionless curvature value. More detail about this calculation process is set out later in this application. Calculating curvature in this way is advantageous since it may be done at practically no additional computational cost when calculating the normals. The measure of curvature is unsigned when calculated in this manner, and it may therefore be necessary to perform additional steps in order to determine e.g. whether a given point is ridge or a crevice. Regions having a high curvature may be either crevices or edges. In order to identify crevices, it is necessary to identify only the concave points. Accordingly, the step of identifying may comprise: for each point within the plurality of points, determining whether the respective local curvature is concave. Determining whether the respective local curvature is concave may comprise for each of the k nearest neighbours of the point: determining the average distance between that nearest neighbour, and the tangent plane defined by the normal ni to the point; and wherein a positive determined average distance value indicates that local curvature is a concave local curvature. This is explained diagrammatically later in this application. Put alternatively, the computer-implemented method may comprise: determining whether the average difference between that nearest neighbour and the tangent plane defined by the normal ni to the point is positive or negative, and if the average difference is positive, determining that the local curvature is concave, and if the difference is negative, determining that the local curvature is convex. When it is determined that the local curvature is concave, the computer-implemented method may further comprise: adding the point to the plurality of points which are determined to be in a crevice. When it is determined that the local curvature is convex, the computer- implemented method may comprise discarding the point. In some cases, the predetermined local curvature threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or surface-contacting end- effector thereof; the nature of the cleaning task to be performed; and the nature of the outer surface to be cleaned. Example values for the curvature threshold may range from 0.0 to 0.5 m, preferably around 0.01 m. Herein, distance units are used, as they correspond to an average distance from the tangent plane to a point. Points having a curvature exceeding the curvature threshold are considered part of a crevice. It should be stressed here that this means that the signed value of the curvature should be higher than this value, for an area of high curvature to be considered a crevice. As discussed, in order to form part of a crevice, the local curvature must be concave, i.e. the signed value of the curvature should be no more than zero in order to meet the requirements of a crevice. By varying the threshold depending on the factors set out above, it is possible to ensure that the optimal points are selected for e.g. a given end-effector or a given cleaning task. Aspects of the present invention may further comprise a method of identifying one or more crevices of an object to be cleaned by a 3D cleaning tool, the method comprising: generating an electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; transmitting the generated electronic representation to a computer or other data processing device; and performing the computer- implemented method discussed above. Some aspects of the present invention provide an input generation system. However, this input generation system may be modified in order to provide an input relating to crevices too. Specifically, the input generation system for a control system for a 3D cleaning tool, may further comprise: an input generation module configured to generate a representation of an outer surface S of an object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; a transmitter; and a computer or other data processing unit, wherein: the transmitter is configured to transmit the generated electronic representation to a computer or other data processing unit; and the computer or other data processing unit is configured to perform the computer- implemented method of the first aspect of the invention. In those cases where the method further comprises the identification of crevices, the generation of a 3D cleaning tool path will also need to take account of the crevices. Accordingly, in such cases the computer-implemented method of the first aspect of the invention may further comprise determining which points of the outer surface correspond to crevices that may need to be cleaned. This may be achieved by incorporating a computer-implemented method of generating a 3D tool path defining how a 3D cleaning tool moves along one or more crevices during a cleaning operation, the computer- implemented invention comprising: receiving data comprising an indication of a plurality of crevice points, which are points in 3D space which are located within a crevice; determining a path which traverses the one or more crevices by converting the points located in a crevice into a respective plurality of nodes, the nodes defining the 3D cleaning tool path. The overall output of the method of such a computer- implemented method is a path comprising a plurality of waypoints. Herein, the term “waypoint” should be understood to refer to a point along the path which the end-effector (preferably an active area thereof) passes through, in use. A “node” here refers to a point in 3D space, and may interpreted similarly to, or identically to, “waypoint”. Determining a path may use graphical techniques. For example, in some cases, determining a path may comprise generating a graph based on the plurality of crevice points. Herein, a “graph” is a visualizable representation of the relationship between the locations of points in 3D space. In some cases, an actual visual representation may be generated, and e.g. displayed to a user. Alternatively, the visual representation may not actually be generated. The step of generating a graph may comprise generating a convenient coordinate system for representing the plurality of crevice points. Generating the graph may comprise downsampling the data comprising the indication of the plurality of crevice points at a predetermined downsampling resolution, to generate a plurality of nodes, each node having an associated surface normal and position. In the present context, the “surface normal” represents a vector which is perpendicular to a tangent plane to the surface at the node. Collectively, the normal and position may be referred to as “node attributes”. Herein, “downsampling” refers to a process of sampling a subset of the plurality of crevice nodes at a resolution which is less than the original resolution. This necessarily results in fewer nodes. More specifically, Pointcloud downsampling refers in general to the process of reducing the number of points in a pointcloud by selecting or creating a subset of the original set of points while keeping as much information as possible. The method which may be applied here is often referred to voxel grid downsampling, whereby a 3D mesh grid is created. For each cubic volume in the grid (a voxel), all the enclosed points are averaged, and the output is the resulting average point (also called centroid). The downsampling resolution represents the size of the edge of each of the voxels, or in other words, the size step at which the grid is created. As in a picture, the larger the voxel size (downsampling resolution) is, the coarser the result will be and the more information will be lost. One way in which the downsampling process may take place is to superimpose a 3D grid, or lattice onto the plurality of crevice points. Then, for each cell defined by the grid or lattice, which may contain one or more crevice points therewithin, generating a point which represents the average location of all of the crevice points within that cell. In such cases, the size of the cell may be referred to as the downsampling resolution. This may also be referred to as a “voxel size”. In preferred cases, the voxel size may be around 1cm. There is a trade-off when considering the downsampling resolution: if the resolution is too coarse, details on the path may be missed and displacement of the downsampling grid may heavily influence the results. However, if the resolution is too fine, the number of nodes and connections may create parallel paths along a single crevice edge, which means that it can take a long time to compute a complete traversal which covers all branches. Once the plurality of nodes has been generated by downsampling, generating the graph may further comprise generating a plurality of edges between neighbouring nodes. Herein, “neighbouring nodes” may be defined as notes which are located within a first predetermined distance threshold from each other. The first predetermined distance is preferably approximately 1.5 times as large as the downsampling resolution. This ratio has been shown to give rise to a balanced amount of edges between points. Each edge preferably has a normal difference, and a relative displacement associated therewith. Herein, the “relative displacement” represents the displacement (i.e. the distance and direction) between the two nodes in Euclidean space, and may be represented by the vector resulting from subtracting the position of one node from the position of the other node. In addition to the relative displacement, each edge may have a normal difference associated therewith. This preferably represents or comprises the difference in direction between the normals, which may be expressed as an angle (bearing in mind that where θ is the angle between the
Figure imgf000034_0001
vectors
Figure imgf000034_0002
and Collectively, the relative displacement
Figure imgf000034_0003
and the normal difference may be referred to as the “edge attributes” of each edge. After downsampling, the resulting nodes (or the original crevice points) may be located on the surfaces either side of the true crevice, rather than on the crevice itself. In this case, the normals of those points may not represent the true desired direction of the 3D cleaning tool at that point. In order to address this, the computer-implemented method of the first aspect of the invention may further comprise a step of graph reduction, in which connected nodes are iteratively averaged out, merging those which end up nearby each other. The output of the graph reduction process is a graph skeleton which uniquely represents each of the crevices in the outer surface of the object to be cleaned, and any branches thereof. The aim of graph reduction is to eliminate any small branches of crevices which may result from overly fine sampling, and which do not represent the true geometry of the object to be cleaned. There are many ways in which graph reduction may be effected, but in one case, graph reduction may comprise: (a) for each node, computing an average position and average normal based on the attributes of that node and all nodes connected to it, and generating a respective average node with those attributes, the generated plurality of average nodes replacing the initial plurality of nodes (here, the averaging step does not remove the node, it simply replaces its attributes with the average values of the connected nodes; this is in contrast to the merging step which follows, in which the number of nodes is reduced); (b) merging pairs of average nodes which are located within a second predetermined distance threshold from each other, to generate a modified plurality of average nodes (each node of the modified plurality of nodes having the average of the attributes of the two nodes which merged to form it; the threshold may be, for example about 1 cm, or preferably, equal to the voxel size or downsampling resolution); and (c) repeating, a predetermined number of times (for example, around 5 times), steps (a) and (b) using the modified plurality of average nodes as a starting point; (d) after the process has been repeated the predetermined number of times, generating a plurality of edges between neighbouring pairs of nodes of the modified plurality of average nodes which are located within a third predetermined distance threshold from each other, each newly-generated edge having associated therewith a normal difference, and relative displacement, thereby generating a reduced graph. In preferred cases, the second predetermined distance threshold may be approximately the same as the downsampling resolution, in order to give rise to an overall output resolution which is the same as the initial one. By performing this graph reduction process, from a graph which is initially obtained by using a very fine downsampling resolution, we obtain a graph skeleton which represents every crevice branch uniquely, and accurately. At this point, a reduced graph representing the locations of the crevices in 3D space has been generated. Having generated this graph, it is necessary to determine how the end-effector should traverse this reduced graph. Alternatively, in cases in which neither the graph generation nor graph reduction take place, it is necessary to determine how the end-effector should traverse the plurality of crevice points, or the plurality of nodes. In other cases, where graph generation takes place, but no graph reduction, it is necessary to determine how the end-effector should traverse the generated graph. Accordingly, determining a path may further comprise applying a complete traversal algorithm to an input, where the input is either: the plurality of crevice points; the plurality of nodes; the generated graph; or the reduced graph. The output of a complete traversal algorithm is a path which traverses all of the points or nodes of the input, preferably no more than once (though in some cases, this may not be possible). The complete traversal algorithm may comprise identifying and separating the input into one or more connected components. Herein, a “connected component” is a plurality of points, all of which are connected, in e.g. a chain or branching structure. In order to be “connected” to another point or node, there should be an edge between the two, which reflects the fact that the two are separated by no more than a threshold distance, e.g. the third predetermined threshold distance. The complete traversal algorithm may comprise, for the input, or for each connected component of the input: (a)identifying the nodes which are the furthest away from each other and identifying the shortest path between those nodes, and saving the identified path; and (b) removing the nodes on the identified shortest path; (c) for the remaining nodes after the nodes have been removed in step (b), repeating steps (a) and (b) until all nodes in the input, or connected component of the input have been assigned to a shortest path. Herein, where we refer to “removing” the nodes, this should be understood to mean that the nodes are being removed from consideration. They are not removed from the path or the graph. This is evident to the skilled person. The steps outlined above relate to a computer-implemented method in which graph generation and reduction are used to obtain a graph skeleton which accurately reflects the branching structure of the crevices of the objected to be cleaned. In an alternative case, the graph method may not be used, and instead, determining a path may comprise: identifying one or more branches within the plurality of crevice points, wherein identifying the one or more branches comprises: (a) selecting a crevice point of the plurality of crevice points, the selected crevice point forming a seed point; (b) identifying all of the crevice points of the plurality of crevice points which neighbour the seed point within a predetermined threshold distance, and selecting these points as new seed points; (c) repeating step (b) for the newly-selected seed points, until there are no crevice points which neighbour newly-generated seed points, at which point the branch is complete; and (d) repeating steps (a) to (c) until all crevice points of the plurality of crevice points have been assigned to a branch; (e) determining a path which traverses all of the identified branches. Determining the path may comprise the use of a complete traversal algorithm as before. The output of the complete traversal algorithms or other path determination processes is a list of paths, where each path corresponds to an ordered list of paths, such that all of the branches in the crevice area are visited. In some cases, there may be a threshold number of points which a path must comprise in order to be considered a path. Preferably, this threshold number is 5 (i.e. paths with fewer than 5 points are not stored as a path). Since the nodes in the graph correspond to points in a point cloud, i.e. the crevice points, the attributes of those points can be used to find waypoints that can be sent to the 3D cleaning tool to move the crevice tool along those waypoints. Accordingly, the computer-implemented method may further comprise: generating, based on the determined path, a set of instructions to be sent to the 3D cleaning tool. Generating the set of instructions may comprise fitting a spline along the points on the determined path. This will be discussed in more detail later in the application. In those cases where the input generation system of the present invention is configured also to detect crevices, the invention may further comprise a control system for a 3D cleaning tool, the control system configured to control a surface-contacting end-effector of the 3D cleaning tool to clean one or more crevices of an object to be cleaned, the control system comprising: a computer or other data processing unit configured to perform the computer-implemented method of the first aspect of the invention, and to generate a set of instructions based on the determined path; and a transmitter configured to send the generated set of instructions to the 3D cleaning tool, the set of instructions operable to cause the 3D cleaning tool to clean the one or more crevices using the end-effector according to the determined path. The invention includes the combination of the aspects and preferred features described except where such a combination is clearly impermissible or expressly avoided. BRIEF DESCRIPTION OF THE DRAWINGS Embodiments of the present invention will now be described with reference to the accompanying drawings, in which: - Fig. 1 is a high-level system diagram showing various components associated with the present invention. - Fig. 2 is a schematic of an input generation system and a control system for a 3D cleaning tool. - Figs. 3A and 3B are flowcharts illustrating processes which may be performed by the system of Fig. 1. - Fig. 4 is an illustration of a representation of a chair showing continuous surface segments and crevices therebetween. The representation is shown on a computer screen. - Fig. 5 is a flowchart showing high-level steps involved in the identification of continuous surface segments. - Fig. 6 is a flowchart illustrating the steps of an exemplary RANSAC algorithm. - Fig. 7A shows an example of 3D electronic representation of the outer surface of an object to be cleaned, in this case a table and chairs. - Figs. 7B and 7C illustrate the outcomes of, respectively, a normal RANSAC algorithm and a connected RANSAC algorithm on the representation of Fig. 7A - Fig. 8 is a flowchart illustrating the steps of an exemplary region growing algorithm. - Fig. 9 is a flowchart illustrating the high-level steps involved in generating a 3D cleaning tool path based on the detected continuous surface segments. - Fig. 10 is a flowchart illustrating a series of steps which may be used to generate a 3D cleaning tool path based on an isoparametric representation of continuous surface segments of an object to be cleaned. - Fig. 11 is a schematic illustrating an SOM approach to generating a grid representation. - Fig. 12 is a schematic illustrating various vectors which may be considered when generating isoparametric representations of continuous surface segments of an object to be cleaned. - Figs. 13A to 13C show examples of 3D cleaning tool paths which may be generated based on an isoparametric representation of continuous surface segments of an object to be cleaned. - Fig. 14A is a flowchart illustrating a series of steps which may be used to generate a 3D cleaning tool path based on an isoplanar representation of continuous surface segments of an object to be cleaned. - Fig. 14B is a schematic illustrating the use of intersecting cutting planes of an isoplanar representation of a continuous surface segment of an object to be cleaned, in order to generate a 3D cleaning tool path. - Fig. 15A is a flowchart illustrating the high-level steps involved in detecting crevices in a 3D electronic representation of an outer surface of an object to be cleaned. - Fig. 15B shows an example of an output of the flowchart of Fig. 15A. - Fig. 16 is a flowchart illustrating the high-level steps involved in generating a 3D cleaning tool path based on the detected crevices. - Fig. 17 is a flowchart illustrating an exemplary graph generation process. - Fig. 18 is a flowchart illustrating am exemplary graph reduction process. - Figs. 19A to 19F show the outputs of various iterations of the graph reduction process of Fig. 18. - Fig. 20 is a flowchart illustrating an exemplary complete traversal algorithm. - Figs. 21A to 21C show the outputs of various steps in the flowchart of Fig. 20. - Fig. 22 is a flowchart illustrating an exemplary instruction generation process. DETAILED DESCRIPTION OF THE DRAWINGS Aspects and embodiments of the present invention will now be discussed with reference to the accompanying figures. Further aspects and embodiments will be apparent to those skilled in the art. All documents mentioned in this text are incorporated herein by reference. At a high level, the present disclosure relates to control of a 3D cleaning tool 102. Specifically, the disclosure relates to methods (including computer-implemented methods) for identifying various specified regions of an object to be cleaned (namely continuous surface segments and crevices), and processing data related to the identified regions in order to generate paths to be followed by a surface-contacting end- effector 110 of the 3D cleaning tool 102, in order to perform some cleaning task, such as vacuuming, brushing, wiping, or the like. Fig. 1 is a system diagram showing an overall system 100 which may be used to perform various aspects of the present invention, as well as other related methods which may or may not fall within the scope of the claims. System 100 may include a 3D cleaning tool 102, a control system 104, an input generation system 105, a 3D camera 106, and an external computing device 108. The input generation system 105 may be configured to receive inputs from the 3D camera 106 and the external computing device 108, either via a direct (e.g. wired) connection, or, for example, via a network (not shown), which may be a wired or wireless network such as a Wi-Fi network or a cellular network. The 3D cleaning tool comprises an end-effector 110, and may be configured to receive inputs from the control system 104, again, either directly or via a network (also not shown). The network may be the same network as or a different network from the network via which the input generation system 105 is configured to receive input from the 3D camera 106 and/or the external computing device 108. In some cases, the input generation system 105 and the control system 104 may be part of the same component. Alternatively, the input generation system 105 may be part of the control system 104. Alternatively, the two may be connected either directly, or via a network. The connection may take the same form as the connections previously described, and may be the same or different connections and/or networks. All arrangements are covered by the invention, but throughout this application, the input generation system 105 and the control system 104 are shown as separate modules. This should not be regarded as limiting. For the purposes of this application, the input generation system 105 may contain the modules which are configured to receive the 3D representation of the object to be cleaned from external sources such as the 3D camera 106 and the external computing device 108, and to identify the continuous surface segments and/or the crevices in the outer surface of an object to be cleaned. And, the control system 104 may contain the modules which are configured to generate an appropriate tool path, and/or instructions for the 3D cleaning tool 102, and to transmit those instructions to the 3D cleaning tool 102. Fig. 2 is a schematic diagram of an input generation system 105 and control system 104. To reiterate, even though these are shown as separate components, they may both be included in a single component, or one may contain the other. The structure of these systems is described first, before discussing the function of these systems in more detail with reference to the other drawings. Input generation system 105 may comprise an interface module 1050 which may be configured to receive inputs from the 3D camera 106 and/or the external computing device 108. Input generation system 105 may further comprise a processor 1052 and a memory 1054, which may include a buffer 1056. Processor 1052 may comprise a surface segmentation module 1058 and a crevice detection module 1060. The surface segmentation module 1058 may comprise a RANSAC module 1062 and a region growing algorithm (RGA) module 1064. The input generation module may further include a control system interface module 1066. Control system 104 may comprise an input generation system interface module 1068 via which it may be able to communicate with the input generation system 105. Control system 104, like the input generation system 105, may also comprise a processor 1070 and a memory 1072, which may include a buffer 1074. Processor 1070 may comprise a tool path generation (continuous surface segment, CSS) module 1076 and a tool path generation (crevice) module 1078. The tool path generation module (CSS) 1076 may comprise a representation generation module 1080, which may comprise an isoplanar representation generation module 1082 and an isoparametric representation generation module 1084, and a filtering module 1086. The tool path generation module (crevice) 1078 may include a graph generation module 1088, a graph reduction module 1090, and a traversal algorithm module 1092. The control system 104 may further comprise an instruction generation module 1094, and a 3D cleaning tool interface module 1096, via which it may communicate with the 3D cleaning tool 102. It should be noted that in some cases (not shown), the input generation system 105 and/or the control system 104 may form part of the 3D cleaning tool. Alternatively, they may be separate components, such as computing devices e.g. a laptop, desktop PC, tablet, or smartphone. Figs. 3A and 3B show flowcharts which illustrate high-level methods which may be performed by various aspects of this disclosure. In Fig. 3A, step S1, performed by the surface segmentation module 1058 of the input generation module 105 is the identification of one or more continuous surface segments, and step S2, performed by the tool path generation module (CSS) 1076 is the generation of a 3D cleaning tool path based on the identified continuous surface segments. Analogously, in Fig. 3B, step S3, performed by the crevice detection module 1060 of the input generation system 105 is the identification of one or more crevices, and step S4, performed by the tool path generation module (crevice) 1078 is the generation of a 3D cleaning tool path based on the identified crevices. Of course, an object to be cleaned, e.g. an item of furniture may well include both continuous surface segments and crevices. It may be desirable to clean both of these components in the same cleaning operation. Accordingly, various aspects of the disclosure may include the methods of both Fig. 3A and Fig. 3B. In some cases, the methods of each of Figs. 3A and Fig. 3B may take place simultaneously, and in parallel with each other (e.g. independently of each other). Then, in some cases, a further step may be included in which the resulting 3D cleaning tool paths are combined. Alternatively, steps S1 and S3 may be performed first (in either order), and then the outputs of those steps may be sent simultaneously to the control system 104, and then steps S2 and S4 may take place afterwards (again, in either order). In alternative cases, steps S1 and S2 may take place, then steps S3 and S4 may follow. Or, steps S3 and S4 may take place first, followed by steps S1 and S2. Essentially, steps S1 to S4 may be performed in any order as long as step S1 takes place before S2, and step S3 takes place step S4. Now, the processes of each of steps S1 to S4 will be described in detail with reference to the other drawings. Firstly, the surface segmentation step S1 is described. Fig. 4 illustrates exactly what is meant by the surface segmentation step. A chair (or, more accurately, a representation of a chair) is shown, having four surfaces, labelled (1), (2), (3) and (4). On receiving a 3D electronic representation of the chair (preferably in the form of a 3D point cloud which sets out the location of various points on the outer surface of such a chair), the purpose of the surface segmentation process shown in Fig. 5 in the form of a flowchart is to identify the four separate continuous smooth surface segments (1), (2), (3), and (4), and preferably to indicate which of the plurality of points in the 3D electronic representation belong to each respective surface segment. In step S10 of Fig. 5, the 3D electronic representation of the outer surface of the object to be cleaned may be received by the input generation system 105 (e.g. from the 3D camera 106 or the external computing device 108, via the interface module 1050). Implicitly, it will be appreciated that prior to the method of Fig. 5, the 3D camera 106 may obtain a 3D representation of the object to be cleaned. Alternatively, the 3D representation may exist in the form of a CAD representation on the external computing device 108, which simply transmits that representation to the input generation system 105. In step S12, a surface segmentation algorithm may be applied to the electronic representation of the outer surface S of the object to be cleaned, preferably by the surface segmentation module 1058 of processor 1052 of the input generation system 105. The output of the surface segmentation algorithm is preferably an indication of one or more continuous surface segments Si. Two exemplary surface segmentation algorithms are now described. Fig. 6 is a flowchart illustrating a series of steps of a random sample consensus method (RANSAC) algorithm which may be performed by RANSAC module 1062 of the processor 1052, of the input generation system 105. In a first step S120, N points are selected from the plurality of points, where N is at least the number of points required to estimate the parameters of a parametric model representing the continuous surface segment Si. We consider the example where the parametric model is a plane which may be represented by the equation: ax + by + cz + d = 0 In the case of a plane, it is possible to define a plane in terms of three points, so N may be chosen to be 3. In step S121, the model is fit to the N selected points, in order to find the values of a, b, c, and d. Then, in step S122, the number of points (inliers) from the set of all of the input points (i.e. the points in the 3D representation) fit that model within a predetermined tolerance is determined. In other words, the number of points which are within a predetermined distance of the plane defined by the model is determined. This process is then repeated X times, where X is predetermined value. This is illustrated by decision step S123. After steps S120 to S122 have been repeated X times, in step S124, the model parameters which gave rise to the largest result in step S122 are selected. In step S125, the model parameters are re-determined based on all of the inliers of the model identified in step S124, to generate a final continuous surface segment Si. In step S126, it is determined whether there are any other points which are not accounted for. If not, the process ends, otherwise, the inliers are removed from consideration in step S127, and the process returns to step S120. This continues until all points in the 3D electronic representation are accounted for, at which point the algorithm is terminated. In some cases, a connected RANSAC algorithm may be used. This differs from the “normal” RANSAC algorithm in that it includes a further connected components step is performed on all the planar inliers found by RANSAC, and only the biggest connected cluster is retained. This process is explained in more detail earlier in this application. Figs. 7A to 7C illustrate the effects of the RANSAC and connected RANSAC algorithms. Fig. 7A shows an illustration of a 3D point cloud of a table and chairs. Fig. 7B shows a schematic diagram of a surface segmentation which may be obtained using a RANSAC algorithm, where different types of shading illustrate different identified planar surfaces. From this, it will be noted that the planar surfaces identified from the tabletop and base of the tabletop extend to the middle of the backs of the chairs. Furthermore, the RANSAC algorithm identifies all of the seats of the chairs as belonging to the same surface, as well as a layer in the table legs. Furthermore, the RANSAC algorithm identifies a portion of the bottom right-hand table leg as belonging to the same surfaces as the backs of the chairs on the right-hand side. Fig. 7C shows the same setup when a connected RANSAC algorithm is used, and shows that each of the seats of the chairs are identified as separate surfaces, and the plane of the table does not intersect the chair backs. Of course, in other cases, the RANSAC algorithm works perfectly well, this example is only illustrative of a situation in which a connected RANSAC algorithm is preferable, i.e. when there are a plurality of non-contiguous but coplanar smooth surfaces within on object or objects to be cleaned. Fig. 8 is a flowchart illustrating a series of steps of a region growing algorithm (RGA) which may be performed by RGA module 1064 of the processor 1052, of the input generation system 105. A region growing algorithm is effective when segmenting surfaces of items where a planar (or other easily parameterizable) model cannot be used. So, while a RANSAC algorithm would be useful for e.g. tables, counters, shelves, stairs, it is preferable to use an RGA for furniture types such as sofas, chairs, beds, and bathtubs. These lists are, of course, not exhaustive. At a high-level, an RGA works iteratively by adding contiguous points to a surface as long as they are (a) not edges, and (b) have a low relative curvature. The output of the algorithm is preferably a set of clusters, where each cluster contains a set of points that are considered to belong to the same smooth surface. As with the RANSAC algorithm shown in Fig. 6, before the RGA is performed, in step S10 (see Fig. 5) the 3D camera 106 may obtain a 3D representation of the object to be cleaned. Alternatively, the 3D representation may exist in the form of a CAD representation on the external computing device 108, which simply transmits that representation to the input generation system 105. More specifically, in a first step S130 the RGA may comprise computing the curvature value Ci of all points in the point cloud. The fastest way to estimate the curvature is to use the residuals from the eigenvalue local plane fitting problem, which is already estimated when estimating the normals of the surface. Herein, curvature may be defined in 3D as the degree to which a surface deviates from a plane. Mathematically, it is measured as the reciprocal of the radius of curvature at the point where it is measured. In the present context, calculating the exact reciprocal of the radius of curvature is unnecessary, and computationally expensive, since the considered surfaces have varying curvatures at each point. So, instead, the curvature at any point Pi may be computed via proxies that measure the deviation of the points surrounding Pi from the plane given by the normal at that point ni. Normals may be estimated by estimating the normal of a plane tangent to the surface, which takes the form of a least-square plane fitting estimation problem. On a 3D surface represented by points, the 3D covariance matrix at each point will naturally have the largest eigenvectors in the directions tangent to the surface at that point (since it is where the points are). therefore, the normal at that point will be the third eigenvector, which is perpendicular (normal) to the previous two. The more curved the surface is, the more distance there will actually be between the neighbouring points considered in the equation and the plane defined by the two first eigenvectors. This is why this normal computation method automatically provides a good proxy for the curvature, but not for its sign. The normals and curvatures are therefore computed via the eigenvectors and eigenvalues of a covariance matrix C created from the k nearest neighbours of the query point Pi as follows:
Figure imgf000048_0001
Here, represents the 3C centroid of the nearest neighbours
Figure imgf000048_0002
around the point in question, is the jth eigenvalue and is
Figure imgf000048_0003
th
Figure imgf000048_0004
the j eigenvector. Based on these results, an approximation to the curvature at each point can easily be obtained as the relationship between the eigenvalues of the covariance matrix, as presented above:
Figure imgf000048_0005
This method of calculating the curvature yields a smooth approximation of the curvature with practically no extra computation after normal estimation. This is sufficient for smooth surface estimation, as the smoothness does not depend on whether the surface is concave or convex. We will see later that additional steps are required in order to determine whether the surface is convex or concave at the point, in the context of crevice detection. After the curvature has been estimated for each of the points in the 3D point cloud, in step S132, the points are sorted by their curvature value, and the point with the minimum curvature value is selected as the seed point for the first region. Then, in step S134, a smoothness check is performed, in which the angle between the normal of the seed point and that of each of its neighbouring points is determined. The neighbouring points for which the angle is below a predetermined smoothness threshold θth are then added to the region. This step ensures that the output surface is smooth, i.e. without sudden, large, abrupt variations in the normals of its points. Then, in step S136, the neighbouring points whose curvature Ci is below a predetermined curvature threshold are selected. This step removes edges (i.e. regions with high local curvature) from the set of seeds, preventing the surface from growing past them. In step S138, the current (i.e. initial) seed is removed from the set of seeds. It is then determined whether the seed set is empty, in decision step S140. If the seed set is empty, this means that the algorithm has found the limits of the current region, so the method returns to step S132. If not, then the method returns to step S134 for each seed point, using the new seeds as a starting point. The process continues until all points in the point cloud are accounted for. It will be noted that in both the RANSAC algorithm and region growing algorithm, the points which are included in a given region, i.e. continuous surface segment Si are selected based on various thresholds and tolerances. These thresholds and tolerances may be varied to suit a particular item of furniture, cleaning task, 3D cleaning tool 102 (or end- effector 110) geometry. The thresholds may be variable, and can be changed by a user, e.g. via a user interface of the input generation system 105 (not shown), or of an external computing device such as external computing device 108 as shown in Fig. 1. Returning to Fig. 5, at this point, now that step S12 is completed, the process moves to step S14, in which data representing the identified continuous surface segments Si may be transmitted to the control system 104, preferably the tool path generation module (CSS) 1076 thereof. In implementations in which the input generation module 105 and the control system 104 are the same component, or part of the same component, this step S14 may not take place. At this point, the method moves to step S2 of Fig. 3A, in which a 3D cleaning tool path is generated by the tool path generation module (CSS) 1076 of the processor 1070 of the control system 104. A high-level flowchart illustrating the tool path generation process is set out in Fig. 9. For the avoidance of doubt, it should be understood that the tool path generation process starts after the continuous surface segment data has been received by the control system 104, specifically the tool path generation (CSS) module 1076 thereof. In a first step S20, a representation of a continuous surface segment Si is generated. And, after the representation has been generated in step S22, a 3D cleaning tool path is generated, which covers the representation of the continuous surface segment Si. The generated path may comprise a plurality of path points which are to be traversed by the 3D cleaning tool path, in operation. We now explain these steps in more detail. The explanation refers to one continuous surface segment Si, but it will be appreciated that the same process is performed for each of the identified continuous surface segments. In step S20, a representation of the surface is generated by the representation generation module 1080. The data which is generated by e.g. the surface segmentation module 1058 of the processor 1052 of the input generation system 105 essentially includes several clusters of points (or data identifying those points), each cluster representing a different, respective continuous surface segment Si. The purpose of generating a representation of the continuous surface segments Si is to generate a model of the surface in 3D space, such that a tool path may be plotted geometrically on that model, rather than the continuous surface segment Si being represented only by a somewhat abstract list of points. This meaning will become clearer when we consider examples of representations which may be generated by the representation generation module 1080. It should be stressed that the representations described below do not represent an exhaustive list, and other processes for generating a model representing the continuous surface segments Si in 3D space such that a 3D cleaning tool path can be generated and/or plotted thereon are also envisaged. In a first example, the isoparametric representation generation module 1084 generates an isoparametric representation of the continuous surface segment Si. In an isoparametric representation, the continuous surface segment Si is described in a parametric space. In an exemplary case, the isoparametric representation may be in the form of a polynomial parametric surface, such as a non-uniform rational B-spline, or NURBS, which is a well-defined representation of 3D geometry which can accurately describe any shape. Fig. 10 shows an example process by which a representation in the form of a NURBS may be generated by the isoparametric representation generation module 1084. In a first step S202, the topology of the continuous surface segment Si may be learned, e.g. using a self-organizing map (SOM) approach. A self-organized map is an unsupervised machine learning technique which may be used to produce a low-dimensional (typically two-dimensional) representation of a higher- dimensional data set while preserving the topological structure of the data. More specifically, a SOM is a network of weights:
Figure imgf000051_0001
The weights are topologically laid out as nodes in a lattice grid as shown in Fig. 11. Each weight represents the position of the node in the input (i.e. real, Euclidean) space. The purpose of the SOM is to find the underlying topologies of an input space (in this case, the plurality of points making up the continuous surface segment Si) with a reduced set of m nodes. Intuitively (i.e. as a result of the unsupervised learning) the lattice will assume the underlying shape of the continuous surface segment Si while preserving the property that neighbouring points in the continuous surface segment Si will remain close in the lattice. The result is thus a discrete grid which assumes the shape of the underlying surface segment in an orderly fashion (i.e. it remains a grid). The process via which the SOM is generated may be summarized as follows, with more detail in Kohonen (1990)2: 1. Initialization: random values are selected for each of the initial weight vectors
Figure imgf000052_0005
2. Sampling: draw a sample training input vector
Figure imgf000052_0006
from the input space.
Figure imgf000052_0007
3. Matching: Find the winning neuron (BMU, best matching unit) that has the weight vector
Figure imgf000052_0004
closest to the input vector. 4. Updating: apply the update equation:
Figure imgf000052_0003
in which η(t) is the learning rate at time step t, and is Gaussian neighbourhood function
Figure imgf000052_0002
centred at the current BMU. 5. Continuation: increment t ← t + 1 and go to step 2 until t is greater than a threshold or the weights stop changing. After the grid representation has been generated in step S202, in step S204, a NURBS is fit to the grid representation, preferably by a maximum likelihood method, as described in the NURBS book3. The output of this process is preferably a surface given by:
Figure imgf000052_0001
2 T. Kohonen, "The self-organizing map," in Proceedings of the IEEE, vol.78, no.9, pp.1464-1480, Sept. 1990, doi: 10.1109/5.58325. 3 The NURBS Book (2nd edition) by Les Piegl & Wayne Tiller – pages 410 to 413 on “Least Squares Curve Approximation” The surface S(u,v) is a parametric surface representation of the continuous surface segment, and one may trace paths in that space 0 ≤ u,v ≤ 1. In other words, once the (in this case, but not necessarily) two-dimensional parametric representation has been generated, a 3D cleaning tool path may be generated in that parametric space, in step S206. Examples of 3D cleaning tool paths which may be generated from isoparametric representations which may be generated in the manner outlined above are shown in Figs. 13A to 13B, but before that, we discuss briefly how an arbitrary path in real space may be converted into coordinates in the parametric space with reference to Fig. 12. Specifically, it is necessary to consider how a displacement in real space may
Figure imgf000053_0007
be mapped to a displacement in the parametric space. In
Figure imgf000053_0006
this example, it will be noted that the displacement only
Figure imgf000053_0008
corresponds to a shift in the ^^ direction in the parametric space, but it will be appreciated that the steps below may straightforwardly be generalized to calculate the shift in the ^^ direction too, if necessary. Various vectors are shown in Fig. 12: · B represents the direction of the displacement in the
Figure imgf000053_0009
real space. · represents the normal to the plane at a point
Figure imgf000053_0001
Figure imgf000053_0002
· representing the differential displacement
Figure imgf000053_0003
vector in the metric space as is changed.
Figure imgf000053_0004
· representing the unit vector in metric space
Figure imgf000053_0005
representing the direction in which one would travel as is changed by an amount
Figure imgf000053_0010
Also, we may define though this is not shown in
Figure imgf000054_0004
Fig. 12. Here
Figure imgf000054_0005
represents the unit vector in metric space representing the direction in which one would travel as is
Figure imgf000054_0014
changed by an amount
Figure imgf000054_0006
Using differential geometry and linear algebra, it is therefore possible to obtain a linear approximation to convert to the corresponding displacement in parametric space,
Figure imgf000054_0007
This way, it is possible to map the desired displacement in metric space
Figure imgf000054_0002
which is the spacing between strokes (dependent on tool geometry, etc.) to corresponding to this
Figure imgf000054_0013
displacement, since we are tracing the path in metric space. Based on these, it is possible to calculate as follows:
Figure imgf000054_0003
Figure imgf000054_0001
Here, the angle brackets in the denominator represent an inner product operation. Fig. 13A shows an example in which the surface is to be cleaned using parallel strokes in parametric space. This may be achieved as follows: · Input: NURBS, or any other, parametric surface representing the continuous surface segment Si to be cleaned. · Let be a chosen constant increment in parametric space
Figure imgf000054_0011
for the parameter ^^ in parametric space (e.g. = 0.1).
Figure imgf000054_0010
· Let and
Figure imgf000054_0008
Figure imgf000054_0009
· Let be a chosen constant spacing between the parallel
Figure imgf000054_0012
strokes in real space, e.g. 0.05m. As discussed elsewhere in the application, this parameter may be dependent on cleaning tool geometry and the desired degree of overlap between adjacent cleaning strokes. · Let initially empty, be a list of cleaning
Figure imgf000055_0001
strokes, where each stroke is given by a list
Figure imgf000055_0002
of waypoints. While < 1, do:
Figure imgf000055_0003
1. Initialise to be an empty list of waypoints (empty
Figure imgf000055_0021
stroke). 2. While < 1, do:
Figure imgf000055_0004
a. Query waypoint at position
Figure imgf000055_0007
Figure imgf000055_0005
b. Append waypoint to stroke
Figure imgf000055_0006
c. Move forward on the stroke: i. Compute the normal of which is the
Figure imgf000055_0018
normalised cross product of the partial derivatives with respect to
Figure imgf000055_0016
and (see
Figure imgf000055_0017
above). ii. Define and as above.
Figure imgf000055_0011
Figure imgf000055_0012
iii. Calculate
Figure imgf000055_0013
as outlined above. iv. Update parametric coordinate
Figure imgf000055_0014
as
Figure imgf000055_0015
3. Append generated stroke to the list of cleaning strokes
Figure imgf000055_0019
4. Update parametric coordinate
Figure imgf000055_0009
as
Figure imgf000055_0010
Performing this process until = 1 will yield a list of
Figure imgf000055_0020
parallel strokes, approximately spaced by
Figure imgf000055_0008
Fig. 13B shows an alternative example, in which circular strokes are generated. Below, we set out an equivalent method for generating a path, in which the coordinates and are
Figure imgf000055_0023
Figure imgf000055_0022
mapped to polar coordinates and · Input: NURBS, or any other, parametric surface representing the continuous surface segment Si to be cleaned. · Let define the polar mapping from
Figure imgf000056_0001
for a point lying on a circle with
Figure imgf000056_0017
Figure imgf000056_0020
radius in parametric space at radians.
Figure imgf000056_0019
Figure imgf000056_0018
· Let initially empty, be a list of cleaning
Figure imgf000056_0016
strokes, where each stroke is given by a list
Figure imgf000056_0015
of waypoints. · Initialize where may be e.g. 0.05 (i.e.
Figure imgf000056_0013
Figure imgf000056_0014
the radius of the circle must be at least 5cm. · Given a choice of updates the algorithm then
Figure imgf000056_0010
proceeds as set out below. While do:
Figure imgf000056_0011
1. Initialise to be an empty list of waypoints (empty
Figure imgf000056_0012
stroke). 2. While do:
Figure imgf000056_0003
a. Let
Figure imgf000056_0004
b. Query waypoint at position
Figure imgf000056_0006
Figure imgf000056_0005
c. Append waypoint to stroke
Figure imgf000056_0007
d. Move forward on circle: i. Update angular coordinate
Figure imgf000056_0008
3. Append generated stroke to the list of cleaning strokes
Figure imgf000056_0009
4. Update parametric coordinate as
Figure imgf000056_0002
Performing this process until will yield a list of
Figure imgf000057_0008
concentric circular strokes, approximately spaced by
Figure imgf000057_0009
Fig. 13C shows a third example in which a series of strokes which are parallel to the edge of the surface are generated. · Input: NURBS, or any other, parametric surface representing the continuous surface segment Si to be cleaned. · Let where
Figure imgf000057_0010
Figure imgf000057_0016
define a mapping from for a point lying
Figure imgf000057_0011
Figure imgf000057_0012
on a square with side length in parametric space at
Figure imgf000057_0021
Figure imgf000057_0013
· Let initially empty, be a list of cleaning
Figure imgf000057_0015
strokes, where each stroke is given by a list
Figure imgf000057_0014
of waypoints. · Initialize where may be e.g. 0.05 (i.e.
Figure imgf000057_0017
Figure imgf000057_0018
the side length of the square must be at least 5cm. · Given a choice of updates the algorithm then
Figure imgf000057_0019
proceeds as set out below. While do:
Figure imgf000057_0020
5. Initialise to be an empty list of waypoints (empty
Figure imgf000057_0001
stroke). 6. While do:
Figure imgf000057_0007
a. Let
Figure imgf000057_0003
b. Query waypoint at position
Figure imgf000057_0005
Figure imgf000057_0002
c. Append waypoint to stroke
Figure imgf000057_0006
d. Move forward on the square: i. Update angular coordinate
Figure imgf000057_0004
7. Append generated stroke ^^ to the list of cleaning strokes 8. Update parametric coordinate as
Figure imgf000058_0001
Figure imgf000058_0004
Performing this process until will yield a list of corner
Figure imgf000058_0002
concentric strokes, approximately spaced by
Figure imgf000058_0003
After the 3D cleaning tool path has been generated, for example using the above methods, in step S206 of Fig. 10, the generated 3D leaning tool path (preferably in the form of a list of waypoints) may optionally then be transmitted to the filtering module 1086, step S208. The operation of the filtering module 1086 will be described later, after an explanation of the generation of a 3D tool cleaning path using the isoplanar representation. We now describe the processes involved in generating an isoplanar representation of a continuous surface segment Si. At a high level, the isoplanar method generates 3D cleaning tool paths by intersecting a plane with a given mesh model generated from an input continuous surface segment Si. This process may be performed by the isoplanar representation generation module 1082 of the representation generation module 1080 of the tool path generation module (CSS) 1076 of the processor 1070 of the control system 104. The steps are illustrated in Fig. 14A. In a first step S302, a triangular mesh is generated, and in step S304, a 3D tool path is generated based on the triangular mesh. There are many well-known methods of generating a triangular mesh representation of a surface from a plurality of points. One example method is the use of a ball-pivoting algorithm (BPA). The principle of the BPA is very simple: Three points form a triangle if a ball of a user-specified radius p touches them without containing any other point. Starting with a seed triangle, the ball pivots around an edge (i.e., it revolves around the edge while keeping in contact with the edge's endpoints) until it touches another point, forming another triangle. The process continues until all reachable edges have been tried, and then starts from another seed triangle, until all points have been considered. The process can then be repeated with a ball of larger radius to handle uneven sampling densities. A detailed explanation of the ball- pivoting algorithm can be found in Bernardini et al. (1999)4. In step S304, the 3D cleaning tool path is generated. An example algorithm by which the path may generated is as follows. A schematic diagram showing the plurality of cutting planes is provided in Fig. 14B · Input: triangular mesh model representing the surface to be cleaned. · Let be the maximum and minimum limits of the
Figure imgf000059_0002
surface. · Let be the position and orientation of the 3D
Figure imgf000059_0003
cleaning tool 102 (specifically, the end-effector 110 thereof) with respect to a fixed coordinate frame (the continuous surface segment Si to be cleaned), representing by a 3 × 4 matrix, where all
Figure imgf000059_0001
elements are column vectors of three dimensions. · Based on the current orientation of the robot contained in
Figure imgf000059_0004
the equation of a cutting plane is defined as where a basic choice for can be
Figure imgf000059_0005
Figure imgf000059_0006
either of the horizontal vectors given by the 3D cleaning tool’s pose This choice may be dependent on
Figure imgf000059_0010
cleaning task, and desired direction of cleaning strokes as well as the choice of reference frame. Setting
Figure imgf000059_0007
defines the plane at the starting edge of the
Figure imgf000059_0009
surface, which can then gradually be moved towards
Figure imgf000059_0008
4 F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva and G. Taubin, "The ball-pivoting algorithm for surface reconstruction," in IEEE Transactions on Visualization and Computer Graphics, vol.5, no.4, pp.349-359, Oct.-Dec.1999, doi: 10.1109/2945.817351. · Let initially empty, be a list of cleaning
Figure imgf000060_0010
strokes, where each stroke is given by a list
Figure imgf000060_0009
of waypoints. · Let
Figure imgf000060_0011
be a chosen constant spacing between parallel strokes, e.g. 0.05m. This parameter, as discussed elsewhere, may be dependent on e.g. the geometry of the 3D cleaning tool, and the desired degree of overlap between adjacent cleaning strokes. The algorithm may then proceed as follows: 1. Let
Figure imgf000060_0001
2. While do:
Figure imgf000060_0003
a. Let be a stroke
Figure imgf000060_0004
generated from intersecting the mesh model with the plane equation given by
Figure imgf000060_0005
b. Move the plane forward: i. Update the plane position
Figure imgf000060_0006
c. Append generated stroke to the list
Figure imgf000060_0008
Figure imgf000060_0007
The final output of the algorithm may then be a list of parallel strokes which are spaced by approximately
Figure imgf000060_0002
After the 3D cleaning tool path has been generated, for example using the above methods, in step S304 of Fig. 10, the generated 3D cleaning tool path may optionally then be transmitted to the filtering module 1086, in step S306. The filtering tool may be configured to determine based on data indicating the geometry and kinematics of the 3D cleaning tool 102, e.g. in the form of unified robot description format data, whether the end-effector 110 of the 3D cleaning tool 102 is able to access all of the generated waypoints of each 3D cleaning tool path. If not, the inaccessible points are removed from the plurality of waypoints. In some cases, if it is determined that too many of the waypoints are inaccessible, e.g. by comparison to a threshold, then the 3D cleaning tool path may be rejected entirely. The threshold may take the form of a simple number of waypoints, or may be in terms of an accessible area of the continuous surface segment Si in question. The output of the filtering module 1086 may be a filtered plurality of points forming the waypoints of the generated 3D cleaning tool path, and may be transmitted from the control system 104 to the 3D cleaning tool 102, for execution. In some cases, the instruction generating module 1094 may generate some executable instructions based on the filtered plurality of points, and these instructions may be sent to the 3D cleaning tool 102 for execution. The above detailed description focuses on the detection of continuous surface segments, and the subsequent generation of appropriate 3D cleaning tool paths. While most furniture cleaning tasks are directed towards the surfaces of items of furniture, some tasks require knowledge of the intersections between those surfaces, referred to in this patent application as “crevices”. Crevice detection may be regarded as the complementary problem to the detection of smooth continuous surfaces. While smooth surfaces are sets of contiguous points with low local curvature, crevices can be characterized as the sets of contiguous points with a high, concave local curvature. We therefore now consider the detection of crevices, and the subsequent generation of 3D cleaning tool paths for cleaning those crevices. Crevice detection, i.e. step S3 of Fig. 3B, may be performed by the crevice detection module 1060 of the processor 1052 of the input generation system 105. Then, in step S4 of Fig. 3B, tool path generation based on the detected crevices, may be performed by the tool path generation module (crevice) 1078 of the processor 1070 of the control system 104. We will consider steps S3 and S4 in more detail, in turn. Referring once again to Fig. 4, the crevices are the narrow trench regions between e.g. surfaces (1) and (2), (1) and (3), (1) and (4), (2) and (3), and (2) and (4). On receiving a 3D electronic representation of the chair (preferably in the form of a 3D point cloud which sets out the location of various points on the outer surface of such a chair), the purpose of the crevice detection process shown in Fig. 15 in the form of a flowchart is to identify the crevices described above, and preferably to indicate which of the plurality of points in the 3D electronic representation are located in each respective crevice. In step S30 of Fig. 15, the 3D electronic representation of the outer surface of the object to be cleaned may be received by the input generation system 105 (e.g. from the 3D camera 106 or the external computing device 108, via the interface module 1050). Implicitly, it will be appreciated that prior to the method of Fig. 5, the 3D camera 106 may obtain a 3D representation of the object to be cleaned. Alternatively, the 3D representation may exist in the form of a CAD representation on the external computing device 108, which simply transmits that representation to the input generation system 105. After the 3D electronic representation has been received in step S30, the curvature of each of the points in the 3D representation is determined in step S32. This may be done by considering the eigenvalues of the covariance matrix, as explained earlier in this application. For conciseness, the explanation will not be repeated here. Next, in step S34, those points having a sufficiently high curvature are identified, e.g. by comparison with a threshold. Then, in step S36, the concave points are identified. In some cases, the order of steps S34 and S36 may be reversed. The eigenvalue method of estimating curvature does not yield any information as to whether the surface is concave or convex, and accordingly, an additional step may be required. In order to determine whether the point is concave or convex, at each point Pi with normal ni, the average distance from its k neighbours to the tangent plane perpendicular to ni at the point is obtained. If this distance is positive, it means that the neighbouring points are above the plane, and thus the curvature at the point Pi is concave. And, if the distance is negative, it means that the neighbouring points are below the plane, and thus the curvature at Pi is convex. In some cases, based on the result of the determination, a sign can be added to the curvature values in order to designate concavity or convexity, e.g. + for concave, or – for convex (or vice versa). Then, from that list of signed curvatures, the concave points which have a curvature greater than the predetermined local curvature threshold may be selected. Now that steps S34 and S36 have been completed, the process moves to step S38, in which data representing the identified crevices may be transmitted to the control system 104, preferably the tool path generation module (crevice) 1078 thereof. In implementations in which the input generation module 105 and the control system 104 are the same component, or part of the same component, this step S38 may not take place. An example of an output of the crevice detection process is shown in Fig. 15B in which the light regions represent the detected crevices of an armchair. At this point, the method moves to step S4 of Fig. 3B, namely the generation of a 3D cleaning tool path based on the detected crevices. This step is performed by the tool path generation module (crevice) 1078, which may comprise: a graph generation module 1088, a graph reduction module 1090, and a traversal algorithm module 1092. As is indicated from the names of the modules, and shown in Fig. 16, the tool path generation process may include, in step S40 generation of a graph by the graph generation module 1088, in step S42 reduction of the graph by the graph reduction module 1090, and in step S44 applying a complete traversal algorithm to the reduced graph by the traversal algorithm module 1092. The output of this process is, as with the 3D cleaning tool paths generated in respect of the continuous surface segments, a path in the form of e.g. a series of nodes. In a final step S46, the instruction generation module 1094 may, based on the path, generate instructions, perhaps in the form of a plurality of waypoints to be visited by the end-effector 110 of the 3D cleaning tool 102, in use. The path may include a series of segments, each in turn made up from a six-dimensional vector specifying the position and orientation of the end-effector 110 of the 3D cleaning tool 102, akin to the paths generated in respect of the continuous surface segments, explained in detail above. There are, however, a few differences between crevices and continuous surface segments which affect how the path should be computed. Firstly, crevices include elongated stretches which are long in one dimension (i.e. along the crevice), but short in the other two dimensions (i.e. the boundary surfaces). Secondly, the points in the crevice have, by definition, high concave curvature. And, thirdly, a single detected crevice may comprise a plurality of branches. This means that the techniques which are used to generate the 3D cleaning tool paths for continuous surface segments may not be suitable for generating 3D tool cleaning paths along crevices. At a high- level, the techniques used to generate a 3D tool path covering all of the crevices involves finding a path along the centre of a ridge, converting it into a series of connected nodes, and applying a complete traversal algorithm to generate a multipath message. Fig. 17 shows a graph generation process, in which a graph is generated from the 3D electronic representation of the outer surface of the object to be cleaned, which may be in the form of a point cloud. Nodes in the graph correspond to points and normals, and the position and normals of the points are stored as node attributes. The edges between the nodes store information on the relationship between the points which they connect. In a first step S402, the point cloud may be downsampled at a predetermined downsampling resolution, which may be expressed as a voxel size. Every resulting point from the downsampling may then be considered a node of the graph. Then, in step S404, edges are generated between pairs of nodes which are located within a threshold distance of each other. Finally, in step S406, edge attributes are calculated, namely a Euclidean distance between the two nodes, and a normal difference. These terms are defined earlier in this application. The output of the process of Fig. 17 is thus a graph comprising a plurality of nodes (each having associated attributes) and edges connecting various pairs of nodes (each edge also having associated attributes). When generating the graph, there is a trade-off to be considered when selecting the downsampling resolution: too coarse and details on the path may be missed, and displacements of the voxel grid may heavily influence the results. Too fine, and the number of nodes and connections sometimes creates parallel paths along a single crevice edge, and complete traversal may take too long. Moreover, some of the original points may be located on the limiting surfaces, rather than on the crevice itself, and their normals may therefore not correspond with the desired direction of the crevice cleaning tool. The graph reduction process of step S42 aims to address some of these issues. The process is shown in more detail in Fig. 18. In step S502, for each pair of calculated nodes, an average node is calculated, which is located halfway along the edge connecting those nodes. Then, in step S504, pairs of average nodes which are within a predetermined threshold distance (preferably the same as the voxel size or downsampling resolution) of each other are merged with each other. This process acts to reduce the number of nodes. The process shown in Fig. 18 may be repeated a predetermined number of times, in order to obtain a reduced graph. In this way, it is possible to start with a graph generated using a very fine downsampling resolution, and end up with a reduced graph which represents every crevice branch uniquely. Figs. 19A to 19F show an example of a graph being reduced over five iterations. It will be noted that in the final image, the graph has been reduced to a single path covering each crevice branch, with the exception of a few outliers. It has been shown that, with a higher number of iterations, the path is smoother, however, the branches are also shorter, as a consequence of averaging the extreme points in step S502. There is therefore a trade-off, which may be influenced by many factors. The preferred number of iterations may be 2, 3, 4, 6, 7, 8, 9, or 10. After the graph reduction, in step S44 of Fig. 16 takes, place: the application of a complete traversal algorithm to the reduced graph generated in step S42. Given the reduced graph (or the original graph, in cases in which no graph reduction step is required, or performed for another reason), the purpose of the complete traversal algorithm is to identify one or more paths along the various branches of the detected crevices so that all of the determined points on the graph are covered or traversed by the end-effector 110 of the 3D cleaning tool 102. It should be stressed that various forms of complete traversal algorithm may be used, and the algorithm discussed now with reference to Fig. 20 is only one preferable example. As discussed, the input to the algorithm is either the generated graph, or the reduced graph. In some cases, the input may be the raw data in the form of the 3D point cloud, or other electronic representation of the points which are located within the crevices of the object to be cleaned. In the following discussion, we refer to it simply as the “input”. Broadly speaking, the algorithm works by identifying the longest shortest path (i.e. the shortest path spanning two extremes of the graph), storing (e.g. to a buffer 1074) that path, and removing those nodes from consideration. The process then repeats itself until all nodes have been covered, or the remaining paths are too short (i.e. below a predetermined length threshold, or number of nodes). In a first step S440, a minimum spanning tree is computed, in order to remove cycles from the graph. Intuitively, this just refers to cutting any loops that may exist in the original graph. A cycle in a graph refers to a situation where, starting from node A, and following the edges, it is possible to return to the same node without going twice over any edge. Removing the cycles means that now, departing from node A, it is only possible to return to node A again by backtracking, and hence going at least twice over some of the edges. Then, in order to remove small side branches, or other small branches, in step S442, the tree is pruned. This may include removing all points from the graph which are located in branches which are below a threshold length (either in terms of actual length, or number of nodes). If Fig. 21A shows the original graph, the output of steps S440 and S442 are shown in Fig. 21B, which shows the pruned minimum spanning tree. The crevice areas on a piece of furniture (or other object to be cleaned) may not be all be connected in one large tree – there may be several disconnected trees. In order to address this, connected components are identified in step S444, and the identified connected components are separated. In step S445, one of the connected components may be selected (e.g. the largest, or selection may be random). The next step may be to identify the longest shortest path between all nodes in the selected connected component. In order to do so, in step S446 the two nodes which are located the furthest away from each other may be identified, and in step S448, the shortest path (i.e. through the nodes of the graph) between those two points may be identified, a result of which may be shown in Fig. 21C. Then, in step S450, this path may be added to e.g. a list of paths which may be stored in the buffer 1074, or otherwise stored, at least temporarily. In related step S452, the nodes on the graph from the identified path may be removed from consideration. In step S454, it may be determined whether all of the nodes in the connected component (with the possible exception of outliers) have been included in a path. If not, the method may return to step S446. If so, the method may proceed to step S456 in which it may be determined whether all of the connected components have been processed. If not, the process may return to step S445, and a set of paths may be generated for a next connected component. If the result of the determination in step S456 is positive, a path has been generated which traverses all points in the graph (or raw data), and the algorithm ends. The result of the algorithm is a list of paths, where each path corresponds to an ordered list of nodes, such that all of the branches corresponding to the crevices of the object to be cleaned are visited. Nodes in the graph correspond (at least approximately) to points in the point cloud, so their attributes (i.e. the node attributes) can be used to find, in step S46 of Fig. 16, waypoints that can be sent to the 3D cleaning tool 102, in order to cause the end-effector 110 thereof (in this case preferably in the form of a crevice cleaning tool) to traverse those waypoints. The following process of generating waypoints may also be applied in the case of the path generated to cover the continuous surface segments, as explained previously. The steps may be performed by the instruction generation module 1094 of the control system 104, and are illustrated in Fig. 22. In a first step, S480, the instruction generation module 1094 may receive the paths generated by the tool path generation module (CSS) 1076 or tool path generation module (crevice) 1078. Then, for each path generated in the previous step, the following steps may be performed: in step S482, a spline may be fitted along the path, using standard techniques. Then, for each point on the path, a frame (i.e. a frame of reference or a coordinate system, or set of axes) may be fitted in step S484. x, y, and z may be used to refer to the XYZ vectors which define a frame of reference. A frame of reference defines a 6 degree of freedom “pose” (position and orientation) with respect to any other frame of reference. The end-effector of the 3D cleaning tool has a frame of reference that describes its pose with respect to the reference frame in the base of the robot. The XYZ vectors defined here create a list of frames of reference (one per waypoint) which are the poses the end-effector aims to accomplish in order to complete the cleaning task. At this point, for a given object to be cleaned, such as the chair in Fig. 4, the continuous surface segments and/or the crevices therebetween may be identified. Then, instructions for the 3D cleaning tool 102 may be generated, in order to enable the end-effector 110 thereof to perform the desired cleaning task. The features disclosed in the foregoing description, or in the following claims, or in the accompanying drawings, expressed in their specific forms or in terms of a means for performing the disclosed function, or a method or process for obtaining the disclosed results, as appropriate, may, separately, or in any combination of such features, be utilised for realising the invention in diverse forms thereof. While the invention has been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the exemplary embodiments of the invention set forth above are considered to be illustrative and not limiting. Various changes to the described embodiments may be made without departing from the spirit and scope of the invention. For the avoidance of any doubt, any theoretical explanations provided herein are provided for the purposes of improving the understanding of a reader. The inventors do not wish to be bound by any of these theoretical explanations. Any section headings used herein are for organizational purposes only and are not to be construed as limiting the subject matter described. Throughout this specification, including the claims which follow, unless the context requires otherwise, the word “comprise” and “include”, and variations such as “comprises”, “comprising”, and “including” will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps. It must be noted that, as used in the specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” one particular value, and/or to “about” another particular value. When such a range is expressed, another embodiment includes from the one particular value and/or to the other particular value. Similarly, when values are expressed as approximations, by the use of the antecedent “about,” it will be understood that the particular value forms another embodiment. The term “about” in relation to a numerical value is optional and means for example +/- 10%.

Claims

CLAIMS 1. A computer-implemented method of identifying one or more continuous smooth surface segments of an object to be cleaned by a 3D cleaning tool, the computer-implemented method comprising: receiving a 3D electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; applying a surface segmentation algorithm to the electronic representation of the surface S, thereby identifying one or more continuous surface segments Si of the surface S, wherein: each of the continuous surface segments Si comprises a subset of the plurality of points, and the curvature of each of the continuous surface segments Si is below a predetermined curvature threshold.
2. A computer-implemented method according to claim 1, wherein: the surface segmentation algorithm is a region growing algorithm configured to identify the one or more continuous surface segments Si by: identifying a first seed point P1 and selecting a set of neighbouring points to the first seed point P1, such that the resulting surface sub-segment, comprising the first seed point P1 and its neighbouring points has a curvature below the predetermined curvature threshold; and iteratively repeating the process, with each neighbouring point in the resulting surface sub-segment becoming a seed point.
3. A computer-implemented method according to claim 2, wherein: the region growing algorithm comprises the steps of: (a) selecting a point P1 having the lowest curvature C1 as a first initial seed point, the point P1 having one or more neighbouring points; (b) identifying the subset of neighbouring points which meet: a predetermined smoothness criterion and a predetermined curvature criterion; (c) adding the identified subset of neighbouring points to a cluster of points forming a first continuous surface segment S1; (d) repeating steps (b) and (c) using each of the identified subset of neighbouring points as new seed points, until no identified neighbouring points meet both the smoothness and curvature criteria.
4. A computer-implemented method according to claim 3, wherein: the smoothness criterion is that the seed point and the neighbouring point are not separated by an edge.
5. A computer-implemented method according to claim 4, wherein: identifying the neighbouring points which meet the predetermined smoothness criterion comprises: determining the angle θn between the normal N1 of the first seed point and each of its neighbouring points Pn; selecting the set of neighbouring points for which the angle θn is below a predetermined threshold θTH.
6. A computer-implemented method according to any one of claims 3 to 5, wherein: the curvature criterion is that the local curvature of the neighbouring point is below the predetermined curvature threshold.
7. A computer-implemented method according to any one of claims 3 to 6, wherein: when it is determined that no neighbouring points of a set of new seed points meet both the predetermined smoothness criterion and the predetermined curvature criterion, the computer-implemented method further comprises: of the remaining points of the plurality points which have not yet been added to a cluster representing a continuous smooth surface Si, selecting a point P2 having the lowest curvature C2 as a second initial seed point; and repeating steps (b) to (d) for the second initial seed point P2, thereby generating a cluster of points forming a second continuous surface segment S2.
8. A computer-implemented method according to any one of claims 3 to 7, wherein: the predetermined smoothness threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or surface-contacting end effector thereof; the nature of cleaning task to be performed; and the nature of the outer surface to be cleaned.
9. A computer-implemented method according to any one of claims 3 to 8, wherein: the predetermined curvature threshold is selected based on one or more of: the geometry and/or kinematics of the 3D cleaning tool, or a surface-contacting end effector thereof; the nature of cleaning task to be performed; and the nature of the outer surface to be cleaned.
10. A computer-implemented method according to claim 1, wherein: the surface segmentation algorithm is a random sample consensus method (RANSAC) algorithm.
11. A computer-implemented method according to claim 10, wherein: the RANSAC algorithm is configured to identify a first set of parameters of a model which represents a first continuous surface segment S1; and either: (a) the computer-implemented method further comprises identifying a first subset of points which are located within the first continuous surface segment S1 and adding them to a first cluster of points which form the first continuous surface segment S1; or (b) the computer implemented method further comprises: identifying a first subset of points which are located within the first continuous surface segment S1; identifying one or more connected sets of points within the first subset of points, and adding them to a first cluster of points which form the first continuous surface segment S1.
12. A computer-implemented method according to claim 11, wherein: the computer-implemented method further comprises: applying the RANSAC algorithm to the remaining points which have not been added to the first cluster of points, to identify a second set of parameters of a model which represents a second continuous surface segment S2; and either: (a) identifying a second subset of points which are located within the second continuous surface segment S2 and adding them to a second cluster of points which form the second continuous surface segment S2; or (b) identifying a second subset of points which are located within the second continuous surface segment S3; identifying the largest connected set of points within the second subset of points, and adding them to a second cluster of points which form the second continuous surface segment S2.
13. A computer-implemented method according to any one of claims 10 to 12, wherein: the RANSAC algorithm comprises the steps of: (a) selecting N points from the plurality of points, wherein N is at least the number of points required to estimate the parameters of a parametric model representing the continuous surface segment Si; (b) determining the values of the parameters for a continuous surface segment Si including the N selected points; (c) determining how many points of the plurality of points fit the model within a predetermined tolerance level; (d) repeat steps (a) to (c) a predetermined number of times, and identify the set of parameters which maximize the value determined in step (c).
14. A computer-implemented method according to claim 13, wherein: the RANSAC algorithm further comprises re-estimating the model parameters using all of the points which are located within the continuous surface segment Si defined by the model having the set of parameters identified in step (d) to generate a final set of model parameters.
15. A computer-implemented method according to any one of claims 1 to 14, wherein: the computer-implemented method further comprises generating a 3D tool cleaning path based on the one or more identified continuous smooth surface segments.
16. A method of identifying one or more continuous smooth surface segments of an object to be cleaned by a 3D cleaning tool, the method comprising: generating an electronic representation of an outer surface S of the object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; transmitting the generated electronic representation to a computer or other data processing unit; and performing the computer-implemented method of any one of claims 1 to 15.
17. A method according to claim 16, wherein: generating the electronic representation comprises one or more of the following: obtaining a CAD representation of the object to be cleaned; or obtaining a 3D representation from a 3D camera.
18. An input generation system for a control system for a 3D cleaning tool, the system comprising: an input generation module configured to generate a representation of an outer surface S of an object to be cleaned, the representation comprising a plurality of points Pi, each point representative of a point on the outer surface of the object to be cleaned; a transmitter; and a computer or other data processing unit, wherein: the transmitter is configured to transmit the generated electronic representation to a computer or other data processing unit; and the computer or other data processing unit is configured to perform the computer-implemented method of any one of claims 1 to 15.
PCT/GB2023/050168 2022-01-28 2023-01-26 Continuous smooth surface segmentation for object cleaning using a 3d cleaning tool WO2023144535A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB2201118.3 2022-01-28
GB2201118.3A GB2615304A (en) 2022-01-28 2022-01-28 Continuous smooth surface segmentation

Publications (1)

Publication Number Publication Date
WO2023144535A1 true WO2023144535A1 (en) 2023-08-03

Family

ID=80621191

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2023/050168 WO2023144535A1 (en) 2022-01-28 2023-01-26 Continuous smooth surface segmentation for object cleaning using a 3d cleaning tool

Country Status (2)

Country Link
GB (1) GB2615304A (en)
WO (1) WO2023144535A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103914875A (en) * 2014-04-17 2014-07-09 中国科学院深圳先进技术研究院 Functional modeling method for indoor scene
US20200111251A1 (en) * 2018-10-08 2020-04-09 The Hong Kong Polytechnic University 3d indoor modeling method, system and device based on point cloud data

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103885385B (en) * 2014-03-07 2016-08-24 华侨大学 A kind of processing method of dividing and ruling of triangle grid model
CN109541997B (en) * 2018-11-08 2020-06-02 东南大学 Spraying robot rapid intelligent programming method for plane/approximate plane workpiece
CN110223308B (en) * 2019-04-15 2023-05-09 东南大学 Stacking site cloud positioning method based on edge detection and region growing
CN110716502B (en) * 2019-11-27 2020-10-23 北京航空航天大学 Tool path generation method and device for engine blade repair

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103914875A (en) * 2014-04-17 2014-07-09 中国科学院深圳先进技术研究院 Functional modeling method for indoor scene
US20200111251A1 (en) * 2018-10-08 2020-04-09 The Hong Kong Polytechnic University 3d indoor modeling method, system and device based on point cloud data

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ROYCHOUDHURY ARINDAM ET AL: "Plane Segmentation in Organized Point Clouds using Flood Fill", 2021 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), IEEE, 30 May 2021 (2021-05-30), pages 13532 - 13538, XP033990245, DOI: 10.1109/ICRA48506.2021.9561325 *
RUSU R B ET AL: "Towards 3D Point cloud based object maps for household environments", ROBOTICS AND AUTONOMOUS SYSTEMS, ELSEVIER BV, AMSTERDAM, NL, vol. 56, no. 11, 30 November 2008 (2008-11-30), pages 927 - 941, XP025561289, ISSN: 0921-8890, [retrieved on 20080820], DOI: 10.1016/J.ROBOT.2008.08.005 *

Also Published As

Publication number Publication date
GB202201118D0 (en) 2022-03-16
GB2615304A (en) 2023-08-09

Similar Documents

Publication Publication Date Title
Attene et al. Hierarchical mesh segmentation based on fitting primitives
Kalogerakis et al. Extracting lines of curvature from noisy point clouds
Wand et al. Reconstruction of deforming geometry from time-varying point clouds
CN111986115A (en) Accurate elimination method for laser point cloud noise and redundant data
Jenke et al. Bayesian point cloud reconstruction
Rusu et al. Towards 3D object maps for autonomous household robots
US8004517B1 (en) Methods, apparatus and computer program products that model three-dimensional surface structures
Tam et al. Shape simplification based on the medial axis transform
Dzitsiuk et al. De-noising, stabilizing and completing 3D reconstructions on-the-go using plane priors
WO2008155259A1 (en) Method of segmenting anatomic entities in 3d digital medical images
Leal et al. A linear programming approach for 3D point cloud simplification
Barhak et al. Adaptive reconstruction of freeform objects with 3D SOM neural network grids
WO2023144534A1 (en) Crevice detection
Sagawa et al. Adaptively merging large-scale range data with reflectance properties
WO2023144533A1 (en) 3d cleaning tool path generation
US20090202150A1 (en) Variable resolution model based image segmentation
Branch et al. Automatic hole-filling of triangular meshes using local radial basis function
Liu et al. Reconstructing B-spline curves from point clouds--A tangential flow approach using least squares minimization
Song et al. Active 3D modeling via online multi-view stereo
Jalba et al. Efficient surface reconstruction from noisy data using regularized membrane potentials
WO2008155255A1 (en) Method of constructing a gray value model and/or a geometric model of an anatomic entity in a 3d digital medical image
WO2023144535A1 (en) Continuous smooth surface segmentation for object cleaning using a 3d cleaning tool
WO2023144536A1 (en) 3d cleaning tool path generation
Franchini et al. Implicit shape reconstruction of unorganized points using PDE-based deformable 3D manifolds
Heckel et al. Surface reconstruction using adaptive clustering methods

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23703838

Country of ref document: EP

Kind code of ref document: A1