JP2023530209A - Method for automatically manipulating an object using a tool held by an articulated system - Google Patents

Abstract

The present invention is capable of operating within a work environment (3) in situations where the object (2) and the work environment (3) are variable or poorly defined for operation. The present invention relates to a method and apparatus for automatically performing operations on an object (2) using a tool (4) held by an articulated system (5). According to the invention, the method comprises: - capturing an image of the object (2) and the working environment (3) in the form of a scatter plot using at least one 3D sensor (6); step A of merging with a CAD model of the object (2) and a possible CAD model of the environment into a workpiece image and defining anti-collision parameters; - defining a path for the tool (4) to a part of the workpiece image representing the object (2); , a step B of simulating the corresponding motions of the articulated system (5) and the tool in the working image to ensure that the operation is executable; - if the operation is determined to be executable in step B; , and a step C of actually carrying out the movement of the articulated system (5) holding the tool (4) according to the defined path (10) for performing the operation on the object (2).

Description

The present invention is useful in situations where the position and geometry of the object and its work environment may change and/or are poorly defined for performing operations on the object, and in the work environment. It relates to the technical field of operations such as cutting, welding, marking, stripping, painting, surface treatment, positioning of sensors and all types of analytical tools, etc., on an object.

For example, but not exclusively, the invention applies to demolition engineering departments where operations of measuring, cutting and gripping objects that may have radioactivity-related risks are performed.

More particularly, the present invention is useful for many applications in situations where the position and geometry of objects and work environments are changing and/or are poorly defined for their operation in an automatic mode. The present invention relates to methods and apparatus for automatically performing manipulations on objects using tools held by articulated systems and movable within a work environment. Manipulation can be of any kind performed on any type of object using any type of tool under any type of environment.

The invention is advantageously applied, for example, when performing any manipulation on an object in a hazardous and enclosed environment in which the operator cannot access or freely move his or her body. can do.

In the field of the invention, for example the dismantling of radioactive objects, the dismantling operation can be carried out manually by an operator in a closed environment.

Dismantling operations include operations of measuring, cutting, and gripping objects. Among other things, the purpose of the cutting operation is to produce smaller elements, thereby optimizing, for example, the filling of the waste container that contains them. Therefore, these smaller parts need to be handled and containerized by the system.

Each of these operations requires, for example, an angle grinder or any type of tool enabling the performance of the operations involved, as well as restrictive protective equipment ( by workers with ventilated coveralls, safety gloves, etc.).

Therefore, the performance of these operations is not optimal either in terms of operator safety or in terms of difficulty and time required.

Otherwise, the manipulation cannot be performed in contact with the operator and the tools (for measuring, cutting, gripping) will be held by the articulated system that performs the manipulation. Using a control horn, the operator remotely drives the system while viewing indirect images from the camera and screen. Disadvantages of such systems, besides being expensive, are the low accuracy of their control in terms of speed, force and positioning, long time to completion, and rapid wear of tool consumables (blades, discs). mentioned.

An object of the present invention is to provide a method and equipment for performing operations on objects in a working environment, thereby ensuring optimum conditions in terms of safety, accuracy and speed, thereby compensating for the shortcomings of the prior art. do.

To this end, methods have been developed for automatically performing manipulations on objects using tools held by articulated systems that are movable within a working environment.

According to the invention, the method comprises: o capturing an image of all or part of the object and work environment in the form of an overall point cloud using at least one 3D sensor;
o Merge the image with an existing CAD model of the articulated system and possible existing CAD models of all or part of the as-built environment, resulting in a work image,
o Step A of defining anti-collision parameters;
- The trajectory of the tool can be defined (preferably by the operator) for the portion of the work image representing the object, and the corresponding articulated system and tool motion can be simulated on the work image to perform the manipulation. Step B to ensure that
and at least step C of actually performing the movement of the articulated system holding the tool according to the defined trajectory to perform the manipulation on the object, if the manipulation was determined to be feasible in step B.

In the present invention, 3D sensor means any means for digitizing an object or part of an object in the form of a three-dimensional image, including inter alia 3D cameras, laser scanners, video means, webcams, etc. is included.

The anti-collision parameter is, for example, a threshold distance beyond which a safety function, such as a total stop of any movement of the articulated system and the tool or a deceleration of the movement, is automatically triggered, e.g. by which the anti-collision parameter up to a threshold speed where

In this way, the present invention enables operations that were initially in uncertain or unknown or inaccessible environments to be performed remotely and in an automated mode while ensuring that they are performed correctly. to

The present invention is advantageous in that the working environment is digitized in the form of a global point cloud prior to any movement of the articulated system. In doing so, the positions of various obstacles to the motion of the articulated system are known, ie the positions of various elements in the work environment and objects of upcoming manipulations. As such, collisions between the articulated system or its tools and various elements in the environment can be avoided by anti-collision parameters.

This is particularly advantageous when the operator controlling the method being carried out does not have a direct view of the working environment. The operator visualizes the work environment and the object as a point cloud on the display and defines the motion trajectory of the tool to perform the desired operation, eg cutting operation.

The present invention makes it possible to simulate, particularly in CAD, the corresponding movements of an articulated system and its tools by adopting the method, so that the movements can be performed without causing collisions. But it is advantageous.

Therefore, once the feasibility of the motion is confirmed by the simulation, the multi-joint system holding the tool can actually be moved and the object can be manipulated accordingly.

The method can be implemented in at least three steps. i.e.
- step A of determining the environment and objects by digitizing them in the form of a global point cloud;
- step B of defining and simulating the trajectory, and - step C of performing.

Preferably, the method uses a 3D sensor held by an articulated system to capture detailed images of zones of the object in the form of point clouds that are denser and more accurate than the global point cloud. Between steps A and B further includes step A' of controlling the motion of the articulated system and incorporating its captured image of the corresponding zone of the object into the work image instead of the global point cloud counterpart.

This additional step provides an improved workpiece image with greater precision for defining the various trajectories of the maneuver to be performed. Furthermore, this additional step allows the motion of the articulated system to capture images of zones not visible to the sensors used in step A, thereby completing the workpiece image.

The 3D sensor held by the articulated system can remain attached to the articulated system at all times or stored in its vicinity. In the latter configuration, the method includes a step A'' between steps A and A' of automatically operating the articulated system to grasp and connect to a 3D sensor stored nearby.

According to one specific embodiment, the method includes step C' of automatically operating the articulated system to optionally remove and unload the 3D sensor and connect it to a tool stored nearby, step B and step C. Including between

This feature allows multiple tools to be kept nearby for connection to an articulated system.

If several different tools are made available, the method further includes a step B' prior to step B of selecting a tool for performing the operation from the tool database for the simulation in step B and a tool database for the simulation. include.

Advantageously and to prevent damage to the tool, in step C the method determines the speed of motion of the articulated system holding the tool in direct contact with the object based on a direct or indirect measurement of the force exerted on the tool. It's also great because you can adjust it in real time as needed.

In order to make the motion of the articulated system holding the 3D sensor in step A' safe, the motion of the articulated system automatically occurs when the 3D sensor or part of the articulated system approaches an object or element of the working environment. and the motion automatically stops when the 3D sensor or part of the articulated system has come to a safe distance according to the anti-collision parameters defined in step A against the object or elements of the work environment.

The definition of the trajectory of the tool for the portion of the work image representing the object according to step B is at least one start-end point pair or at least one predefined geometric shape, comprising at least one line, plane, It consists of positioning in the work image any geometric shape selected from a library of masks and the like.

The present invention provides an installation for implementing the method described above, an articulated system and at least one 3D sensor connected to a computer processing system and a display, comprising:
- representing on a display a global point cloud of all or part of the object and work environment, based on the images captured by at least one 3D sensor;
- A 3D sensor designed to represent the workpiece image obtained by merging the global point cloud, existing CAD models of the articulated system, and possible existing CAD models of all or part of the as-built environment. Regarding the equipment provided,
The computer processing system also allows definition of the trajectory of the tool with respect to the portion of the work image representing the object, and simulates the articulated system and the corresponding motion of the tool on the work image to determine that the manipulation is feasible. It is configured to actually move the articulated system holding the tool according to a defined trajectory to perform the manipulation on the object, once the manipulation has been determined to be feasible.

Advantageously, the installation comprises means for adjusting in real time the speed of movement of the articulated system holding the tool in direct contact with the object in step C based on direct or indirect measurement of the force exerted on the tool.

Other features and advantages of the present invention will become more apparent from the following description, given by way of representation and not by way of limitation, with reference to the accompanying drawings.

1 is a schematic perspective view of an installation according to the invention in which an articulated system holding a tool is about to perform an operation on an object in its working environment; FIG. FIG. 2 is a schematic view similar to FIG. 1, with the articulated system holding a 3D camera; A work image obtained by merging the entire point cloud of the part of the object and work environment obtained from the images captured by one 3D camera and the CAD model of the articulated system on the other hand is represented on the display. is a perspective view. FIG. 4 is a view similar to FIG. 3 in which the articulated system holds a 3D camera for capturing an image of a detail of the object, the detail of the object being the workpiece instead of the corresponding portion of the global point cloud; Figure 2 is a view embedded in an image; FIG. 5 is a view similar to FIG. 4 in which the trajectory of the tool is defined in the portion of the work image representing the object; FIG. 6 is a schematic diagram similar to FIG. 5 showing a simulation of the articulated system and tool motion in the work image; FIG. 10 is a detailed diagram of the positioning of the plane that intersects the portion of the workpiece image representing the object; FIG. 8 is a view similar to FIG. 7, wherein the motion trajectory of the articulated system is automatically calculated by automatically calculating the trajectory obtained by the intersection of the positioned plane and the portion of the workpiece image representing the object; Figure 3 is a calculated figure. Fig. 3 shows the simulation stage of the motion of the articulated system; Fig. 3 shows a simplified flow chart of the method according to the invention;

The present invention relates to a method and installation (1) for automatically performing manipulations on objects (2) placed in a working environment (3).

The invention is not limited to any one operation, but involves operations such as measuring, cutting, gripping, welding, writing, marking, stripping, painting, surface treatment, positioning of sensors and other types of analytical tools and the like. can be Such manipulations are performed by a tool (4) held by an articulated system (5) movable within the working environment (3).

The object (2) to be manipulated according to the method of the invention can be a radioactive object or any other object for demolition or any kind of repair or welding.

The working environment (3) associated with the object (2) is inaccessible to the worker and does not allow the worker to move freely, e.g. It can be of any kind, such as an environment such as

Looking at FIG. 1, the installation (1) comprises an articulated system (5), in particular in the form of a robotic arm that is omnidirectionally movable within the working environment (3). A facility (1) captures and digitizes images of all or part of an object (2) and a work environment (3) and renders the object (2) and work environment (3) by means of known computer processing systems and displays. At least one 3D sensor, eg a 3D camera (6A, 6B, 6C), is provided to render a three-dimensional representation of the global point cloud (7) of all or part of the .

In the example shown, the installation (1) comprises three 3D cameras (6A, 6B, 6C) positioned and fixed in an arch (8) that wraps around and over the articulated system (5).

With reference to FIG. 3, in order to safely operate on the object (2), the equipment (1) arches (8) an image of all or part of the object (2) and the working environment (3). captured in the form of a global point cloud (7) using 3D cameras (6A, 6B, 6C) attached to an existing CAD model of the articulated system (5) and At least one step A of merging with an existing CAD model of the as-built environment, such as the CAD model of the arch (8), and displaying the resulting workpiece image (17) on a display (assembly of Figure 3). implement a method that includes

Thereby, possible obstacles in the work environment (3) can be identified and the operation of the articulated system (5) can be made safe. All or part of the standard elements of the working environment (3) are known, such as the arches (8), the 3D cameras (6A, 6B, 6C) as well as the articulated system (5) itself and its tools (4), It may be noted that it may already be modeled in CAD.

In this way, the motion of the articulated system (5) and its tools (4) can be known in relation to various elements of the work environment (3), automatically avoiding collisions with pre-defined anti-collision parameters. be able to.

In order to perform the desired manipulations on the object (2), the equipment (1) includes several tools (4) of various types, 3D sensors such as a 3D camera (16), etc. in a dedicated storage box. It is preferable to store it in the form of storing it nearby.

This 3D camera (16) can be of the same type as the 3D cameras (6A, 6B, 6C) or different, but in the latter case it is advantageously ) is considered to be more accurate than

Therefore, the method according to the present invention automatically and therefore safely operates the articulated system (5) to deliver it to the 3D camera (16) stored nearby and grasp the 3D camera (16). Advantageously, after step A, a step A'' of connecting to it is included.

After the articulated system (5) grasps the 3D camera (16), the method controls the motion of the articulated system (5) to obtain a 3D camera ( 16) is used to capture detailed images of various zones of the object (2) in the form of point clouds with higher density and precision than the global point cloud (7), and the corresponding zones of the object (2) It includes a step A' (see Figure 2) of incorporating the captured image (9) into the work image (17) (see Figure 4) instead of the corresponding part of the global point cloud (7).

The movement of the articulated system (5) to capture the various images is controlled automatically or by an operator remotely in the working environment (3), e.g. It is possible to capture an image of a zone, or it can be controlled, for example, by directly selecting a specific zone from the global point cloud (7), the selection of which triggers the automatic movement of the articulated system. In this configuration, the actual operator selects the zones in which he wishes to refine the modeling of the object. The software then calculates the position of the articulated system (5) required to capture the image from the best viewpoint. It should be noted here that the motion of the articulated system (5) is all modeled in the work image (17) either by CAD or through the global point cloud (7) or through the detail image (9). , so that there is no conflict between the various elements of the articulated system (5), the 3D camera (16), the object (2) and the working environment (3).

Also, in order to increase the operational safety of the equipment (1), in step A′, a part of the tool (4) or the articulated system (5) whose CAD modeling results are known is the object (2) or the working environment. Upon approaching the element of (3), the motion of the articulated system (5) automatically decelerates until the tool (4) or part of the articulated system (5) is at a safe distance against the obstacle. Operation stops.

Due to the above, the operator controlling the movement of the articulated system (5), and in particular the movement of the held 3D camera (16), captures a detailed image (9) of the zone of interest in which he intends to operate. Its detailed image (9) of the object (2) is automatically incorporated into the work image (17), thereby partially reconstructing the object (2) with greater accuracy and its position at the location where the manipulation is performed. You can know the details of the geometric shapes.

Given that several different types of tools (4) are available, the method selects a tool (4) for performing the operation in the simulation and for the simulation in the tool database step B. 'including.

In that case, referring to FIG. 5, the method defines the trajectory (10) of the tool (4) for the portion of the work image (17) representing the object (2), and the corresponding articulated system (5). ) and the movement of the tool (4) on the workpiece image (17) to ensure that the operation is feasible in terms of orientation, accessibility, collision-freeness, etc. (see Figure 6) .

By capturing an image (9) of details of the object (2) and incorporating it into the workpiece image (17), the operator can achieve greater accuracy in defining and positioning the trajectory (10) of the tool (4). support can be obtained.

To define the trajectory (10), the operator can position start-to-point pairs or geometric shapes selected from a library of lines, planes, masks, etc. on the work image (17) shown on the display. The computer processing system is designed to automatically calculate the trajectory (10) on the object (2). If necessary, the processing system can make manual adjustments to the trajectory (10) or the operator can draw the trajectory (10) line directly on the representation of the object (2). .

For example, looking at FIG. 5, a plane (11) is positioned with respect to a representation of the surface of an object (2) to be cut, and the computer processing system determines the trajectory (10) by the intersection of the plane (11) and its surface. ). This approach is also illustrated in FIG. 7 where the positioning of the plane (11) can be seen, and also in FIG. 8 the trajectory (10) is calculated and the plane (11) and object (2) You can see how it is plotted by the intersection of the expressions.

Once the trajectory (10) is calculated, the computer processing system enables the motion of the corresponding articulated system (5) and tool (4) to be simulated within the workpiece image (17) so that it can be manipulated. of course, depending on the operability of the trajectory (10), tool (4) and articulated system (5) tested. The operational test steps are shown in FIGS. 6 and 9, for example. The simulation results show that the motion is possible, i.e. in terms of orientation and access, and that the motion does not cause conflicts between the various elements of the equipment (1) and work environment (3). can actually perform the action. Step B is performed as many times as necessary to obtain workable operation.

In this case, the method removes the 3D camera (16), stores it in a dedicated storage box or the like, and automatically activates the articulated system (5) to connect to a preselected tool (4) that is also stored nearby. Advantageously, it includes a step C' operating at .

After connecting the tool (4), the method is an articulated system ( ) that holds the tool (4) according to its defined and validated trajectory, once the simulation shows that the manipulation is feasible. 5) is actually operated to perform the operation on the object (2).

Preferably, in step C, the equipment adjusts the speed of movement of the articulated system (5) holding the tool (4) in direct contact with the object (2) in step C to prevent damage to the tool (4). means for adjusting in real time based on direct or indirect measurement of the force applied to the For example, this measurement is obtained by measuring the current consumed by the tool (4) or motors to move the articulated system (5), or by measuring the current drawn between the articulated system (5) and the tool (4). It can be obtained by an arranged force sensor.

For ease of understanding, FIG. 10 shows the sequence of steps of the method in the form of a simplified flow chart.

Advantageously, if multiple operations have to be performed on the object (2) in succession, such as multiple cutting operations, subsequent cutting trajectories (10) are defined at mask time between previous cutting operations. or multiple trajectories (10) are simulated and recorded sequentially in an order that can be altered by the operator.

The invention is also applicable to changing environments due to new elements or potential obstructions due to objects being added or spaces being made available, for example in the case of cutting an object (2). and in which case by repeating steps A to C of the method of the invention, it is easy to obtain up-to-date images and simulations for continued manipulation of the object (2) without risk of collision. Obtainable.

The display shows equipment (1), objects (2) and various elements of the environment in CAD mode. Preferably, the articulated system (5) is represented in interactive colors. That is, the articulated system (5) is for example represented in green, and one part of it approaches the obstacle, the articulated system (5) and/or the tool (4) are defined by the collision management parameters. The part turns orange when entering the collision risk zone and red when the articulated system (5) and/or tool (4) reaches the threshold distance defined in the collision management parameters and stops moving. sequentially change to

From the above, it can be seen that the present invention provides a method for working with objects (2) and objects in a work environment (3) where the position and geometry of the work environment changes and/or is not well defined for manipulation. It is clear that it provides a method and equipment (1) for automatically and safely performing operations on objects (2).

The method can be adapted to any type of geometry or properties of the object (2). The first step of determining the environment and the object (2) in the form of a global point cloud by digitization is done by a 3D camera, which allows any type of position and geometry of the object (2) and the various elements of the environment. Adaptation to the shape is also possible. The processing time is short, and 500,000 measurement points are processed per second. Details from the environment are important and the information is continuous. Therefore, realistic rendering and modeling can be performed in real time, and the operator can easily visually check whether the reconstructed point cloud (7) is correct.

1 equipment 2 object 3 work environment 4 tool 5 articulated system 6 3D camera, 3D sensor 7 point group 8 arch 9 image 10 trajectory 11 plane 16 3D camera, 3D sensor 17 workpiece image

Claims (10)

A method for automatically performing an operation on an object (2) using a tool (4) held by a movable articulated system (5) within a working environment (3), comprising:
- capturing an image of all or part of said object (2) and said working environment (3) in the form of a global point cloud (7) using at least one 3D sensor (6A, 6B, 6C), said The image (7) is merged with an existing CAD model of said articulated system (5) and a possible existing CAD model of all or part of said working environment (3) in as-built, this merging yields a work image (17 ) is provided and further defines anti-collision parameters;
- defining the trajectory of the tool (4) with respect to the portion of the work image representing the object (2) and of the articulated system (5) and the corresponding movement of the tool in the work image (17); A step B of performing a simulation to ensure that the operation is executable;
- said articulated holding said tool (4) according to said trajectory (10) defined to perform said manipulation on said object (2), if said manipulation was determined to be feasible in step B; C. actually performing the operation of the system (5). A 3D sensor (16) held by the articulated system (5) is used to generate detailed images (9) of zones of the object (2) in a denser and more accurate point cloud than the global point cloud. and the captured image (9) of the corresponding zone of the object (2) instead of the corresponding part by the global point cloud (7) 2. A method according to claim 1, characterized in that it comprises between steps A and B a step A' of incorporating into said work image (17) immediately. It is characterized by including a step A″ of automatically operating the multi-joint system (5) to grip and connect the 3D sensor (16) stored nearby between step A and step A′. 3. The method of claim 2. Step C' of automatically operating the articulated system (5) to optionally remove and unload the 3D sensor (16) and connect it to the tool (4) stored nearby between steps B and C. 4. A method according to any one of claims 1 to 3, characterized in that it comprises 2. A method according to claim 1, characterized in that it includes a step B' of selecting a tool (4) for performing said operation in the simulation of step B and from among a tool database for the simulation before step B'. Method. In step C, the speed of movement of said articulated system (5) holding said tool (4) in direct contact with said object (2) is a direct or indirect measure of the force exerted on said tool (4). 2. The method of claim 1, wherein the adjustment is made in real-time based on. In step A′, when said 3D sensor (16) or a part of said articulated system (5) approaches said object (2) or an element of said working environment (3), the motion of said articulated system is automatically performed. and the 3D sensor (16) or part of the articulated system (5) is secured against an object (2) or an element of the work environment (3) by the anti-collision parameters defined in step A. 2. Method according to claim 1, characterized in that the movement stops automatically when the distance is reached. The trajectory of said tool (4) relative to the portion of said work image representing said object (2) in step B is determined by at least one start-end point pair or at least one predefined geometric shape, in particular a plane ( 11), defined by positioning in said workpiece image (17) either at least one line or a mask. An installation (1) for implementing the method according to claim 1, comprising an articulated system (5) and at least one 3D sensor (6, 16) connected to a computer processing system and a display, said computer processing system
- representing on a display a global point cloud (7) of all or part of said object (2) and said work environment (3), based on images captured by at least one 3D sensor (6, 16); ,
- Work image obtained by merging the global point cloud (7) of the existing CAD model of the articulated system (5) and possible existing CAD models of all or part of the working environment (3) ( 17),
- enabling the definition of the trajectory (10) of the tool (4) with respect to the portion of the work image (17) representing the object (2);
- simulating the corresponding movements of the articulated system (5) and the tool (4) in the work image (17) to ensure that the manipulation is feasible;
- said articulated system holding said tool (4) according to said defined trajectory (10) for performing said manipulation on said object (2), once said manipulation has been made feasible; A facility characterized by being designed to actually operate (5). determining in step C the speed of movement of said articulated system (5) holding said tool (4) in direct contact with said object (2) based on direct or indirect measurement of the force on said tool (4); 10. Equipment according to claim 9, characterized in that it comprises means for adjusting in real time the

Images