EP2493664B1 - Système de robot semi-autonomes polyvalent et procédé pour l'opération - Google Patents

Système de robot semi-autonomes polyvalent et procédé pour l'opération Download PDF

Info

Publication number
EP2493664B1
EP2493664B1 EP10779112.1A EP10779112A EP2493664B1 EP 2493664 B1 EP2493664 B1 EP 2493664B1 EP 10779112 A EP10779112 A EP 10779112A EP 2493664 B1 EP2493664 B1 EP 2493664B1
Authority
EP
European Patent Office
Prior art keywords
robot
scanner
track
carriage
arm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP10779112.1A
Other languages
German (de)
English (en)
Other versions
EP2493664A1 (fr
Inventor
Darren Krasny
Richard Shoaf
Jeffrey Keip
Scott Newhouse
Timothy J. Lastrapes
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Battelle Memorial Institute Inc
Original Assignee
Battelle Memorial Institute Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Battelle Memorial Institute Inc filed Critical Battelle Memorial Institute Inc
Publication of EP2493664A1 publication Critical patent/EP2493664A1/fr
Application granted granted Critical
Publication of EP2493664B1 publication Critical patent/EP2493664B1/fr
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • B25J9/1666—Avoiding collision or forbidden zones
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40183—Tele-machining
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40418—Presurgical planning, on screen indicate regions to be operated on
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40424—Online motion planning, in real time, use vision to detect workspace changes

Definitions

  • This invention relates in general to robotic systems and in particular to a semi-autonomous multi-use robot system and a method for cost effective operation of the system within an enclosed space.
  • Aircraft integral fuel tanks are internally coated to prevent corrosion, and operational aircraft are flying with tank coatings that are as much as 40 years old. Coating degradation, including peeling, swelling, puckering, and blistering, has been observed in some tanks. To prevent fuel system clogging and damage from dislodged paint chips and corrosion products, tank coatings must be examined, assessed, and repaired or replaced as they age.
  • Document US 5,416,321 discloses an apparatus which accurately maps and detects contaminants within interior surfaces is provided.
  • An optical vision system is used to accurately map the surface to be investigated or treated as a reference.
  • Analytical probes are used to sample the mapped surface to detect inorganic and organic contaminants in situ.
  • Automated controls which manipulate the analytical probes and high speed analytical equipment, such as a high speed gas chromatograph, are optionally used to take large numbers of samples quickly and remotely.
  • the optical vision system tracks the position of the analytical probes, providing an accurate map of the location of the contaminants.
  • This invention relates to a method for operation of a semi-autonomous multi use robot system within an enclosed space.
  • the present invention provides a cost-effective means for employing industrial robotic technology in the maintenance and repair of legacy systems including aircraft integral fuel tanks.
  • the present invention contemplates a new and novel approach to overcome the lack of digital data, the requirement for fixed tooling, and the need to generate scripted path planning algorithms.
  • the present invention contemplates a robot system that acquires the geometrical data to define the required "structured environment" in order to perform operations without the need for fixed tooling, i.e., using "virtual tooling".
  • the invention contemplates using a laser scanning device to gather digital information that will be used to define the required structured environment. Also, the invention contemplates combining the data gathered from the laser scanning device with the "user-defined" geometric data that defines the robotic system, to calculate the precise relative location between the robotic system and the work piece. This relative location is updated as the robotic system moves, which ensures that the robot "understands" its environment at all times for collision free operation without fixed tooling.
  • the invention contemplates the use of a Rapidly-exploring Random Tree (RRT) motion planning algorithm to control movement of the robot within an enclosed area.
  • RRT Rapidly-exploring Random Tree
  • the present invention is unlikely to completely eliminate human intervention, it should sharply reduce the amount of time that personnel must spend inside aircraft fuel tanks and improve tank maintenance productivity, efficiency, and effectiveness.
  • the present invention will enable aircraft maintenance personnel to remain outside a fuel tank, guiding the robotic system using an integrated positioning and vision system and remote controls.
  • the MURS 10 includes a compact industrial robot 12 having an arm 14 capable of providing six degrees of freedom comprising movement along three orthogonal axes (X, Y and Z) as well as rotation about three axes (pitch, roll and yaw) to a device mounted upon the operational end 15 of the arm.
  • the robot 12 is mounted upon a movable carriage 16 that is carried upon a sectional track 18.
  • the sectional track 18 includes two parallel rails for guiding the carriage 16.
  • the movable carriage 16 is shown in Fig.
  • the carriage 16 includes a servo motor (not shown) that drives a gear.
  • the gear engages a rack (not shown) on the track 18 to move the carriage 16 along the track.
  • the servo motor is reversible to allow bi-directional movement of the robot 12 along the track 18.
  • Use of sectional track 18 allows tailoring the length of the MURS installation to fit a specific application by combining two to four, or more, sections of track.
  • a section of the track is illustrated in Fig. 3 .
  • a calibration plate (not shown) is incorporated into the end rail section for set-up calibration of the MURS.
  • the track 18 is held in place by a plurality of support clamps 20 (one shown) that are secured to frame members of an aircraft's internal wing structure.
  • the invention contemplates two different support clamps.
  • a first support clamp 24, which is shown in Fig. 4 pivots with three degrees of freedom to allow an installer to adjust the track rails and afford easy alignment with the second clamp.
  • the second clamp 26, which is shown in Fig. 5 pivots with less freedom and stabilizes the rail assembly when tightened into position.
  • These two clamps facilitate installation and set-up of the MURS within a confined space.
  • the clamps were designed so that each can attached to any section of the rail at any point along the length of the section. This design eliminates a requirement of a third, center rail support.
  • a tool mounting bracket 28 is attached to the operational end 15 of the robot arm 14 and carries a tool which is shown as a three-dimensional (3D) scanner (30).
  • the invention contemplates using a commercially available 3D scanner, such as, for example, Mesa Imaging's Swiss Ranger 4000 (SR4000, depicted in Fig. 17 ), which has been integrated with the system.
  • This particular 3D scanner is smaller, lighter, and faster than some other commercially available 3D laser scanners, and has a frame rate of over 50 Hz.
  • the SR4000 has much lower resolution and accuracy than other available scanners, but the lower resolution and accuracy are not critical for acquiring the surrounding environment for collision-free path planning.
  • other 3D scanners may be utilized with the MURS 10 if more accuracy is required, including a 3D laser scanner from 3DDigital Corp. which has a depth resolution of 0.007 inches (0.18mm) and as shown in Fig. 9 .
  • Fig. 5 illustrates the mounting bracket 28 in greater detail.
  • Multiple considerations drove the design of the mounting for the 3D scanner. First of all, rough handling can knock the internal optics in the 3D scanner out of alignment, so the scanner mounting bracket 28 had to enable the operator to install the scanner and be reasonably sure that it was properly positioned before letting go of it. Furthermore, the mounting bracket 28 also had to provide support the 3D scanner because the light gauge material of the scanner housing may flex and bend as the robot moved the scanner from vertical to horizontal positions and back. Finally, the 3D scanner mounting bracket 28 had to prevent improper installation since a "rotated" orientation could prove detrimental to MURS operation.
  • the mounting bracket 28 was designed as a Manfrotto rapid connect camera mount, such as a COTS mounting system; however, it will be appreciated that the invention also may be practiced utilizing other mounting hardware.
  • This mount designed for quick mounting of a camera onto a tripod, employs three contact faces for fast, repeatable connection and tool-less operation.
  • a clamping arm engages a passive portion of the mounting apparatus and snaps into position with a notable and definite click.
  • the mounting bracket 28 includes a modified hexagonal design of the passive mount as a simplified tab and slot to "key" the installation, thus ensuring that the 3D scanner 30 can only be installed in the correct position.
  • the tab also captures the scanner when the installer presses it into position. If the latching mechanism fails to snap into position, the scanner will sit at a noticeable "tilt" to indicate to the installer that it is not properly secured, yet the tab assures that the scanner does not fall if it is inadvertently released.
  • a mounting plate (not shown) attached to the bottom of the 3D scanner 30 provides additional rigidity to the scanner.
  • the mounting plate also holds a COTS digital camera (not shown) to provide the digital picture for overlay of the scan files. Accurate positioning of the camera is necessary for proper registration of the overlay to the scanned image and the mounting plate provides the required alignment.
  • the invention also contemplates including lights on the robot to illuminate the work place for the digital camera.
  • the present invention also contemplates that the 3D scanner may be either replaced by a job specific tool (not shown) or may carry a job specific tool.
  • the tool mount is perpendicular to the robot's J6 axis of rotation, thus providing an additional degree of freedom for scanner positioning.
  • An umbilical cable 32 connects the robot 12 and carriage 16 to an operator control station, which is shown generally at 34 in Fig. 1 .
  • fiber optic cable is utilized instead of typical twisted pair cable because the twisted pair cable may pick up noise from the carriage drive motor that could drown out communications with the scanner.
  • a cable trough (not shown) is included at the base of the robot 12 to manage the loose portion of cable 32. This trough provides a workable solution, acting to control the lower portion of the cable 32 to prevent it from becoming entangled in the carriage drive motor as the robot travels to the extremes of its motion. Mounting the trough at the proper working level requires a "riser” that is designed to provide a double duty as a handle for carrying the robot. Clips (not shown) are provided to hold the cable 32 onto the robot and are installed along the robot to reduce the chance of snagging the cable on an external structure.
  • the operator control station 34 includes a personal computer 36 that is responsive to a control algorithm stored in a computer hard drive 38 to control the MURS 10.
  • Operator interface with the computer 36 is shown as being provided with a display screen 40, a keyboard 42 and a mouse 44. It will be appreciated that the operator interface may utilize different components than are shown. For example, a touch screen display may replace the display screen, keyboard and mouse. Additionally, a dedicated processor may be utilized in place of the personal computer 36.
  • the umbilical cable 32 is connected to the computer 26 to provide communication with the robot 12 and carriage 16. The invention contemplates that the track, carriage and robot would be installed within the aircraft fuel tank by means of an access hatch.
  • the umbilical cable 32 would pass through the access hatch (not shown) to allow placement of the operator station 34 outside of the fuel tank.
  • the invention also contemplates optional use of transmitters and receivers (not shown) in place of the umbilical cable to allow placement of the operator station at a location remote from the aircraft.
  • MURS Geometry Manager which comprises an algorithm that controls system movements, numerous user interface enhancements, integration of the 3D scanner and the digital camera, and motion planning and collision avoidance strategies.
  • the digital camera provides color imagery used in a photographic overlay.
  • Common 3D graphics texturing techniques integrate 2D color information from the digital camera with the 3D geometry obtained by the scanner to provide a "photo-textured" 3D model, which is displayed on the control station screen 40.
  • FIG. 7 A flow chart for the MURS MGM control algorithm is shown in Fig. 7 .
  • the track 18, carriage 16 and robot 12 are installed within the aircraft fuel tank being serviced.
  • one or more job specific tools may be attached to the end 15 of the robot arm14.
  • an infrared camera may be utilized to "see through” a coating.
  • a conventional handheld or robotic laser head may be mounted upon the end of the robot arm for coating and sealant removal from the fuel tank surfaces.
  • tools for drilling, painting, paint removal, inspection, polishing, grinding, grasping and/or marking may be attached to the end of the robot arm.
  • the algorithm is entered via block 50 and proceeds to functional block 52 where a specific job is activated by the operator requesting stored data for a specific job.
  • the algorithm then advances to decision block 54 where it is determined whether or not data for the job is available. If the data is not available, the algorithm transfers to functional block 56 where data for the specific job is developed and saved, as will be described below. The algorithm then advances to functional block 58 to carry out the specific job. Returning to decision block 54, if the data for the specific job is available, the algorithm transfers directly to functional block 58 to carry out the specific job.
  • a "mixed-reality" 3D display of the MGM, as illustrated in Fig. 8 is displayed upon the screen 40 and provides the primary way for the operator to self-orient to the position and pose of the robot 12 within the wing tank.
  • the system 10 includes an operator-controlled "Virtual camera” that can be set at orthogonal view (front, back, left side, right side, top, and bottom) presets, or dragged around to arbitrary viewpoints.
  • the virtual camera positions may be activated by means of the preset buttons shown at the bottom of the screen in Fig. 8 .
  • Various operator prompts are also shown in Fig. 8 along the right side of the display.
  • Wing tank walls are acquired by the 3D scanner and rendered in perspective on the display. Similar to a cut-away illustration, wing tank walls which are between MURS and the virtual camera position are not rendered, so that the virtual camera need not be confined to views within the volume of the wing tank.
  • the 3D display shown in Fig. 8 is fundamental to tool positioning where an operator clicks in this view on the surface that the robotic tool will point at, as illustrated in Fig. 9 .
  • the destination position may lie along the tool-pointing axis within a range of distances from the work surface. MURS can support other tools which require a precise offset.
  • an algorithm for developing data for a specific job is illustrated in Fig. 10 .
  • the algorithm of Fig. 10 may also be utilized in a standalone manner to initially develop data for a new job.
  • the algorithm is entered through block 70 and proceeds to functional block 72 where carriage homing is carried out. Since two, three, four, or more, rail sections of any length may be used in a MURS installation, the software needs to discover its permitted travel length along the rail. Accordingly, in functional block 72, the operator commands MURS carriage homing after installation, and the carriage moves end-to-end on the rail, recording a travel length that is stored in the software. This operation has been optimized with the system automatically determining the number of rails in use and can then reliably translate the system along the rail with high precision and repeatability.
  • the MURS software advances to functional block 74 and prompts the operator to perform a Scanner Calibration operation to validate the installation and configuration of the scanner. During this check, the robot is commanded to point the scanner at a known point on the rail. The algorithm then advances to decision block 76 where the system compares the result obtained from the scanner to the actual location of the known point. If the known point appears at the correct 3D location in each dissimilar scanner orientation, the system is functioning properly and the algorithm transfers to directly to functional block 78. If this test fails, the system must be validated before use and the algorithm advances to functional block 80.
  • the scanner is validated.
  • a least squares regression algorithm uses the system check data to determine the precise true position and orientation of the reference point of the 3D scanner with respect to the robot arm. This validation process will normally be performed once in a controlled environment outside the wing tank and then repeated if the scanner validation test fails. Once validation is completed, the algorithm advances to functional block 78.
  • the operator will typically initiate a Survey Scan.
  • This triggers a sequence of scans of the surrounding tank elements that builds up the surrounding geometric volume in the software.
  • the Survey Scan commands a sequence of robotic moves resulting in varied 3D scanner positions, allowing the system to "have a look around" its bay in order to acquire the surrounding environment geometry.
  • the software operates the robot within a restricted volume to minimize risk of collision against tank surfaces not yet acquired.
  • the laser scanner provides the system with direct knowledge of its surroundings. To compensate for its limited field of view, the scanner takes multiple scans and integrates them to detail the tank volume.
  • laser scanner integration consists of taking the scanner-relative 3D points and translating them into MURS coordinates by applying the known robot end-effector position and orientation, as defined by robotic system angles and rail positions.
  • the acquired array of 3D points is also formed into a mesh.
  • the operator monitors this automated process through the in-tank overview cameras and the virtual display.
  • the virtual display included as part of the MURS MGM, renders the current position of the robot arm and the current rail position.
  • the virtual display renders the wing tank meshes acquired during the Survey Scan.
  • the invention contemplates that all meshes and photographs developed by the MURS during the Survey Scan are stored on the hard drive of the control station personal computer (PC) as shown by the dashed line extending from functional block 78 to block 84. This historical information can be recalled and displayed on the PC at a later date. Historical meshes and photographs can also be referenced within the optional robotic simulation mode, affording system training even when the robotic elements are not installed.
  • PC personal computer
  • the algorithm proceeds to decision block 82, where the operator may choose to operate the MURS to complete a task or exit the application. If the operator chooses to not operate the MURS, the algorithm exits through block 60. However, if the operator chooses to operate the MURS, the algorithm transfers to functional block 86, where the operator may manually change the end-effector for the desired task, if necessary. If the operator changes the end-effector, the operator must also choose the new end-effector from a list of pre-populated choices within the MGM software to ensure that the Motion Planning (MP) module computes collision-free paths with the correct end-effector geometry. The algorithm then proceeds to functional block 88.
  • MP Motion Planning
  • the operator selects a discrete location on the acquired meshes in the MGM software to position the end-effector for the given task. This is done by pointing and clicking the mouse on the desired location of the rendered mesh. This is referred to as "point-and-click robotic positioning," which is one of the benefits afforded by the present invention.
  • the algorithm then advances to functional block 90.
  • the MGM software computes a series of motions required to move the end-effector to a desired standoff distance from the surface of the tank.
  • the target direction of the end-effector will be normal to the surface of the tank with the end-effector rolled to one of several acceptable angles that is configurable by the user.
  • the algorithm then advances to decision block 92.
  • decision block 92 the algorithm determines if the series of motions can be executed based on whether a collision-free solution was found. If no collision-free solution was found, the algorithm transfers to decision block 82, where the operator can choose a different point or exit the application. If, in decision block 92, it is determined that a collision-free solution was found, the algorithm transfers to functional block 94, where the actual robot arm and carriage execute the series of points found from the MP module. This positions the tool at the desired location and offset from the work surface. At this point, the actual end-effector operation will be enacted to complete the task in functional block 58. This may consist of inspection, polishing, paint stripping, recoating, or other actions. The algorithm then returns to decision block 82, in which the operator may continue or exit the application.
  • the MURS software must not only command and sequence the movements of the MURS, but must also ensure that the movements do not cause collisions between the robot and the wing tank.
  • the collision avoidance approach utilized with MURS is essentially a form of Look-Ahead Verification.
  • the MURS software calculates a series of target waypoints for the end-effector's travel and then simulates the joint rotations required to traverse the path. The simulation is sampled at regular intervals within the path to ensure that no collision between any two parts of the robot and between any part of the robot and any part of the wing tank is possible during the motion. If the desired path would not cause any collisions, it is sent to the robot and the carriage controllers to be executed.
  • the MP module is activated in functional block 90 and uses random-search with a tree-structured approach to efficiently find paths between a start and goal location in joint space.
  • RRT Rapidly-Exploring Random Tree
  • the MP module is activated in functional block 90 and uses random-search with a tree-structured approach to efficiently find paths between a start and goal location in joint space.
  • the corresponding robot configuration is checked for collision using a bounding box approach.
  • Robot elements are contained in virtual boxes that are checked against each other and against the surrounding tank geometry for intersection. If any robot bounding box indicates a collision, with either another robot bounding box or the surrounding tank geometry, then the collision check fails and the motion path is rejected.
  • Pre-collision avoidance ensures that a chosen tool path will not drive the robot into the structure, either at the end of the movement or during the repositioning.
  • a greedy heuristic is then added to encourage optimal straight-line paths through joint space whenever possible.
  • the robot joint space is partitioned into the robot arm joint space and the rail translational space such that motion can occur in only one subspace at a time. This precludes having to coordinate arm and rail motion simultaneously if not required for the given task. Tests of the MP algorithm have demonstrated that complex, non-straight-line trajectories into highly confined spaces are now possible.
  • a refined and enhanced Inverse Kinematics (IK) algorithm determines candidate goal positions for the robot.
  • the IK algorithm module is utilized so that a number of different IK configurations are generated for a given tank inspection point. These configurations include alternative scanner orientations and offset distances from the virtual tank wall in order to increase the number of possible robot arm/rail position configurations and maximize the probability of finding a collision-free path to at least one.
  • This set of goal positions is ordered in terms of "distance" in joint space from the current configuration, i.e., in terms of how similar each configuration is to the current one.
  • the ordered set of possible IK solutions is then passed to the MP module, which attempts to find a path to at least one. During the search, intermediate preset positions are considered in order to facilitate the search.
  • the MP module is capable of finding solutions to extremely challenging poses, although search time is obviously an issue.
  • search time is obviously an issue.
  • the maximum number of search steps for each path is limited while the number of different possible routes is increased.
  • this allows the MP module to quickly find solutions to most locations inside the tank, with simple moves taking a few seconds and relatively complex moves taking less than a minute.
  • the MP module In order to find safe paths, the MP module must evaluate potential robot configurations against its current estimate of the tank working volume. Before the Survey Scan is completed, this volume is initially computed based on certain minimum clearances from the installed rail sections to provide the robot with just enough space to translate along the rail in a tucked position and rotate the scanning head without colliding with the tank. The installer must guarantee that this "default" volume is safe for operation by ensuring that the robot can translate along the rail in a tucked position without colliding with the tank. However, this volume is generally too small to permit arbitrary positioning of the system for inspecting different locations throughout the bay. Consequently, after the initial overview scan, the scanned data from the tank is used to compute a larger, better fitting estimate of the tank volume.
  • Evolutionary search is a stochastic optimization approach based on artificial evolution where candidate solutions from a population are evaluated against a fitness function. Depending on the variant of the evolutionary algorithm, the best solutions "survive” and are used to generate new solutions during subsequent trials, or generations. Evolutionary search has been well-accepted in a variety of fields because of its ability to efficiently generate acceptable solutions in large-dimensional, non-linear, and unknown search spaces.
  • an evolutionary search is used to find the largest tank volume that fits snugly inside the scanned vertices without any scanned vertex actually piercing the volume.
  • the fitness function is constructed such that there is a heavy penalty for intrusion of scanned vertices into the working volume. This encourages the algorithm to quickly find solutions that are at least acceptable, with no intruding vertices. After a number of generations, the fit is improved as the fitness function rewards larger volumes.
  • a suitable volume can be computed in twenty generations using 100 individuals in the population, which requires two to three minutes of computation time on the MURS PC.
  • the MP module is integrated with the action-related moves and scripts in the MGM algorithm in order to prevent dangerous, unchecked robot moves.
  • unsafe i.e., not collision checked
  • moves are still permitted-but are intended for advanced, knowledgeable operators only. These moves are clearly marked as not collision-checked and thoroughly documented in the MURS manual.
  • the robot umbilical is not rigid, it does not remain consistently within a reasonably-sized bounding box, and it is thus not amenable to Pre-Collision Avoidance calculations.
  • responsibility for umbilical clearance remains on the system operator, who sees the umbilical through the operator station display of an in-tank overview video camera.
  • MURS software's choice of robotic position before starting the move.
  • the MURS software characterizes the direction of rail travel as the X-axis, with the X-Y plane horizontal and the Z-axis pointing up.
  • the MURS software considers a wing tank to be "installed" around itself, and wing tank surfaces thus are referenced to the XYZ coordinate system defined by the rail.
  • translating MURS installation-specific coordinates to aircraft-referenced coordinates would be a separate effort that would likely require operator input or selection from a database of places where the MURS could be installed on the aircraft.
  • MURS graphical point-and-click tool positioning markedly simplifies operation of the robotic system.
  • the operator need not be concerned with choosing robotic arm angles or determining the precise XYZ coordinates of the end-effector. Instead, the operator chooses the desired location and allows the system to automatically determine how to point a tool at the desired location and how to find a collision-free path from its current pose to the desired pose.
  • the present invention also contemplates an alternate method for collision-checking the surrounding environment.
  • the environment is scanned and a parameterized volume fitted to the scanned data. This volume was then used to perform collision checking for path planning. Although this approach was efficient, it may be limited to applications in which a similar volume could be used to describe the environment, namely, confined space applications.
  • scanned data is voxelized by boxing the data into 3D volumetric pixels, called voxels. This is the equivalent of 3D rasterization and produces a 3D occupancy grid of the scanned data.
  • the alternate method provides a much more general approach, which allows the robot system to be operated in almost any environment, including both confined spaces and non-confined spaces. Additionally, the alternate method utilizes a Graphics Processor Unit (GPU), which supports vector operations and greatly reduces computation times, for collision avoidance processing.
  • GPU Graphics Processor Unit
  • HPC High-Performance Computing
  • Multiple computing nodes are connected via Ethernet and each computing node includes multi-core processors with multiple GPUs.
  • the path planner generates many potential paths to different candidate goal configurations and distributes a set of these paths to each computing node.
  • Each computing node spawns multiple threads to find potential paths using available computing cores and GPUs.
  • Each computing node then returns its results to a head node. As soon as a solution is found, the head node notifies all computing nodes to stop processing.
  • One of the major additions to the original MURS MGM software is the ability to generate collision-free continuous trajectories for contour following. Under the original MURS software, only paths to a discrete point could be found. Multiple points could be selected on the surface, although the motion planner would find individual paths to each point, which would not, in general, produce the expected straight-line motion of the end-effector in Cartesian space. In order to produce continuous trajectories for contour following, it is necessary to find collision-free candidate "seed" solutions. Then, using standard pseudo-inverse Jacobian control, compute the required joint angles and rates to achieve the desired path of the end-effector while maintaining the proper orientation and path speed, i.e., commonly known as resolved motion rate control.
  • each step in the trajectory must be collision-checked to ensure that no part of the robot contacts any other part of the robot, i.e., experiences a self-collision, and that no part of the robot contacts the surroundings, i.e., experiences an external collision.
  • Using a large number of seed solutions increases the probability of finding acceptable solutions that meet the following criteria:
  • An optional mode for creating trajectories is a "free-hand" approach, where the user can click the mouse over the desired work surface to "draw” the desired trajectory on the work surface, in a manner similar to other graphics editing software, as illustrated in Fig. 17 .
  • the invention also contemplates utilization of the free-hand approach to create different shapes such as, for example, arcs, ellipses, circles, etc., within the defined region. If an ellipse is desired, the user would click and drag to create a bounding rectangle and select an ellipse to fill it. This feature would be useful for manually created trajectories for avoiding holes or other complex surface features.
  • the invention also contemplates facilitating motion planning by synchronization of the rail and the robot arm movements.
  • an upgraded robot controller with extended axis control capability (a common feature among many robot manufacturers)
  • both the arm and the rail can be moved synchronously to permit contour following along longer surfaces that lie parallel to the rail.
  • the extra Degree Of Freedom (DOF) with the extended axis control capability provides redundancy in the system, which allows for more robust trajectory planning that includes standard singularity and joint limit avoidance.

Claims (4)

  1. Procédé pour exploiter un système de robot comprenant les étapes consistant à :
    (a) fournir un système de robot (12) qui comprend :
    un bras (12) ayant plusieurs degrés de liberté de mouvement ;
    un chariot mobile (16) pour transporter le robot ;
    une piste, ledit chariot mobile (16) transportant ledit robot (12) sur ladite piste (18) ;
    un dispositif de balayage (30) monté sur une extrémité du bras de robot ; et
    un poste d'opérateur (34) situé à distance du bras de robot, le poste d'opérateur comprenant un ordinateur (36) et un logiciel associé, dans lequel le poste d'opérateur est opérationnel pour commander un déplacement du bras de robot et du chariot ;
    (b) définir un volume de travail sûr d'un environnement d'exploitation entourant le robot sur la base de jeux minimums par rapport à la piste, le volume de travail sûr offrant au robot un espace suffisant pour se déplacer en translation le long de la piste dans une position repliée sans entrer en collision avec l'environnement de fonctionnement ;
    (c) collecter des données concernant l'environnement d'exploitation entourant le robot à l'aide du dispositif de balayage, ledit robot se déplaçant en translation le long de ladite piste dans la position repliée tout en collectant les données environnementales, de sorte que le robot se trouve dans le volume de travail sûr de l'environnement d'exploitation et n'entre pas en collision avec toute partie de l'environnement opérationnel ;
    (d) utiliser les données numérisées pour définir des surfaces dans l'environnement d'exploitation du robot, les surfaces définies constituant une meilleure estimation du volume de l'environnement d'exploitation entourant le robot ;
    (e) sélectionner une position souhaitée pour le robot dans l'environnement d'exploitation ;
    (f) déterminer un trajet pour déplacer le robot vers la position souhaitée sélectionnée à l'étape (e), qui évite toute collision entre le robot et les surfaces définies à l'étape (d) ; et
    (g) déplacer le chariot de robot jusqu'à la position sélectionnée à l'étape (e) via le trajet déterminé à l'étape (f) .
  2. Procédé selon la revendication 1, dans lequel le poste opérateur prévu à l'étape (a) comprend un écran d'affichage en communication avec une souris, l'écran d'affichage affichant un curseur commandé par la souris, et dans lequel en outre l'étape (d) comprend l'utilisation de la souris pour positionner un curseur sur l'écran d'affichage dans la position souhaitée pour le robot, puis l'enfoncement d'un bouton sur la souris pour informer le poste d'opérateur de la position souhaitée pour le robot.
  3. Procédé selon la revendication 1, dans lequel l'étape (e) comprend également la détermination d'un déploiement souhaité du bras de robot dans une position souhaitée sans provoquer de collision entre le bras de robot et les surfaces définies à l'étape (c) et en outre dans lequel l'étape (f) inclut le déploiement du bras de robot tel que déterminé à l'étape (e).
  4. Procédé selon la revendication 3, dans lequel le robot effectue une opération souhaitée une fois que le chariot et le bras du robot se sont déplacés vers les positions sélectionnées à l'étape (d).
EP10779112.1A 2009-10-27 2010-10-27 Système de robot semi-autonomes polyvalent et procédé pour l'opération Not-in-force EP2493664B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US25525709P 2009-10-27 2009-10-27
PCT/US2010/054225 WO2011056633A1 (fr) 2009-10-27 2010-10-27 Système robotisé polyvalent semi-autonome et procédé de fonctionnement

Publications (2)

Publication Number Publication Date
EP2493664A1 EP2493664A1 (fr) 2012-09-05
EP2493664B1 true EP2493664B1 (fr) 2019-02-20

Family

ID=43530266

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10779112.1A Not-in-force EP2493664B1 (fr) 2009-10-27 2010-10-27 Système de robot semi-autonomes polyvalent et procédé pour l'opération

Country Status (3)

Country Link
US (2) US9643316B2 (fr)
EP (1) EP2493664B1 (fr)
WO (1) WO2011056633A1 (fr)

Families Citing this family (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011056633A1 (fr) 2009-10-27 2011-05-12 Battelle Memorial Institute Système robotisé polyvalent semi-autonome et procédé de fonctionnement
US10475240B2 (en) * 2010-11-19 2019-11-12 Fanuc Robotics America Corporation System, method, and apparatus to display three-dimensional robotic workcell data
US10040141B2 (en) 2013-05-23 2018-08-07 Crc-Evans Pipeline International, Inc. Laser controlled internal welding machine for a pipeline
US10480862B2 (en) 2013-05-23 2019-11-19 Crc-Evans Pipeline International, Inc. Systems and methods for use in welding pipe segments of a pipeline
US11767934B2 (en) 2013-05-23 2023-09-26 Crc-Evans Pipeline International, Inc. Internally welded pipes
US9821415B2 (en) 2014-03-28 2017-11-21 Crc-Evans Pipeline International, Inc. Internal pipeline cooler
US10589371B2 (en) 2013-05-23 2020-03-17 Crc-Evans Pipeline International, Inc. Rotating welding system and methods
US10695876B2 (en) 2013-05-23 2020-06-30 Crc-Evans Pipeline International, Inc. Self-powered welding systems and methods
JP5845211B2 (ja) * 2013-06-24 2016-01-20 キヤノン株式会社 画像処理装置、画像処理方法
US9617692B2 (en) * 2013-10-10 2017-04-11 Racine Railroad Products, Inc. Tie plugging machine and method
JP2016536150A (ja) * 2013-10-11 2016-11-24 リシンク ロボティクス インコーポレイテッド 向上した正確度を伴うロボット配置および操作
EP3102914B1 (fr) * 2014-02-03 2018-08-08 Husqvarna AB Détection d'obstacles pour un outil de travail robotisé
US9841749B2 (en) 2014-04-01 2017-12-12 Bot & Dolly, Llc Runtime controller for robotic manufacturing system
US9701018B2 (en) 2014-04-01 2017-07-11 Bot & Dolly, Llc Software interface for authoring robotic manufacturing process
US9920488B2 (en) 2014-04-08 2018-03-20 Encore Rail Systems, Inc. Railroad tie plugging system
US9555545B2 (en) 2014-05-21 2017-01-31 Bot & Dolly, Llc Systems and methods for time-based parallel robotic operation
US9278449B1 (en) 2014-05-21 2016-03-08 Bot & Dolly, Llc Closed-loop control system for robotic operation
US9308647B2 (en) * 2014-06-03 2016-04-12 Bot & Dolly, Llc Systems and methods for instructing robotic operation
BR112017003933A2 (pt) 2014-08-29 2018-03-06 Crc evans pipeline int inc método e sistema para solda
DE102014118001A1 (de) * 2014-12-05 2016-06-09 Broetje-Automation Gmbh Verfahren zur Bewegungssimulation eines Manipulators
EP3250347B1 (fr) 2015-01-26 2023-11-08 Duke University Matériel de planification de mouvements de robot spécialisé et procédés de fabrication et d'utilisation de celui-ci
WO2016172718A1 (fr) * 2015-04-24 2016-10-27 Abb Technology Ltd. Système et procédé de télécommande à distance à l'aide d'une scène 3d reconstruite
GB2538231A (en) * 2015-05-07 2016-11-16 Airbus Operations Ltd Method and apparatus for aircraft inspection
US9682476B1 (en) * 2015-05-28 2017-06-20 X Development Llc Selecting robot poses to account for cost
US9895803B1 (en) 2015-06-19 2018-02-20 X Development Llc Calculating trajectory corridor for robot end effector
US20170068863A1 (en) * 2015-09-04 2017-03-09 Qualcomm Incorporated Occupancy detection using computer vision
US9751211B1 (en) 2015-10-08 2017-09-05 Google Inc. Smart robot part
US9987745B1 (en) 2016-04-01 2018-06-05 Boston Dynamics, Inc. Execution of robotic tasks
US10065311B1 (en) * 2016-06-08 2018-09-04 X Development Llc Singularity handling for robot jogging
WO2017214581A1 (fr) 2016-06-10 2017-12-14 Duke University Planification de déplacement pour véhicules autonomes et processeurs reconfigurables de planification de déplacement
US10272572B2 (en) * 2016-06-10 2019-04-30 The Boeing Company Remotely controlling robotic platforms based on multi-modal sensory data
US11458571B2 (en) 2016-07-01 2022-10-04 Crc-Evans Pipeline International, Inc. Systems and methods for use in welding pipe segments of a pipeline
US10668577B2 (en) 2016-09-01 2020-06-02 Crc-Evans Pipeline International Inc. Cooling ring
US10108194B1 (en) * 2016-09-02 2018-10-23 X Development Llc Object placement verification
US10131053B1 (en) * 2016-09-14 2018-11-20 X Development Llc Real time robot collision avoidance
US10351189B2 (en) 2016-12-13 2019-07-16 Boston Dynamics, Inc. Whole body manipulation on a legged robot using dynamic balance
US11351673B2 (en) 2017-03-06 2022-06-07 Miso Robotics, Inc. Robotic sled-enhanced food preparation system and related methods
WO2018165038A1 (fr) 2017-03-06 2018-09-13 Miso Robotics, Inc. Système de préparation d'aliments à réalité augmentée et procédés associés
US10766140B2 (en) 2017-04-13 2020-09-08 Battelle Memorial Institute Teach mode collision avoidance system and method for industrial robotic manipulators
EP3651943B1 (fr) * 2017-07-10 2024-04-24 Hypertherm, Inc. Procédés et systèmes mis en uvre par ordinateur de génération de trajectoires d'outil robotique de traitement de matériau
US10578453B2 (en) * 2017-07-14 2020-03-03 Rosemount Aerospace Inc. Render-based trajectory planning
JP6950347B2 (ja) * 2017-08-10 2021-10-13 オムロン株式会社 情報処理装置、情報処理方法およびプログラム
CN108364340A (zh) * 2017-12-03 2018-08-03 广东康云多维视觉智能科技有限公司 同步空间扫描的方法及系统
WO2019139815A1 (fr) 2018-01-12 2019-07-18 Duke University Appareil, procédé et article pour faciliter la planification de mouvement d'un véhicule autonome dans un environnement comportant des objets dynamiques
EP3518059B1 (fr) * 2018-01-24 2020-04-01 Siemens Aktiengesellschaft Procédé d'aide à l'utilisateur assistée par ordinateur lors de la mise en service d'un planificateur de déplacement pour une machine
TWI822729B (zh) 2018-02-06 2023-11-21 美商即時機器人股份有限公司 用於儲存一離散環境於一或多個處理器之一機器人之運動規劃及其改良操作之方法及設備
ES2928250T3 (es) 2018-03-21 2022-11-16 Realtime Robotics Inc Planificación del movimiento de un robot para diversos entornos y tareas y mejora del funcionamiento del mismo
US11879958B2 (en) 2018-06-06 2024-01-23 Honeywell International Inc. System and method for using an industrial manipulator for atmospheric characterization lidar optics positioning
US11407111B2 (en) * 2018-06-27 2022-08-09 Abb Schweiz Ag Method and system to generate a 3D model for a robot scene
US11285607B2 (en) * 2018-07-13 2022-03-29 Massachusetts Institute Of Technology Systems and methods for distributed training and management of AI-powered robots using teleoperation via virtual spaces
US11192258B2 (en) * 2018-08-10 2021-12-07 Miso Robotics, Inc. Robotic kitchen assistant for frying including agitator assembly for shaking utensil
US11577401B2 (en) 2018-11-07 2023-02-14 Miso Robotics, Inc. Modular robotic food preparation system and related methods
IT201900000995A1 (it) * 2019-01-23 2020-07-23 Nuovo Pignone Tecnologie Srl Apparecchiatura robotica industriale con generazione di percorso di lavorazione migliorata e metodo per azionare un' apparecchiatura robotica industriale secondo un percorso di lavorazione migliorato
GB2580961A (en) * 2019-02-01 2020-08-05 Granta Group Ltd Modular assembly components
JP6656627B1 (ja) * 2019-02-13 2020-03-04 日本金銭機械株式会社 紙幣取扱いシステム
JP7479064B2 (ja) 2019-06-03 2024-05-08 リアルタイム ロボティクス， インコーポレーテッド 動的障害物を有する環境における動作計画を容易にする装置、方法及び物品
US11255053B2 (en) * 2019-08-19 2022-02-22 Railroad Solutions, Inc. Composition, device, and method for maintaining wood railroad ties
EP3993963A4 (fr) 2019-08-23 2022-08-24 Realtime Robotics, Inc. Planification de mouvement destinée à des robots afin d'optimiser la vitesse tout en conservant des limites d'accélération et de secousse
US11234779B2 (en) * 2019-09-10 2022-02-01 Verb Surgical. Inc. Handheld user interface device for a surgical robot
US11958183B2 (en) 2019-09-19 2024-04-16 The Research Foundation For The State University Of New York Negotiation-based human-robot collaboration via augmented reality
US11351669B2 (en) 2019-10-29 2022-06-07 Kyndryl, Inc. Robotic management for optimizing a number of robots
TW202146189A (zh) 2020-01-22 2021-12-16 美商即時機器人股份有限公司 於多機器人操作環境中之機器人之建置
US11724387B2 (en) 2020-04-03 2023-08-15 Fanuc Corporation Fast robot motion optimization with distance field
TWI776214B (zh) * 2020-08-24 2022-09-01 國家中山科學研究院 機器手臂控制系統及其控制方法
US11882964B2 (en) * 2020-10-06 2024-01-30 Lab2Fab. LLC Automated frying system
CN113256708B (zh) * 2021-04-07 2022-09-20 深圳群宾精密工业有限公司 一种理论3d空间与实际机器人空间的标定方法
US20220346598A1 (en) 2021-05-01 2022-11-03 Miso Robotics, Inc. Automated bin system for accepting food items in robotic kitchen workspace
CN113290548B (zh) * 2021-05-06 2023-05-30 湖南工业职业技术学院 一种便于拆卸和安装的机械制造用关节臂
JP2023068268A (ja) * 2021-11-02 2023-05-17 株式会社日立製作所 軌道計画装置、軌道計画方法および軌道計画プログラム

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008124713A2 (fr) * 2007-04-09 2008-10-16 Sam Stathis Système et procédé capables de naviguer et/ou de mapper tout espace multidimensionnel

Family Cites Families (61)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5017084A (en) * 1954-07-28 1991-05-21 Lemelson Jerome H Automatic manipulation system and method
US4663726A (en) 1985-04-15 1987-05-05 General Electric Co. Robot control utilizing cubic spline interpolation
US4644237A (en) 1985-10-17 1987-02-17 International Business Machines Corp. Collision avoidance system
US4922430A (en) 1987-10-30 1990-05-01 U.S. Philips Corporation Method and apparatus for controlling the movement of a guided object
US5150452A (en) 1989-07-28 1992-09-22 Megamation Incorporated Method and apparatus for anti-collision and collision protection for multiple robot system
JP2769052B2 (ja) * 1991-04-09 1998-06-25 インターナショナル・ビジネス・マシーンズ・コーポレイション 自律移動機械、移動機械の制御装置及び方法
US5318254A (en) 1991-06-28 1994-06-07 Conceptual Solutions, Inc. Aircraft maintenance robot
US5347459A (en) * 1993-03-17 1994-09-13 National Research Council Of Canada Real time collision detection
US5416321A (en) * 1993-04-08 1995-05-16 Coleman Research Corporation Integrated apparatus for mapping and characterizing the chemical composition of surfaces
JP3056256B2 (ja) 1994-06-17 2000-06-26 株式会社ナムコ ３次元シミュレータ装置及び画像合成方法
US5673377A (en) 1994-09-02 1997-09-30 Ray Dream, Inc. Method and system for displaying a representation of a three-dimensional object with surface features that conform to the surface of the three-dimensional object
US5548694A (en) 1995-01-31 1996-08-20 Mitsubishi Electric Information Technology Center America, Inc. Collision avoidance system for voxel-based object representation
US6113343A (en) * 1996-12-16 2000-09-05 Goldenberg; Andrew Explosives disposal robot
JP3765949B2 (ja) 1999-02-17 2006-04-12 富士通株式会社 オブジェクト指向ソフトウェア開発支援装置および開発支援方法
US9345544B2 (en) 1999-09-17 2016-05-24 Intuitive Surgical Operations, Inc. Systems and methods for avoiding collisions between manipulator arms using a null-space
GB0000672D0 (en) 2000-01-13 2000-03-08 Atlas Ward Structures Limited Method of designing a structural element
US6873944B1 (en) 2000-10-11 2005-03-29 Ford Global Technologies, Llc Method of real time collision detection between geometric models
ES2885435T3 (es) * 2001-04-30 2021-12-13 Activemap Llc Mapa interactivo presentado electrónicamente
JP4364634B2 (ja) * 2001-07-13 2009-11-18 ブルックス オートメーション インコーポレイテッド 二次元３自由度ロボットアームの軌道プラニング及び移動制御戦略
US6678582B2 (en) 2002-05-30 2004-01-13 Kuka Roboter Gmbh Method and control device for avoiding collisions between cooperating robots
JP4513568B2 (ja) 2002-07-18 2010-07-28 株式会社安川電機 ロボット制御装置
JP3708083B2 (ja) 2003-02-28 2005-10-19 ファナック株式会社 ロボット教示装置
EP1652634B1 (fr) 2003-07-29 2011-12-28 Panasonic Corporation Procede de commande du bras d'un robot et dispositif de commande associe
JP4027350B2 (ja) 2004-06-29 2007-12-26 ファナック株式会社 ロボットの待機位置復帰プログラム作成装置
US7818910B2 (en) * 2004-09-29 2010-10-26 The United States Of America As Represented By The Secretary Of The Army Weapon integrated controller
US7474945B2 (en) * 2004-12-14 2009-01-06 Honda Motor Company, Ltd. Route generating system for an autonomous mobile robot
US8024066B2 (en) * 2005-01-18 2011-09-20 Redzone Robotics, Inc. Autonomous inspector mobile platform
US20060218680A1 (en) * 2005-03-28 2006-09-28 Bailey Andrew D Iii Apparatus for servicing a plasma processing system with a robot
JP4544145B2 (ja) 2005-11-24 2010-09-15 株式会社デンソーウェーブ ロボットの干渉回避方法およびロボット
US8326469B2 (en) * 2006-07-14 2012-12-04 Irobot Corporation Autonomous behaviors for a remote vehicle
JP4508164B2 (ja) * 2006-06-26 2010-07-21 トヨタ自動車株式会社 多関節ロボット及びその制御プログラム
US8271132B2 (en) * 2008-03-13 2012-09-18 Battelle Energy Alliance, Llc System and method for seamless task-directed autonomy for robots
US7974738B2 (en) * 2006-07-05 2011-07-05 Battelle Energy Alliance, Llc Robotics virtual rail system and method
US7584020B2 (en) * 2006-07-05 2009-09-01 Battelle Energy Alliance, Llc Occupancy change detection system and method
US8355818B2 (en) * 2009-09-03 2013-01-15 Battelle Energy Alliance, Llc Robots, systems, and methods for hazard evaluation and visualization
US8843244B2 (en) * 2006-10-06 2014-09-23 Irobot Corporation Autonomous behaviors for a remove vehicle
JP5112666B2 (ja) * 2006-09-11 2013-01-09 株式会社日立製作所 移動装置
DE502006009264D1 (de) * 2006-09-30 2011-05-19 Abb Technology Ag Verfahren und System zur Auslegung und Überprüfung von Sicherheitsbereichen eines Roboters
US9747698B2 (en) * 2006-10-21 2017-08-29 Sam Stathis System for accurately and precisely locating and marking a position in space using wireless communications and robotics
EP1972415B1 (fr) 2007-03-23 2019-01-02 Honda Research Institute Europe GmbH Robots avec fonction d'évitement de collision
EP2142132B1 (fr) * 2007-04-16 2012-09-26 NeuroArm Surgical, Ltd. Système de restriction non mécanique et/ou de programmation du mouvement d'un outil d'un manipulateur le long d'un axe unique
EP2142133B1 (fr) * 2007-04-16 2012-10-10 NeuroArm Surgical, Ltd. Méthodes, dispositifs et systèmes de mouvements automatisés concernant des robots médicaux
FR2916152B1 (fr) 2007-05-14 2010-03-05 Robosoft Robot polyvalent
US8255092B2 (en) * 2007-05-14 2012-08-28 Irobot Corporation Autonomous behaviors for a remote vehicle
JP2009211571A (ja) * 2008-03-06 2009-09-17 Sony Corp 軌道計画装置及び軌道計画方法、並びにコンピュータ・プログラム
US8244469B2 (en) * 2008-03-16 2012-08-14 Irobot Corporation Collaborative engagement for target identification and tracking
US8315738B2 (en) * 2008-05-21 2012-11-20 Fanuc Robotics America, Inc. Multi-arm robot system interference check via three dimensional automatic zones
US9144904B2 (en) 2008-05-21 2015-09-29 Fanuc Robotics America Corporation Method and system for automatically preventing deadlock in multi-robot systems
JP4655147B2 (ja) * 2008-12-26 2011-03-23 ソニー株式会社 電子機器と地図表示方法およびコンピュータ・プログラム
WO2011056633A1 (fr) 2009-10-27 2011-05-12 Battelle Memorial Institute Système robotisé polyvalent semi-autonome et procédé de fonctionnement
FR2954518B1 (fr) 2009-12-18 2012-03-23 Aripa Service Innovation Ind " systeme anticollision pour le deplacement d'un objet dans un environnement encombre."
KR101633357B1 (ko) * 2010-01-14 2016-06-28 삼성전자 주식회사 매니퓰레이터 및 그 제어방법
US8340820B2 (en) 2010-02-26 2012-12-25 Agilent Technologies, Inc. Robot arm and method of controlling robot arm to avoid collisions
DE102010063208A1 (de) 2010-12-16 2012-06-21 Robert Bosch Gmbh Verfahren zum Betreiben einer Sicherungseinrichtung für eine Handhabungsvorrichtung, Sicherungseinrichtung für eine Handhabungsvorrichtung und Handhabungsvorrichtung
KR101732902B1 (ko) * 2010-12-27 2017-05-24 삼성전자주식회사 로봇의 경로 계획 장치 및 그 방법
EP2508956B1 (fr) * 2011-04-06 2013-10-30 Kollmorgen Särö AB Procédé et système d'évitement de collisions
US20130343640A1 (en) 2012-06-21 2013-12-26 Rethink Robotics, Inc. Vision-guided robots and methods of training them
EP2838698B2 (fr) 2012-07-10 2020-01-01 Siemens Aktiengesellschaft Agencement d'un robot et procédé de commande d'un robot
US9132551B2 (en) 2012-11-30 2015-09-15 Abb Technology Ag Teleoperated industrial robots
US9283678B2 (en) 2014-07-16 2016-03-15 Google Inc. Virtual safety cages for robotic devices
JP6379874B2 (ja) 2014-08-29 2018-08-29 株式会社安川電機 ティーチングシステム、ロボットシステムおよびティーチング方法

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008124713A2 (fr) * 2007-04-09 2008-10-16 Sam Stathis Système et procédé capables de naviguer et/ou de mapper tout espace multidimensionnel

Also Published As

Publication number Publication date
US10011012B2 (en) 2018-07-03
US9643316B2 (en) 2017-05-09
EP2493664A1 (fr) 2012-09-05
US20120215354A1 (en) 2012-08-23
WO2011056633A1 (fr) 2011-05-12
US20170305015A1 (en) 2017-10-26

Similar Documents

Publication Publication Date Title
US10011012B2 (en) Semi-autonomous multi-use robot system and method of operation
US10585167B2 (en) Relative object localization process for local positioning system
JP6426143B2 (ja) 複雑な表面検査及び加工のための制御された自律型ロボットシステムおよびその方法
Mineo et al. Robotic path planning for non-destructive testing–A custom MATLAB toolbox approach
JP6969886B2 (ja) 飛行機組み立て構築プロセスに対するファクトリオートメーションの適用
US20180024521A1 (en) Control device, robot, and robot system
JP5144785B2 (ja) ロボットの着目部位と周辺物との干渉を予測する方法及び装置
KR20180052045A (ko) 레이저 스캐너를 이용한 대상 물체의 실시간 동적 위치 추적이 가능한 3ｄ 레이저 스캐너 시스템
US20220092766A1 (en) Feature inspection system
CN116568465A (zh) 用于机器人单元的改进的自动校准的方法和系统
CN115841516A (zh) 用于对摄像头的动态本征参数进行建模的方法和设备
US20020120359A1 (en) System and method for planning a tool path along a contoured surface
Schneider et al. Optical 3-D measurement systems for quality control in industry
de Souza et al. Industrial robot programming by demonstration using stereoscopic vision and inertial sensing
CA3235422A1 (fr) Systemes et procedes de nettoyage controle de vehicules
CN115972192A (zh) 具有可变空间分辨率的3d计算机视觉系统
Saukkoriipi Design and implementation of robot skill programming and control
Hamel et al. Elements of telerobotics necessary for waste clean up automation
Zaki et al. On the use of low-cost 3D stereo depth camera to drive robot trajectories in contact-based applications
JP2003271993A (ja) モニタ画像処理方法及び画像モニタシステム並びに保全作業システム
Nakhaeinia et al. Surface following with an RGB-D vision-guided robotic system for automated and rapid vehicle inspection
US20230398688A1 (en) Motion trajectory generation method for robot, motion trajectory generation apparatus for robot, robot system, and program
CN111065977B (zh) 控制表面的方法
WO2022163580A1 (fr) Procédé de traitement et dispositif de traitement pour la génération d'une image de section transversale à partir d'informations de position tridimensionnelle acquises par un capteur visuel
WO2022191148A1 (fr) Outil d'apprentissage et dispositif d'apprentissage permettant d'utiliser la main d'un opérateur pour régler un point d'apprentissage

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20120523

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20140722

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20180926

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602010057074

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1097575

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190315

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

Ref country code: NL

Ref legal event code: MP

Effective date: 20190220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190520

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190620

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190521

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190520

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190620

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1097575

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602010057074

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

26N No opposition filed

Effective date: 20191121

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191027

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20191031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191027

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602010057074

Country of ref document: DE

Representative=s name: HL KEMPNER PATENTANWALT, RECHTSANWALT, SOLICIT, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20101027

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20210921

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20210929

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20210916

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20211013

Year of fee payment: 12

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190220

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602010057074

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20221027

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221031

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230503

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221027

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221027