GB2405465A

GB2405465A - Using stripe laser technology for determining the displacement offset to a surface, in order for a robot to find a target within its space

Info

Publication number: GB2405465A
Application number: GB0320069A
Authority: GB
Inventors: Dale Read
Original assignee: PROPHET CONTROL SYSTEMS Ltd
Current assignee: PROPHET CONTROL SYSTEMS Ltd
Priority date: 2003-08-27
Filing date: 2003-08-27
Publication date: 2005-03-02
Also published as: WO2005022081A3; WO2005022081A2; GB0320069D0

Abstract

A method of determining the displacement offset to a surface in real time, in order for a robot to find a target within its space. This method may be used in robotic windscreen decking (figures. 4a-4f). The method may be used for continuous checking of a sealer bead position, dimensions and quality. It may involve scanning the surface of an object to detect indentations or protrusions for an analysis. Such indentations may be alphanumeric text characters (fig. 3b), and the analysis may be through optical character recognition to capture the text into an ASCII format. The method may comprise scanning the surface by a laser profile scanner, using the data obtained to determine the offset of the workpiece surface, and utilising the offset data to control a computer. An OCR program / routine may be used to identify alphanumeric text/symbols.

Description

Title: Manufacturing Applications of stripe laser technology The present

invention relates to laser measurement for automation, particularly in the application of stripe laser scanning in manufacturing industry.

Non-contact displacement measurement has been the key to the success of adaptive robot automation. Two techniques used to date are reflective edge detection and fixed distance sensing: The first method relies on detecting the thin reflections of natural light scattered off the edges of an object and, through computer calculation, working out the offset from the sensor to the desired edge. This has the advantage of scanning a wide area at once and giving a continuous real-time output. However, the output can be affected by ambient lighting conditions and the colour of the object, although much less so than when using ordinary video cameras.

The fixed distance method uses a laser displacement sensor set to trip at a fixed distance and then calculates the object position from the robot joint positions at the moment of tripping. The robot executes a series of manoeuvres, each depending on the result of the previous one, to find a plane, then an edge and then home in to a point in space. This is a highly accurate method and totally independent of ambient light or colour, however, it takes several readings before making a conclusion which is time consuming and means that the object must be stationary.

A laser displacement sensor can be set to deliver continuous real-time output, but it is then only aimed at one point. The system associated with such a laser displacement sensor would know how far away it was from that point, but not know which part of the object on which that point was located it was looking at.

Recent developments in laser measurement technology have combined the distance laser measurement technology with the technique, (used in barcode readers to detect black and white markings along a line), of rapidly sweeping; the laser beam used in a linear fashion across a surface to be measured to produce a visible line or 'stripe', i.e. of scanning the laser beam across the; surface. A device using this technique can output a profile of the scanned surface along the cross sectional plane covered by the scan line, continuously in real-time and is henceforth referred to as a stripe laser.

A real time surface profile 'stripe' laser is used to scan the surface profile of the component(s) that the robot is working on. This scan data is analysed to detect features on the target component. Simple features like highest point and edge detection solve the majority of robot tracking needs. More complex requirements can be programmed into the system to detect specific features.

The key inventive step is to use this technology to combine the advantages of i the first two mentioned techniques and measure in real time across a wide area with high accuracy and repeatability and without cycle time penalty.

A need exists, in modern manufacturing, for a method to determine, in real time, the offsets from a robot's current position to identifiable points on a surface. The next step in robot automation is then to give the real time displacement data to a robot, this allows the robot to adapt to the situation presented and allows it to custom build each unit. When applied to car manufacture it allows for much higher accuracy and flexibility than existing methods with considerable cost savings. t It is an object of the present invention in one aspect to make use of the data from a stripe laser scanner to meet this need.

According to this aspect of the invention, there is provided a method of determining the displacement offset, relative to a reference location, (e. g. the current position of a robot), to a particular feature of a surface in real time, the method comprising scanning the surface with a laser profile scanner having a known position or path relative to said reference location (e.g. mounted on the end effecter of the robot), supplying the resulting data to a computer, using the data to determine the offset of the workpiece having said surface relative to the scanner. The computer may control a robot, for example, and may, for instance, utilising said offset and/or other data enable the robot to find a target presented anywhere within its space.

Thus, for robot automation, the offset to a particular point on a presented surface can be continually recalculated in real-time. This enables the robot to adapt its path depending on the situation presented to it, removing the need for heavy tooling and clamps.

A second practical application of the present invention stems from the need, in automated manufacturing, to verify the quality of the bead of sealant, adhesive or the like applied to one component prior to the application of that component to the surface of another component. Sealer controllers and dosers have advanced significantly to the point where they measure the volume of material dispensed against time. If this volume differs significantly from a target signature' then this indicates that the bead consistency is compromised.

However this information, in known sealer controllers and dosers comes from piston movement in the closer and does not take into account any problems beyond that point such as air pockets or obstructions in the nozzle of the sealer closer or damage to the shape of the bead of sealant after it is dispensed. A need exists for real-time scanning of the profile of such a bead of sealant, once it is laid on the workpiece.

Thus, according to another aspect of the invention, there is provided, in a manufacturing procedure involving the application of a bead of sealant, adhesive or the like to a workpiece, a method of continuously checking bead position and dimensions and detecting inconsistencies in bead quality, comprising (a) providing a laser profile scanner and (b) using said laser profile scanner repeatedly to check the profile, position or other parameter of said bead after it has been applied.

For sealer application, the bead profile immediately after dispensing can be continually monitored in real-time. This is an automatic quality check which will highlight any gaps or excess material in the bead, allowing the system to reject out of specification results. Also small drifts in position, height or width can be detected and fed back into the control system for automatic adjustment, so that the system exhibits "learning intelligence", and continuous improvement of bead quality, or continuous maintenance of good bead quality, is made possible.

A further aspect of the invention is concerned with a specific problem encountered in the aluminium smelting industry, and with analogous problems in other fields. In aluminium smelting, smelting anodes are made by compacting carbon slurry into large cuboid blocks having holes for hangers and having serial numbers stamped on top. These numbers need to be tracked through the process, but are difficult to read since they comprise indentations in a rough black surface and always containing some (black) dust. A need exists for a method of reading these numbers reliably and automatically.

Thus, according to a third aspect of the invention, there is provided a method of scanning the surface of an object to find indentations or protrusions for analysis, comprising using a stripe laser profile scanner to effect such scanning and providing corresponding information signals, significant of said indentations or protrusions on or in said surface, to data processing means for analysis.

For surface scanning, an object is passed underneath a stripe laser scanner in a direction perpendicular to the line scan direction, so that the whole surface is covered. The data from the stripe laser scanner is fed to a computer and the distance profiles of successive scans on the surface measured by the laser beam are stored in a data array in the computer so that a 3D map or image of the scanned surface is built up in the computer and this can then be analysed to detect features of the surface, for example indented numerals.

Thus, the present invention provides techniques for using stripe laser profile measurement scanners in a variety of manufacturing applications.

The reason this invention is important is because the robot's behaviour is determined by its actual relationship to the prevailing conditions in its environment, rather than making assumptions about the state of the environment and hoping that the conditions in the environment will conform to them. This makes the robot's behaviour adaptive and appropriate to the situation in which it finds itself.

This is achieved by sensing in real time the relative location of target objects in its environment using a stripe laser, and acting upon this information in a manner which homes in on a desired outcome.

Embodiments of the invention are described below by way of example with reference to the accompanying illustrations, in which: Figure la illustrates a computer monitor display in an apparatus utilising the invention to control a robot engaged in decking automobile windscreens, and more particularly illustrating identification of nearest point and calculation of offset, Figure lb is a view of a section of windscreen pillar to which the profile shown in Figure 1 a relates, Figure 2a illustrates a computer monitor display in an apparatus utilising the invention to control the application of a bead of sealant, adhesive, or the like to a workpiece, Figure 2b is a perspective view of such a bead of sealant, adhesive, or the like F on an automobile window in the course of automobile manufacture, Figure 3a illustrates a computer monitor display in an apparatus utilising the invention to read automatically alpha numeric symbols appearing in relief on a surface, Figure 3b is a perspective view showing, on the right hand, the surface to which the monitor display of Figure 3a relates, and, on the left hand, the surface to which the monitor display of Figure 3c relates, and Figure 3c is a view corresponding to Figure 3a but showing a computer monitor display in an apparatus utilising the invention to read automatically alpha numeric symbols impressed on a surface.

Figure 3d shows a schematic of one possible configuration of equipment to achieve the surface scanning process.

Figure 4 contains eight photographs of typical automotive industry robotic processes which would be greatly enhanced through the use of stripe laser technology.

Figures 4a shows the pre-measurement followed by the application of sealer to an engine compartment surface in an automotive body shop. 7 i

Figures 4b shows the pre-measurement followed by the end result of application of seam sealer Figures 4c, 4d and 4e show the assembly plant processes for insertion of windscreen, instrument panel and engine into the vehicle automatically using robots.

Figure 4f shows a robot performing a full gauging sequence on a vehicle body by moving the laser to a number of points and measuring the exact dimensions of the produced vehicle.

Figure 5 shows the configuration of laser, PC and robot to achieve a working system.

Figure 6 shows the main Cimtrack running screen with annotations describing various key points.

Figure 7 zooms in on some of the settings inputs and describes their function.

Figure 8 shows the specification of the laser used in trials as an example of appropriate hardware, other manufacturer's specifications are not excluded.

In the embodiments of the invention described below, the laser profile scanner used will be a commercially available item. The principal requirement is that it must be able to return distance information over a scanned line. The appropriate length of scanned line, operating range and measurement resolution will depend on the application required. A suitable specification, based on that used in the trials is given in Figure 8, and the appropriate system configuration setup including PC and robot is shown in Figure 5.

The 'CimTrack' software code (1 in the software attachment) collects data from the stripe laser scanner using a software library routine supplied by the manufacturer. This data can be downloaded in two ways. The waveform from the scanner can be brought into a computer, such as a PC, via an analogue card, and then sampled at regular intervals to produce digitised information which must be processed and linearised (compensating for the angle of arc swept by the scanner). Alternatively, the laser scanner manufacturer may include an analogue to digital converter and the output data may be pre-digitised and linearised, prior to being passed to the computer.

In either case, the scanned profile data is brought into the computer memory into one or more arrays containing the following information: point number, Z and X co-ordinates (parallel and perpendicular to the centre line of the scan) and, optionally, intensity. The full data can be used, or the data may be sampled every other frame to increase the scan speed. The user may change these settings using the controls shown in Figure 7.

Once the data is in manageable arrays it must be cleaned and smoothed. The software used crops off noisy data at the limits of each scan and also removes any spurious out-of-range points within the scanned region. The remaining data is then checked for continuity and any gaps are filled by interpolating between neighbouring points. The result is a set of points defining a continuous profile in a form which can be analysed such as that shown in Figure 6.

The trial system is capable of performing at a sample period of 40ms giving 25 samples per second. Communication to the robot is via serial communication with baud rates between 9600 and 115000 depending on the robot's capabilities. The PC system is typically a lGhz industrial PC. Offset data in Z and X are calculated by the system and sent to the robot. Z represents the forward distance of the target away from the robot and X the lateral displacement from the centre line of the scan. The field of view in Z is from 90mm to 290mm and in X is 65mm each side of centre. Resolution in both dimensions is 0.2mm. Different values for all the attributes above may also produce a satisfactory result, but the above figures create a working configuration.

The system can be retrofitted to existing robot cells with minimal disturbance to the process. Component offset data is given to the robot selectable on a request or continuous basis.

No calibration of the laser head itself is required, as this is done by the manufacturer. When the system is first commissioned the laser is pointed at a fixed location fixture in the cell. The X and Z reading are taken and recorded. If the laser is replaced, then the same calibration point is measured and the difference (if any) is noticed. To correct the calibration offset the robot can deduct the difference between two readings, or the laser sensor can be moved until no error is measured. Laser disconnection and reconnection time is typically 5 minutes.

Communication between the robot and the computer is via a simple serial ASCII communication protocol using the RS232 serial ports in the computer and robot. The robot normally initiates the communication and the computer responds to commands. Each ASCII value is separated by a carriage return and line feed (crlf). The software code attachment contains the routines written for Fanuc robots (2a) and Comau robots (2b) The robot-computer communications protocol includes provision for the following commands: Laser off Laser on Snapshot data Continuous on Continuous off When continuous data mode is selected, the cimtrack software continuously sends the position data out of the serial port to the robot. In snapshot mode a single instantaneous position is sent.

Communications speed in the trials is set to 38400,n,8,1 In the first embodiment, (cf. Figures la and lb), stripe laser profile scanning is used to control an industrial robot through real time identification of the location of the target. The example used is the placing, or "decking", of windscreens in the correct location in the windscreen frames or openings of automobile bodies. In an automobile manufacturing plant, a stripe laser profile scanner mounted on a limb or effecter of the robot is used to determine the offset from a robot-mounted scanner to a specific feature on the profile. For example in Figure la, the measured profiles of two car windscreen pillars are shown. The pillar profile itself is shown in Figure lb. The robot needs to centre the glass it is carrying between these pillars in the decking operation shown in Figure 4c.

In this case, the software continuously analyses the real time waveform from the scanner to determine the point along the profile closest to the scanner. The robot start position and orientation of the scanner on the end effecter of the robot mean that this identifies with confidence the top corner of the A-pillar profile as shown in Figure 1.

The returned values are the Z and X components of this displacement. Note that this represents the actual displacement vector from where the robot is at a point in time to where the measured point is at the same point in time, and is independent of absolute co-ordinates in space. The control system can then calculate the appropriate relative motion for the robot to take to execute its task.

The displacement is recalculated on a continuous basis as the robot gets closer to its target and thus it homes in to achieve the perfect location.

In the windscreen example, four scanners are required in order to centre the glass between the A-pillars, (i.e. the portions of the vehicle body which define opposite sides of the "frame" in which the windscreen is to be mounted), and align the glass with the top edge of the body aperture for proper positioning.

Windscreen fitting on a continuously moving line has been achieved.

This analysis identifies the nearest point to achieve its goal, however the same technique could find any feature on the profile, for example the lower corner, or work out the angle of the line between the inflection points thus defining the plane of the aperture boundary into which the glass must fit.

Therefore this embodiment is applicable to any component location and assembly as well as sealer processes as described below: In the instrument panel insertion system (Figure 4d), robots first measure the A' pillar dimensions and pass this data to the insertion robot. The insertion robot positions the instrument panel in the correct position for the car, the measuring robots pick up the bolts to secure the instrument panel and run them into the car. The car body is not clamped in place during the cell operation.

Auto decking of engine, gearbox and drive train is achieved using laser measurement (figure 4e). The car is delivered on a carrier, which stops within 50mm; two robots measure the car position and send this data to the other robots. The front robots move the suspension strut into position as the car is decked. After the car is decked the robots measure the final position to reach the nut runners to run down the fixing bolts.

The cimtrack laser measuring system can be used to pre-measure the car body before fitting doors, hood, roof panels, instrument panels, glass, lights etc. A robot equipped with the laser measures the component to be fitted and the area on the car where the component is to be fitted to (Figure 4f). From this measurement data the best possible position for accurate fit can be calculated.

The robot then assembles the component using this data. This system greatly simplifies the mechanical content of a machine, provides greater accuracy and flexibility. Inline body measurement is also possible where the body is measured by the robot as part of the normal process with no addition to cycle time.

There are many examples of robot-applied sealer in industry. Most of these applications consist of clamping a component into a fixture for the robot to apply the sealer material to the component. These systems are relatively 13; inflexible (each component requires an individual clamping fixture) and potentially inaccurate. Because there is only one robot program for each I component, individual tolerances in the component cannot be compensated for.

This results in the robot being programmed for the 'typical component, this is usually the largest component so the sealer application nozzle does not crash on the surface of the component. The smallest component will not need the same quality of sealer application as the largest component. Some sealer systems require the sealer to be applied at 1-2mm above the surface of the component, if the tolerance of the component or clamping fixture is 1-2mm then the quality of application will suffer.

In the body shop example pictured in Figure 4a, an engine compartment is delivered in front of the robot on a transport system. The engine compartment is not clamped or mechanically located for the sealer application. The robot uses its cimtrack laser system (mounted behind the yellow bracket) to measure the actual position of the component. The robot then uses this offset data to fine tune the sealer application path and apply the sealer. The sealer application nozzle needs to be 2mm away from the panel surface. As the stack up of tolerances of the transport system and component can add up to lOmm, the cimtrack laser system ensures accurate application of sealer to each individual component.

In the paint shop example in Figure 4a the car body is not clamped or located in the cell, only the skid is held. The robot uses its cimtrack laser system to find i the offsets for the roof ditch sealer application. The robot then applies the sealer to the roof ditch area. The second picture shows the finished sealer applied on the roof ditch seam and around the clip, which located the roof trim on the finished vehicle. ] Similar sealer applications exist in the paint shop areas of car manufacture.

Often these applications are manual, with extensive manual brushing to finish I the sealer application and to overcome accuracy problems. In a traditional automated system the car body is located on tooling pins to try to fix the position of the car for the robot to work on. Tooling pins have a tolerance which the robot cannot compensate for, car bodies are each a different size, varying from end to end by as much as 5mm. If a car body is pinned at the underbody level it does not guarantee the location for example of the roof of that car. Often CCD (charge coupled device) camera vision systems are used to compensate for this movement but they suffer from their own problems. Light, I colour, environment, set-up all have detrimental influences on CCD solutions. i Additionally each camera on a CCD solution can only look at one area of the car body, to get distance information 2 cameras are required for each point. If either are both are knocked out of line then positioning will be affected.

A robot equipped with a cimtrack laser tracking system overcomes these difficulties. The laser can directly measure the position in space of the seam to which the robot is applying the sealer. This eliminates the need for accurate clamping of the body (also eliminating potential paint damage), and gives greater flexibility as each body type can be software programmed in the robot.

The robot then uses the offset data delivered from the laser system to accurately apply the sealer to the correct place. Virtually all manual brushing is eliminated.

In a second embodiment, (cf. Figures 2a and 2b), laser profile scanning is used for checking the quality of sealer beads. There are two main scenarios. Either a robot moves a sealer nozzle over a fixed workpiece, or the robot manipulates the workpiece under a fixed nozzle. In both situations the profile scanner must I be set so that it trails the nozzle and 'sees' the bead that has just been laid. I The line scanned by the laser must be perpendicular to the bead centre line and must cover the whole width of the bead plus an adequate amount of the original workpiece surface each side. The profile scanner continually captures the distance measurements into an array in exactly the same manner as described above. As the bead moves under the scanner the computer program "sees" a real time image of the scanned bead cross-section.

This data array is then analysed in a similar manner to that used in the first embodiment. The nearest point is identified as the top of the bead. The flat part (or a known shape) defines the back surface. The two points either side of the I high point where the profile begins to rise from the base up to the high point i define the edges of the bead. The apparatus can, in fact, be set to sense and measure any specified dimensional or displacement parameter of the sealant bead, and to detect and correct any deviation of such parameter outside of customer specified tolerances.

From these measurements, the following properties of the bead can be checked against the acceptable ranges: Height = nearest point to back surface - shows up gaps or excess material Width = distance between detected edges - shows up cosmetic irregularities Position = Y coordinate of highest point - indicates any lateral drift Shape = ratio of lateral distances from highest point to left and right hand edges. If a specific cross-section perimeter shape is required then the program can compare the real-time profile with a 'signature' template. i Any of these parameters that stray outside the specified tolerances will be highlighted to the control system and appropriate action will be taken. This will depend on the end customer's specification, but for example, the process could I be aborted to await manual intervention or the product could be marked as I requiring rework but production allowed to continue. An advanced system will actually go back and fix the problem itself. Direct feedback into the control system means it can also adjust itself so the problem does not happen again.

Special treatment for corners may be required as the bead may move relative to the scanner as the robot re-orientates itself. This must not be taken as an error, but by teaching the system what a good operation looks like, it will be able to pass or fail process cycles accordingly or to rework any defect automatically.

Referring to Figures 3a and 3b, in a third embodiment laser profile scanning is used to read serial numbers stamped into a surface. Where, as discussed above, the surface in question is that of a carbon anode, the surface of the block is already rough, has a few deep holes for the hangers and several serial number digits stamped in that surface and which are the same colour as the surface. As there are various combinations of hole and number locations on the surface, a physical method of finding the depressions is not suitable.

In this embodiment of the invention, a laser profile scanner is used to map the surface in much the same way as a computer image scanner, but capturing distance information. The scanner may be mounted in a fixed position and the surface moved under it. The profile scanner captures a shape of the immediate cross section exactly as described above, and as the surface moves, the computer builds up information about the whole surface. The rough surface will show up as minute distance differences, the stamped numbers will be noticeable depressions and the holes will show up as infinity. A schematic of the setup is shown in Figure 3d.

To minimise the information stored and thus the information to be processed, the program decides on each scan which data to record. The rough surface data will be averaged out and any points within a certain range of this average will be ignored as basic surface. The holes at infinity will be ignored. Only points where the distance deviates significantly from the average surface level either side of a low point will be recorded. After the full surface scan, the set of point coordinates is transformed into a graphical image forming the digits of the serial number. The graphic will be captured and rotated to create a standalone image.

This will then be fed through standard Optical Character Recognition (OCR) software such as that received with any PC scanner to covert the graphic to recognizable text. The pattern of dots may require enhancing before a standard OCR will read it, but within the computer environment this will be handled within the computer program.

This text string can now be forwarded to whatever system requires the serial number, for example material tracking or quality traceability.

Laser scanners are independent of ambient lighting conditions, immune to colour differences and accurate to 0.001mm. Therefore the black on black digits should present no difficulties, even with dust in the grooves. As the computer program is manipulating the data, the system can cope with any position or orientation of the number.

Claims

Claims: 1. A method of determining the displacement offset to a particular

feature of a surface in real time, in order for a robot to find a target presented anywhere within its space.
2. The use of the method of claim 1 for the specific purpose of robotic windscreen decking.
3. A methodology for continually checking sealer bead position and dimensions and detecting inconsistencies in quality.
4. A method of 'distance' scanning the surface of an object to detect indentations or protrusions for analysis.
5. A method of detecting alphanumeric text characters in or on a surface of an object comprising scanning the surface of the object to obtain data concerning indentations or protrusions and analysing said data including and feeding said data or data derived therefrom through optical character recognition software to capture the text into ASCII format or other format readily stored or manipulated in a computer, for use as required.
6. A method of determining the displacement offset to a particular feature of a surface in real time, in order for a robot to find a target presented anywhere within its space, the method comprising scanning the surface with a laser profile scanner, supplying the resulting data to a computer, using the data to determine the offset of the workplace having said surface relative to the scanner and utilising said offset and/or data to control the computer. 9 i 1'
7. In a manufacturing procedure involving the application of a bead of sealant, adhesive or the like to workpiece, a method of continuously checking bead position and dimensions and detecting inconsistencies in bead quality, comprising (a) providing a laser profile scanner and (b) using said laser profile scanner repeatedly to check the profile, position or other parameter of said bead after it has been applied.
8. A method of scanning the surface of an object to find indentations or protrusions for analysis, comprising using a laser profile scanner to effect such scanning and providing corresponding information signals, significant of said indentations or protrusions on or in said surface, to data processing means for analysis.
9. A method according to claim 8 wherein said data processing means is arranged to run an OCR program or routine to identify alpha-numeric symbols defined by said indentations or protrusions and to store, and/or output, data significant of said alpha-numeric symbols.
10. A method substantially as hereinbefore described with reference to the accompanying drawings.