EP2726813A1 - Surface measurement system and method - Google Patents

Surface measurement system and method

Info

Publication number
EP2726813A1
EP2726813A1 EP12804202.5A EP12804202A EP2726813A1 EP 2726813 A1 EP2726813 A1 EP 2726813A1 EP 12804202 A EP12804202 A EP 12804202A EP 2726813 A1 EP2726813 A1 EP 2726813A1
Authority
EP
European Patent Office
Prior art keywords
irregular workpiece
samples
workpiece surface
peaks
total number
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP12804202.5A
Other languages
German (de)
French (fr)
Other versions
EP2726813B1 (en
EP2726813A4 (en
Inventor
Raymond BITZEL
Johnathan JONES
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wheelabrator Group Inc
Original Assignee
Wheelabrator Group Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wheelabrator Group Inc filed Critical Wheelabrator Group Inc
Publication of EP2726813A1 publication Critical patent/EP2726813A1/en
Publication of EP2726813A4 publication Critical patent/EP2726813A4/en
Application granted granted Critical
Publication of EP2726813B1 publication Critical patent/EP2726813B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/30Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces
    • G01B11/303Measuring arrangements characterised by the use of optical techniques for measuring roughness or irregularity of surfaces using photoelectric detection means
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B7/00Measuring arrangements characterised by the use of electric or magnetic techniques
    • G01B7/34Measuring arrangements characterised by the use of electric or magnetic techniques for measuring roughness or irregularity of surfaces

Definitions

  • This disclosure relates generally to the field of abrasive surface preparation, and more particularly to methods and apparatus for measuring the profile of a prepared surface.
  • Abrasive surface preparation is a process used to clean or condition the surface of a material, typically metal, to strengthen it, prepare it for receiving a coating, such as paint, or for other purposes.
  • a coating such as paint
  • an abrasive media is blasted onto the surface under high pressure.
  • the resulting profile of the prepared surface i.e., the three-dimensional variations of the surface from a perfectly smooth or "true" plane
  • the resulting profile of the prepared surface describes the relative roughness of the surface, and is known to be a factor in the performance of various coatings applied to metals, such as steel.
  • One existing standard for quantifying surface irregularities is described by ANSI B46.1-2009: Surface Texture, Surface Roughness, Waviness and Lay (American Society of Mechanical Engineers, August 20, 2010, ISBN 9780791832622).
  • Method A a visual comparator (e.g., a template) is viewed in comparison with the surface being measured, usually with a magnifier.
  • Method B a gauge having a needle or stylus is laid on top of the surface at various locations to measure the distance between the peaks and valleys of the surface.
  • Method C a tape having a compressible foam layer is pressed onto a portion of the surface to make an impression of the profile, which is then measured with a spring micrometer.
  • a surface measurement method includes inspecting a plurality of samples of a first irregular workpiece surface in two dimensions, determining, based on the inspections of the samples, a total number of surface peaks on each of the plurality of samples, and deriving a control limit from a statistical variation of the total number of surface peaks on each of the samples.
  • the control limit specifies an out- of-tolerance condition for the total number of surface peaks on a second irregular workpiece surface.
  • the method further includes inspecting a portion of the second irregular workpiece surface in two dimensions, determining, based on the inspection of the portion of the second irregular workpiece surface, a total number of surface peaks on the portion, and comparing the total number of surface peaks on the portion to the control limit to determine whether the second irregular workpiece surface is in the out-of-tolerance condition.
  • inspecting the samples of the first irregular workpiece surface in two dimensions may include optically inspecting the samples of the first irregular workpiece surface in two dimensions.
  • Inspecting the portion of the second irregular workpiece surface in two dimensions may include optically inspecting the portion of the second irregular workpiece surface in two dimensions.
  • the method may include alerting an operator in response to determining that the second irregular workpiece surface is in the out-of-tolerance condition.
  • alerting the operator may include at least one of illuminating a lighted warning indicator, sounding an audible alarm, and displaying an alert message in a user interface of an operator terminal.
  • the method may include measuring an average height of the surface peaks on at least one of the plurality of samples, and correlating the total number of surface peaks on each of the plurality of samples to the average height such that a surface profile of the second irregular workpiece surface can be determined based on the total number of surface peaks on the portion of the second irregular workpiece surface.
  • the method may include determining the surface profile of the second irregular workpiece surface based on the correlation, wherein the surface profile represents an average roughness R a of the second irregular workpiece surface.
  • the plurality of samples may include at least 100 samples of the first irregular workpiece surface.
  • the size of each of the samples may be approximately 0.04 inches square.
  • the control limit may be derived from a +/-3 sigma statistical variation of the total number of surface peaks on each of the plurality of samples.
  • a surface measurement apparatus includes a sensing unit and a controller coupled to the sensing unit.
  • the controller is configured to inspect, using the sensing unit, a plurality of samples of a first irregular workpiece surface in two dimensions; determine, based on the inspections of the samples, a total number of surface peaks on each of the samples; derive a control limit from a statistical variation of the total number of surface peaks on each of the samples, the control limit for specifying an out-of- tolerance condition for the total number of surface peaks on a second irregular workpiece surface; inspect, using the sensing unit, a portion of the second irregular workpiece surface in two dimensions; determine, based on the inspection of the portion, a total number of surface peaks on the portion; and compare the total number of surface peaks on the portion to the control limit to determine whether the second irregular workpiece surface is in the out-of-tolerance condition.
  • Each of the plurality of samples has substantially the same size, and the portion has substantially the same size as one of the plurality of samples.
  • the sensing unit may include a camera configured to capture an image of the portion of the first irregular workpiece surface and an image of the portion of the second irregular workpiece surface.
  • the sensing unit may include an inclinometer configured to determine an angle of the camera with respect to the portion of the irregular workpiece surface.
  • the sensing unit may include a proximity sensor configured to determine a distance between the camera and the first irregular workpiece surface, the second irregular workpiece surface, or both.
  • the sensing unit may include an illumination device configured to illuminate the first irregular workpiece surface, the second irregular workpiece surface, or both.
  • the apparatus may include a user interface coupled to the controller.
  • the user interface may be configured to alert an operator in response to the controller determining that the second irregular workpiece surface is in the out-of-tolerance condition.
  • the user interface may include a lighted warning indicator and/or an audible alarm.
  • the controller may be configured to measure an average height of the surface peaks on at least one of the samples, and correlate the total number of surface peaks on each of the samples to the average height such that a surface profile of the second irregular workpiece surface can be determined based on the total number of surface peaks on the portion of the second irregular workpiece surface.
  • the controller may be configured to determine the surface profile of the second irregular workpiece surface based on the correlation, wherein the surface profile represents an average roughness R a of the second irregular workpiece surface.
  • the size of each of the samples of the first irregular workpiece surface may be approximately 0.04 inches square.
  • the sensing unit may be configured to enable a measurement of a height of at least one of the surface peaks of the first irregular workpiece surface.
  • FIG. 1 is a block diagram of a surface profile measurement system in accordance with one embodiment of the present disclosure
  • FIG. 2A is top view of a portion of a workpiece surface to be inspected in accordance with one embodiment of the present disclosure
  • FIG. 2B is a cross-sectional elevation view taken along a cut line B-B' of FIG. 2A;
  • FIG. 2C is a cross-sectional elevation view taken along a cut line C-C of FIG. 2B;
  • FIG. 3 is a flow diagram of a surface measurement process in accordance with one embodiment of the present disclosure.
  • FIG. 4 is another flow diagram, showing details of the surface measurement process of FIG. 3, in accordance with one embodiment of the present disclosure
  • FIG. 5 is yet another flow diagram, showing further details of the surface measurement process of FIG. 3, in accordance with one embodiment of the present disclosure
  • FIG. 6 illustrates a user interface for displaying process data generated by the system of FIG. 1 in accordance with one embodiment of the present disclosure
  • FIGS. 7A and 7B are side elevation and bottom views, respectively, of a sensing head device in accordance with one embodiment of the present disclosure.
  • one disadvantage of the visual comparator is that, because not all surface profiles are the same, a highly experienced inspector must account for any differences, either real or perceived, between the comparator tool and the surface under inspection. Such inspection can be highly subjective to the inspector, and there is no analytical data collected from the inspection.
  • one disadvantage of the handheld gauge with needle or stylus is that several measurements at different points on the surface must be made and averaged together to provide an accurate reading. This technique is also prone to error if the gauge is not precisely used.
  • one disadvantage of the foam tape is that it can be costly and time consuming to take a sufficient number of surface measurements.
  • a non-contact measurement of a surface profile can be acquired through an optical inspection of the surface. Such an inspection can be made using, for example, a machine vision system.
  • a correlation can be made, based on known properties of the material when subjected to a surface preparation technique, between the roughness of the workpiece surface and the number of peaks observed on the surface.
  • the average peak-to-peak distance may be used to describe the profile of a surface
  • the correlation may be utilized to associate a peak count with the average peak-to-peak distance, and thus the average surface roughness (as quantified according to, e.g., ANSI standards). It is further appreciated that embodiments of the methods and apparatus disclosed herein can be automated for fast processing and used in a real-time production process with a relatively small amount of human intervention.
  • FIG. 1 illustrates a block diagram of a surface measurement system 100 in accordance with one embodiment of the present disclosure.
  • the system 100 includes a sensing head 110, a controller 112, and one or more user interface systems 114.
  • the sensing head 110 is positioned near a workpiece 116 (such as a piece of metal), which may move in relation to the sensing head 110 (e.g., on a conveyor).
  • a workpiece 116 such as a piece of metal
  • the sensing head 110 includes one or more of the following: a camera 120, one or more lamps 122 or other illumination devices, an inclinometer 124, and a proximity sensor 126 (e.g., an inductive proximity sensor, a double lensed LED ranging sensor, a laser ranging sensor, or other device configured to detect the presence and/or absence of a metal surface).
  • a proximity sensor 126 e.g., an inductive proximity sensor, a double lensed LED ranging sensor, a laser ranging sensor, or other device configured to detect the presence and/or absence of a metal surface.
  • an inductive proximity sensor is suitable for use when the distance is approximately one inch
  • a double lensed LED or laser ranging sensor is suitable for use when the distance is approximately three inches.
  • the camera 120 and/or lamps 122 are incorporated within a conventional machine vision system having a separate programmable controller, as will be understood by one of skill in the art.
  • Each component of the sensing head 110 may be physically attached to a frame 128 or other supporting structure such that the camera 120, the lamps 122, the inclinometer 124, and/or the proximity sensor 126 remain in substantially fixed spatial relation and orientation to each other at all times. It should be understood that each element of the sensing head 110 may include an attachment feature that allows for adjusting the respective positions during, for example, setup or calibration procedures.
  • the sensing head 110 is coupled to the controller 112 through one or more communication interfaces 118 that provide for the exchange of, among other things, control and monitor data (e.g., camera control, image acquisition, lighting control, sensor monitoring, and so forth). Additional interfaces (not shown) may also be included in the system 100 for providing power to the respective elements of the sensing head 110.
  • the camera 120 can be configured to inspect and identify features of the workpiece 116, including variations and irregularities of the workpiece surface, and to communicate data relating to the workpiece to the controller 112.
  • the camera 120 may be configured to acquire images of portions of the workpiece surface 210 (e.g., each portion having an area of approximately 0.04 square inches).
  • the sensing head 110 and/or the workpiece 116 move relative to one another, the camera 120 may acquire images of different portions of the surface 210.
  • the system 100 can be used to measure the surface 210 in real-time as the workpiece 116 undergoes surface preparation. In one
  • the camera 120 is configured to detect surface variations at a resolution of at least approximately 11 microns or approximately 0.5 mills.
  • the system 100 includes a non-contact surface measurement apparatus.
  • the inclinometer 124 and the proximity sensor 126 enable the system 100 to identify the orientation of the sensing head 110 with respect to the workpiece 116 without physically touching the workpiece.
  • the inclinometer 124 provides data to the controller 112 relating to the angle of the sensing head 110 with respect to gravity. This data is useful for, among other things, determining whether the sensing head 110 is correctly aligned with respect to the workpiece 116 (e.g., parallel to the surface).
  • the proximity sensor 126 provides data to the controller 112 relating to the distance between the sensing head 110 and the surface of the workpiece 116. This data is also useful for alignment purposes.
  • the user interface system 114 is coupled to the controller 112.
  • HMI Human-Machine Interface
  • a graphical HMI such as a Fuji HMI - V815X, sold by Fuji Electric Corporation of America, Fremont, Calif.
  • a web server 132 and web browser 134 e.g., a web server 132 and web browser 134
  • e-mail electronic mail
  • the HMI executes software that enables secure communications (e.g.,
  • a data recorder 140 may be coupled to the controller 112 for capturing and storing data generated by the system 100, such as data associated with the surface profile measurements. Such data may be subsequently retrieved by a user for further analysis or reporting purposes.
  • the workpiece 116 undergoes a form of surface preparation, such as blasting, that is performed prior to inspection and measurement. Such preparation manipulates the surface of the workpiece 116 and causes the surface to acquire a profile that can be visualized as a set of peaks and valleys. Peaks are typically points of the surface that, when viewed in cross-section, are higher than an imaginary plane that is substantially parallel to the surface.
  • the imaginary plane may be, for example, located at an average height of the surface between the highest and lowest points, or at another elevation.
  • valleys are typically points of the surface that are lower than the imaginary plane. Therefore, if an observer inspects the surface of the workpiece 116 at a substantially perpendicular angle to the imaginary plane (e.g., as viewed from above), the peaks will be closer to the observer than the valleys. Likewise, the highest peaks will be closer to the observer than the lowest peaks.
  • the system 100 can be used to inspect and measure the surface of the workpiece 116, for example, to determine if the surface profile is within desired specifications, control limits, or tolerances by identifying peaks and/or valleys on the surface.
  • the system 100 may further be configured to identify surface variations and irregularities caused by other conditions occurring during surface preparation, such as vibration, temperature, wheel speed, and pressure.
  • the peaks and valleys which are identifiable by the camera 120 (or by a system that is configured to analyze the images generated by the camera), can be used to describe a characteristic of the workpiece 116 surface, such as roughness. Roughness can be quantified by the deviations of the surface from a perfectly smooth form. Large deviations may correspond to high roughness.
  • the surface profile of a particular material e.g., steel or iron
  • a particular material e.g., steel or iron
  • the nature of the material is such that the surface can only be manipulated or distorted to a certain extent before it fractures, tears, or resists further manipulation by the surface preparation process (also referred to as overworking the surface). Therefore, the maximum slope between any adjacent peak and valley (or adjacent peaks, or adjacent valleys), which is the rate at which the elevation of the surface changes with respect to distance along the imaginary plane, can be calculated with a level of confidence that is sufficient for determining the average expected distance between any adjacent peaks and valleys for a given average roughness, or R a .
  • the surface profile of the workpiece 116 can be determined by counting the number of surface peaks found within a certain two dimensional view of the inspection area.
  • the inspection area contains a high number of peaks, it can be said that the surface has a relatively rough (or high) profile (and a relatively small peak-to-peak distance), whereas if the number of peaks is relatively low, it can be said that the surface has a relatively smooth (or low) profile (and a relatively large peak-to-peak distance).
  • FIG. 2A is a two-dimensional top view a portion of an exemplary workpiece surface 210, such as would be observed by the camera 120 of FIG. 1.
  • the area under inspection is x by y; although typically the area will be square, it need not be so.
  • the inspection area may be approximately 0.04 square inches (i.e., 0.2 inches by 0.2 inches).
  • illumination and a camera or magnifier such as described above with respect to the system 110 of FIG. 1
  • variations and irregularities in the workpiece surface 210 can be observed, for example, by a machine vision system, or by a human operator. Appearing on an image of the workpiece surface 210 are one or more surface peaks at or above a threshold height h, each indicated at 212.
  • FIG. 1 Appearing on an image of the workpiece surface 210 are one or more surface peaks at or above a threshold height h, each indicated at 212.
  • FIG. 2B shows a cross-section of the workpiece surface 210 along a cut line B- B' showing the corresponding peaks 212 along the cross-section.
  • FIG. 2C shows another cross- section of the workpiece surface 210 along a cut line C-C showing the corresponding peaks 212 along the cross-section.
  • Each peak 212 may be identified by inspection based on the height of the peak relative to other points on the surface using known image processing techniques. In other words, the threshold height h may be established at or above which each peak is identified and counted and below which no peak is observed or counted.
  • the threshold height can be established as the average height of the surface above the lowest point, although it should be understood that other factors may be used to determine the threshold height. For example, some peaks may be insufficiently high for properly characterizing the surface profile, and therefore should be discounted or ignored. In the example workpiece surface 210 of FIG. 2A, there are seven peaks, each identified at 212, at or above the threshold height.
  • FIG. 3 is a flow diagram of a surface measurement process 300, according to one embodiment.
  • the system 100 of FIG. 1 can operate in at least two modes: Teach Mode 310 and Run Mode 312, each of which are described in further detail below with respect to FIGS. 4 and 5, respectively.
  • Teach Mode 310 the surface of the workpiece 116 is prepared, such as by blasting or peening, under controlled conditions prior to measurement by the system 100.
  • the controls may include separate, periodic inspections (e.g., by an operator) to ensure that the process is continuously under control; if the process is out of control, the Teach Mode 310 may need to be restarted.
  • Teach Mode 310 the prepared surface is inspected, either automatically or by hand, as part of a configuration procedure that "teaches" the system 100 various measurement parameters that are subsequently used for real-time measurement of the workpiece 116. Once the system 100 has completed Teach Mode 310, the system 100 enters Run Mode 312.
  • FIG. 4 is a flow diagram of the surface measurement process 300, described above with respect to FIG. 3, while operating in Teach Mode 310, according to one embodiment.
  • a sample of the workpiece 116 surface is inspected.
  • the sample should have substantially the same area as, or sufficiently represent, each subsequent sample.
  • the camera 120 acquires an image of the sample, and sends it to the controller 112 for analysis. It should be understood that in some embodiments, the camera 120 can be part of a machine vision system, which may include a separate controller for analyzing the image; in some other embodiments, image processing can be performed by the controller 112, or by a similar processing device.
  • the controller 112 identifies and counts each peak on the sample.
  • 320 and 322 may be repeated for any number of different samples n (i.e., one or more samples), as indicated at block 324. For example, one hundred (100) samples may be analyzed during the Teach Mode 310 process. The number of samples that should be taken in Teach Mode 310 to achieve desirable results may be determined by statistical analysis.
  • the standard deviation and the average of the number of peaks counted in each sample is calculated.
  • one or more control limits are calculated based on the standard deviation, the average, or both.
  • a statistical model may be derived based on the standard deviation, the average, or both, in which a +/- 3-sigma deviation may be calculated, as should be understood by one of skill in the art.
  • the control limit(s) may be based on the corresponding +/- 3-sigma values.
  • a statistical process control is applied around a known variable (e.g., related to a customer part number), which includes all known process variations associated with the variable.
  • control limits e.g., that are used during a surface preparation process
  • the mathematical model also enables a user to determine how capable and repeatable the surface preparation process will be.
  • the control limits will be used during the Run Mode 312, as described below.
  • FIG. 5 is a flow diagram of the surface measurement process 300, described above with respect to FIG. 3, while operating in Run Mode 312, according to one embodiment.
  • an area of the workpiece 116 surface is inspected.
  • the area should have substantially the same area as or representative of each of the samples inspected during the Teach Mode process 310.
  • the camera 120 acquires an image of the sample, and sends the image to the controller 112 (or other processor) for analysis.
  • each surface peak in the image is identified and counted.
  • the number of peaks counted within the inspection area is compared to one or more of the previously derived control limits, for example, +/- 3-sigma, which can be used to define a tolerance range.
  • the process 312 may continue at block 350 by inspecting a different area of the workpiece 116 surface. Process 312 may continue indefinitely or until a pre-defined condition occurs (e.g., operator intervention).
  • the operator alert described at block 350 may take on various different forms.
  • the operator may be alerted by a series of indicator lamps located proximate to the surface preparation machine, by a message on the HMI 130, by a display on the web browser 134, and/or by an e-mail message on the e-mail client 138.
  • a green indicator light will change to yellow.
  • the lamp may also flash to attract the operator' s attention.
  • the peak count for several different surface inspection areas is outside of the control limits within 60 seconds of one another, the lamp will change from yellow to red.
  • FIG. 6 illustrates a user interface 600 for displaying process data generated by the system 100 of FIG. 1, according to one embodiment. At least three types of data may be displayed, for example, along a timeline 602 that is updated in real-time while the process 300 of FIGS. 3-5 is executing.
  • peak count data 610 represents the number of peaks that are counted in each surface inspection area. Also shown are an upper control limit 612 and a lower control limit 614.
  • the upper and lower control limits 612 and 614 may, for example, represent +/- 3-sigma limits derived during the Teach Mode process 310.
  • R a a calculated average roughness 620
  • R a a quantified representation of the surface profile of the workpiece 116.
  • the upper and lower control limits 622 and 624 may, for example, represent +/- 3-sigma limits derived during the Teach Mode process 310.
  • Yet another type of data that may be displayed in the user interface 600 is a number of peaks per square inch 630. Also shown are an upper control limit 632 and a lower control limit 634.
  • the upper and lower control limits 632 and 634 may, for example, represent +/- 3-sigma limits derived during the Teach Mode process 310.
  • FIGS. 7 A and 7B show one embodiment of a sensing head, such as the sensing head 110 of FIG. 1.
  • FIG. 7 A is a side view and FIG. 7B is a bottom view.
  • the sensing head 110 includes the frame 128 supporting the camera 120, lamps 122, the inclinometer 124, and the proximity sensor 126.
  • the sensing head 110 may be positioned such that the workpiece 116 is
  • the inclinometer 124 may be used to ensure that the above described orientations of the sensing head 110 correspond to the orientation of the workpiece 116.
  • FIG. 6 Various aspects and functions in accordance with the present disclosure, including the user interface 600 shown in FIG. 6, may be implemented as specialized hardware or software executing in one or more special purpose or general purpose computer systems.
  • Such computer systems may include a processor that performs a series of instructions that result in manipulated data.
  • the processor may be a commercially available processor such as an Intel Pentium,
  • the computer system may include specially-programmed, special-purpose hardware, for example, an application-specific integrated circuit (ASIC) tailored to perform a particular operation disclosed herein.
  • ASIC application-specific integrated circuit
  • Other embodiments may perform the same function using a general purpose computing device.
  • the computer system may include an operating system that manages at least a portion of the hardware elements included in the computer system.
  • an operating system which may be, for example, a Windows-based operating system, such as, Windows NT, Windows 2000 (Windows ME), Windows XP or Windows Vista operating systems, available from the Microsoft Corporation, a MAC OS System X operating system available from Apple Computer, one of many Linux-based operating system
  • the Enterprise Linux operating system available from Red Hat Inc.
  • a Solaris operating system available from Sun Microsystems
  • a UNIX operating system available from various sources.
  • Many other operating systems may be used, and the
  • the processor and operating system together define a computer platform for which application programs in high level programming languages may be written.
  • These component applications may be executable, intermediate, for example, C-, bytecode or interpreted code, which communicates over a communication network, for example, the Internet, using a communication protocol, for example, TCP/IP.
  • aspects in accord with the present disclosure may be implemented using an object-oriented programming language, such as .Net, SmallTalk, Java, C++, Ada, or C# (C-Sharp).
  • object-oriented programming languages such as .Net, SmallTalk, Java, C++, Ada, or C# (C-Sharp).
  • Other object-oriented programming languages may also be used.
  • functional, scripting, or logical programming languages may be used.
  • various aspects and functions in accordance with the present disclosure may be implemented in a non-programmed environment, for example, documents created in HTML, XML or other format that, when viewed in a window of a browser program, render aspects of a graphical-user interface or perform other functions.
  • various embodiments in accord with the present disclosure may be implemented as programmed or non-programmed elements, or any combination thereof.
  • a web page may be implemented using HTML while a data object called from within the web page may be written in C++.
  • the disclosure is not limited to a specific programming language and any suitable programming language could also be used.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Length Measuring Devices With Unspecified Measuring Means (AREA)

Abstract

A surface measurement method includes inspecting a plurality of samples of a first irregular workpiece surface in two dimensions, determining, based on the inspections of the samples, a total number of surface peaks on each of the plurality of samples, and deriving a control limit from a statistical variation of the total number of surface peaks on each of the samples. The control limit specifies an out-of-tolerance condition for the total number of surface peaks on a second irregular workpiece surface. The method further includes inspecting a portion of the second irregular workpiece surface in two dimensions, determining, based on the inspection of the portion of the second irregular workpiece surface, a total number of surface peaks on the portion, and comparing the total number of surface peaks on the portion to the control limit to determine whether the second irregular workpiece surface is in the out-of- tolerance condition.

Description

SURFACE MEASUREMENT SYSTEM AND METHOD
BACKGROUND OF THE INVENTION
1. Field of the Disclosure
This disclosure relates generally to the field of abrasive surface preparation, and more particularly to methods and apparatus for measuring the profile of a prepared surface.
2. Discussion of Related Art
Abrasive surface preparation is a process used to clean or condition the surface of a material, typically metal, to strengthen it, prepare it for receiving a coating, such as paint, or for other purposes. In one conventional technique, an abrasive media is blasted onto the surface under high pressure. The resulting profile of the prepared surface (i.e., the three-dimensional variations of the surface from a perfectly smooth or "true" plane) describes the relative roughness of the surface, and is known to be a factor in the performance of various coatings applied to metals, such as steel. One existing standard for quantifying surface irregularities is described by ANSI B46.1-2009: Surface Texture, Surface Roughness, Waviness and Lay (American Society of Mechanical Engineers, August 20, 2010, ISBN 9780791832622).
Several known techniques for measuring surface profiles are described, for example, in ASTM International Book of Standards, Volume 06.02, Paint - Products and Applications; Protective Coatings; Pipeline Coatings, D4417 (Methods A, B and C). In ASTM D4417 Method A, a visual comparator (e.g., a template) is viewed in comparison with the surface being measured, usually with a magnifier. In Method B, a gauge having a needle or stylus is laid on top of the surface at various locations to measure the distance between the peaks and valleys of the surface. In Method C, a tape having a compressible foam layer is pressed onto a portion of the surface to make an impression of the profile, which is then measured with a spring micrometer.
Other known techniques are described in, for example, U.S. Patent Application
Publication No. 2008/0240510, entitled "Method and System for Examining a Surface;" U.S. Patent No. 5,581,483, entitled "Measurement of Shot Peening Coverage by Correlation Analysis of Surface Line Data;" U.S. Patent No. 5,814,722, entitled "System for Measurement of Peaks on Surfaces;" U.S. Patent No. 4,126,036, entitled "Surface Roughness Analyzer;" and U.S. Patent No. 3,123,999, entitled "Apparatus for Measuring Surface Roughness Peaks." BRIEF SUMMARY OF THE INVENTION
According to one embodiment of the present disclosure, a surface measurement method includes inspecting a plurality of samples of a first irregular workpiece surface in two dimensions, determining, based on the inspections of the samples, a total number of surface peaks on each of the plurality of samples, and deriving a control limit from a statistical variation of the total number of surface peaks on each of the samples. The control limit specifies an out- of-tolerance condition for the total number of surface peaks on a second irregular workpiece surface. The method further includes inspecting a portion of the second irregular workpiece surface in two dimensions, determining, based on the inspection of the portion of the second irregular workpiece surface, a total number of surface peaks on the portion, and comparing the total number of surface peaks on the portion to the control limit to determine whether the second irregular workpiece surface is in the out-of-tolerance condition.
In another embodiment, inspecting the samples of the first irregular workpiece surface in two dimensions may include optically inspecting the samples of the first irregular workpiece surface in two dimensions. Inspecting the portion of the second irregular workpiece surface in two dimensions may include optically inspecting the portion of the second irregular workpiece surface in two dimensions.
In another embodiment, the method may include alerting an operator in response to determining that the second irregular workpiece surface is in the out-of-tolerance condition. In yet another embodiment, alerting the operator may include at least one of illuminating a lighted warning indicator, sounding an audible alarm, and displaying an alert message in a user interface of an operator terminal.
In another embodiment, the method may include measuring an average height of the surface peaks on at least one of the plurality of samples, and correlating the total number of surface peaks on each of the plurality of samples to the average height such that a surface profile of the second irregular workpiece surface can be determined based on the total number of surface peaks on the portion of the second irregular workpiece surface. In yet another embodiment, the method may include determining the surface profile of the second irregular workpiece surface based on the correlation, wherein the surface profile represents an average roughness Ra of the second irregular workpiece surface. In another embodiment, the plurality of samples may include at least 100 samples of the first irregular workpiece surface. In yet another embodiment, the size of each of the samples may be approximately 0.04 inches square. In yet another embodiment, the control limit may be derived from a +/-3 sigma statistical variation of the total number of surface peaks on each of the plurality of samples.
According to one embodiment of the present disclosure, a surface measurement apparatus includes a sensing unit and a controller coupled to the sensing unit. The controller is configured to inspect, using the sensing unit, a plurality of samples of a first irregular workpiece surface in two dimensions; determine, based on the inspections of the samples, a total number of surface peaks on each of the samples; derive a control limit from a statistical variation of the total number of surface peaks on each of the samples, the control limit for specifying an out-of- tolerance condition for the total number of surface peaks on a second irregular workpiece surface; inspect, using the sensing unit, a portion of the second irregular workpiece surface in two dimensions; determine, based on the inspection of the portion, a total number of surface peaks on the portion; and compare the total number of surface peaks on the portion to the control limit to determine whether the second irregular workpiece surface is in the out-of-tolerance condition. Each of the plurality of samples has substantially the same size, and the portion has substantially the same size as one of the plurality of samples.
In another embodiment, the sensing unit may include a camera configured to capture an image of the portion of the first irregular workpiece surface and an image of the portion of the second irregular workpiece surface. In yet another embodiment, the sensing unit may include an inclinometer configured to determine an angle of the camera with respect to the portion of the irregular workpiece surface. In yet another embodiment, the sensing unit may include a proximity sensor configured to determine a distance between the camera and the first irregular workpiece surface, the second irregular workpiece surface, or both. In yet another embodiment, the sensing unit may include an illumination device configured to illuminate the first irregular workpiece surface, the second irregular workpiece surface, or both.
In another embodiment, the apparatus may include a user interface coupled to the controller. The user interface may be configured to alert an operator in response to the controller determining that the second irregular workpiece surface is in the out-of-tolerance condition. In yet another embodiment, the user interface may include a lighted warning indicator and/or an audible alarm. In another embodiment, the controller may be configured to measure an average height of the surface peaks on at least one of the samples, and correlate the total number of surface peaks on each of the samples to the average height such that a surface profile of the second irregular workpiece surface can be determined based on the total number of surface peaks on the portion of the second irregular workpiece surface. In yet another embodiment, the controller may be configured to determine the surface profile of the second irregular workpiece surface based on the correlation, wherein the surface profile represents an average roughness Ra of the second irregular workpiece surface. In yet another embodiment, the size of each of the samples of the first irregular workpiece surface may be approximately 0.04 inches square. In yet another embodiment, the sensing unit may be configured to enable a measurement of a height of at least one of the surface peaks of the first irregular workpiece surface.
BRIEF DESCRIPTION OF THE DRAWINGS
The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures is represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. In the drawings:
FIG. 1 is a block diagram of a surface profile measurement system in accordance with one embodiment of the present disclosure;
FIG. 2A is top view of a portion of a workpiece surface to be inspected in accordance with one embodiment of the present disclosure;
FIG. 2B is a cross-sectional elevation view taken along a cut line B-B' of FIG. 2A;
FIG. 2C is a cross-sectional elevation view taken along a cut line C-C of FIG. 2B;
FIG. 3 is a flow diagram of a surface measurement process in accordance with one embodiment of the present disclosure;
FIG. 4 is another flow diagram, showing details of the surface measurement process of FIG. 3, in accordance with one embodiment of the present disclosure;
FIG. 5 is yet another flow diagram, showing further details of the surface measurement process of FIG. 3, in accordance with one embodiment of the present disclosure;
FIG. 6 illustrates a user interface for displaying process data generated by the system of FIG. 1 in accordance with one embodiment of the present disclosure; and FIGS. 7A and 7B are side elevation and bottom views, respectively, of a sensing head device in accordance with one embodiment of the present disclosure.
DETAILED DESCRIPTION OF THE INVENTION
For the purposes of illustration only, and not to limit the generality, the present disclosure will now be described in detail with reference to the accompanying figures. This disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The principles set forth in this disclosure are capable of other embodiments and of being practiced or carried out in various ways. Also the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," "having," "containing," "involving," and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.
As discussed above, several techniques for measuring the profile of a surface are known. However, according to various embodiments of the present disclosure, it is appreciated that these known techniques are not well-suited for use in real-time production. For example, one disadvantage of the visual comparator is that, because not all surface profiles are the same, a highly experienced inspector must account for any differences, either real or perceived, between the comparator tool and the surface under inspection. Such inspection can be highly subjective to the inspector, and there is no analytical data collected from the inspection. In another example, one disadvantage of the handheld gauge with needle or stylus is that several measurements at different points on the surface must be made and averaged together to provide an accurate reading. This technique is also prone to error if the gauge is not precisely used. In yet another example, one disadvantage of the foam tape is that it can be costly and time consuming to take a sufficient number of surface measurements.
It is appreciated, therefore, that none of these known techniques takes advantage of a predictable correlation between a two-dimensional measurement of the surface and a statistical model that can be generated based on a relatively small number of control samples of the material, as disclosed herein. For example, according to one embodiment of the present disclosure, a non-contact measurement of a surface profile can be acquired through an optical inspection of the surface. Such an inspection can be made using, for example, a machine vision system. A correlation can be made, based on known properties of the material when subjected to a surface preparation technique, between the roughness of the workpiece surface and the number of peaks observed on the surface. Further, because the average peak-to-peak distance may be used to describe the profile of a surface, the correlation may be utilized to associate a peak count with the average peak-to-peak distance, and thus the average surface roughness (as quantified according to, e.g., ANSI standards). It is further appreciated that embodiments of the methods and apparatus disclosed herein can be automated for fast processing and used in a real-time production process with a relatively small amount of human intervention.
FIG. 1 illustrates a block diagram of a surface measurement system 100 in accordance with one embodiment of the present disclosure. The system 100 includes a sensing head 110, a controller 112, and one or more user interface systems 114. During operation, the sensing head 110 is positioned near a workpiece 116 (such as a piece of metal), which may move in relation to the sensing head 110 (e.g., on a conveyor). The sensing head 110 includes one or more of the following: a camera 120, one or more lamps 122 or other illumination devices, an inclinometer 124, and a proximity sensor 126 (e.g., an inductive proximity sensor, a double lensed LED ranging sensor, a laser ranging sensor, or other device configured to detect the presence and/or absence of a metal surface). Depending on the distance between the proximity sensor 126 and the workpiece 210, different ranging technology can be used. For example, an inductive proximity sensor is suitable for use when the distance is approximately one inch, and a double lensed LED or laser ranging sensor is suitable for use when the distance is approximately three inches. In some embodiments, the camera 120 and/or lamps 122 are incorporated within a conventional machine vision system having a separate programmable controller, as will be understood by one of skill in the art.
Each component of the sensing head 110 may be physically attached to a frame 128 or other supporting structure such that the camera 120, the lamps 122, the inclinometer 124, and/or the proximity sensor 126 remain in substantially fixed spatial relation and orientation to each other at all times. It should be understood that each element of the sensing head 110 may include an attachment feature that allows for adjusting the respective positions during, for example, setup or calibration procedures. The sensing head 110 is coupled to the controller 112 through one or more communication interfaces 118 that provide for the exchange of, among other things, control and monitor data (e.g., camera control, image acquisition, lighting control, sensor monitoring, and so forth). Additional interfaces (not shown) may also be included in the system 100 for providing power to the respective elements of the sensing head 110. The camera 120 can be configured to inspect and identify features of the workpiece 116, including variations and irregularities of the workpiece surface, and to communicate data relating to the workpiece to the controller 112. For example, the camera 120 may be configured to acquire images of portions of the workpiece surface 210 (e.g., each portion having an area of approximately 0.04 square inches). As the sensing head 110 and/or the workpiece 116 move relative to one another, the camera 120 may acquire images of different portions of the surface 210. Thus, according to at least one embodiment, the system 100 can be used to measure the surface 210 in real-time as the workpiece 116 undergoes surface preparation. In one
embodiment, the camera 120 is configured to detect surface variations at a resolution of at least approximately 11 microns or approximately 0.5 mills.
As described above, in at least one embodiment, the system 100 includes a non-contact surface measurement apparatus. The inclinometer 124 and the proximity sensor 126 enable the system 100 to identify the orientation of the sensing head 110 with respect to the workpiece 116 without physically touching the workpiece. The inclinometer 124 provides data to the controller 112 relating to the angle of the sensing head 110 with respect to gravity. This data is useful for, among other things, determining whether the sensing head 110 is correctly aligned with respect to the workpiece 116 (e.g., parallel to the surface). The proximity sensor 126 provides data to the controller 112 relating to the distance between the sensing head 110 and the surface of the workpiece 116. This data is also useful for alignment purposes.
The user interface system 114 is coupled to the controller 112. The user interface system
114 may include various types of devices and/or systems that are configured to communicate information to and from the controller 112, and enable one or more users of the system 100 to control, monitor, and otherwise interact with the controller. Examples of user interfaces that may be utilized include a Human-Machine Interface (HMI) (e.g., a graphical HMI such as a Fuji HMI - V815X, sold by Fuji Electric Corporation of America, Fremont, Calif.), a web server 132 and web browser 134, and/or an electronic mail (e-mail) server 136 and e-mail client 138. In one embodiment, the HMI executes software that enables secure communications (e.g.,
TELLUS and V-Server software, also sold by Fuji Electric Corp. of America). In one embodiment, a data recorder 140 may be coupled to the controller 112 for capturing and storing data generated by the system 100, such as data associated with the surface profile measurements. Such data may be subsequently retrieved by a user for further analysis or reporting purposes. According to one embodiment, during normal operation of the system 100 the workpiece 116 undergoes a form of surface preparation, such as blasting, that is performed prior to inspection and measurement. Such preparation manipulates the surface of the workpiece 116 and causes the surface to acquire a profile that can be visualized as a set of peaks and valleys. Peaks are typically points of the surface that, when viewed in cross-section, are higher than an imaginary plane that is substantially parallel to the surface. The imaginary plane may be, for example, located at an average height of the surface between the highest and lowest points, or at another elevation. Likewise, valleys are typically points of the surface that are lower than the imaginary plane. Therefore, if an observer inspects the surface of the workpiece 116 at a substantially perpendicular angle to the imaginary plane (e.g., as viewed from above), the peaks will be closer to the observer than the valleys. Likewise, the highest peaks will be closer to the observer than the lowest peaks.
According to one embodiment, the system 100 can be used to inspect and measure the surface of the workpiece 116, for example, to determine if the surface profile is within desired specifications, control limits, or tolerances by identifying peaks and/or valleys on the surface.
The system 100 may further be configured to identify surface variations and irregularities caused by other conditions occurring during surface preparation, such as vibration, temperature, wheel speed, and pressure. As will now be described, the peaks and valleys, which are identifiable by the camera 120 (or by a system that is configured to analyze the images generated by the camera), can be used to describe a characteristic of the workpiece 116 surface, such as roughness. Roughness can be quantified by the deviations of the surface from a perfectly smooth form. Large deviations may correspond to high roughness.
In one embodiment, it is appreciated that the surface profile of a particular material (e.g., steel or iron) will have predictable characteristics when subjected to preparation such as blasting. That is, the nature of the material is such that the surface can only be manipulated or distorted to a certain extent before it fractures, tears, or resists further manipulation by the surface preparation process (also referred to as overworking the surface). Therefore, the maximum slope between any adjacent peak and valley (or adjacent peaks, or adjacent valleys), which is the rate at which the elevation of the surface changes with respect to distance along the imaginary plane, can be calculated with a level of confidence that is sufficient for determining the average expected distance between any adjacent peaks and valleys for a given average roughness, or Ra. Accordingly, within any given area of the surface under inspection, there will be a predictable number of peaks present for a corresponding average roughness. Because there is a predictable relationship between the surface profile (e.g., roughness), the average distance between peaks, and the number of peaks on the portion of the surface being inspected, the surface profile of the workpiece 116 can be determined by counting the number of surface peaks found within a certain two dimensional view of the inspection area. For example, if the inspection area contains a high number of peaks, it can be said that the surface has a relatively rough (or high) profile (and a relatively small peak-to-peak distance), whereas if the number of peaks is relatively low, it can be said that the surface has a relatively smooth (or low) profile (and a relatively large peak-to-peak distance).
FIG. 2A is a two-dimensional top view a portion of an exemplary workpiece surface 210, such as would be observed by the camera 120 of FIG. 1. The area under inspection is x by y; although typically the area will be square, it need not be so. For example, the inspection area may be approximately 0.04 square inches (i.e., 0.2 inches by 0.2 inches). With the use of illumination and a camera or magnifier, such as described above with respect to the system 110 of FIG. 1, variations and irregularities in the workpiece surface 210 can be observed, for example, by a machine vision system, or by a human operator. Appearing on an image of the workpiece surface 210 are one or more surface peaks at or above a threshold height h, each indicated at 212. FIG. 2B shows a cross-section of the workpiece surface 210 along a cut line B- B' showing the corresponding peaks 212 along the cross-section. FIG. 2C shows another cross- section of the workpiece surface 210 along a cut line C-C showing the corresponding peaks 212 along the cross-section. Each peak 212 may be identified by inspection based on the height of the peak relative to other points on the surface using known image processing techniques. In other words, the threshold height h may be established at or above which each peak is identified and counted and below which no peak is observed or counted.
According to one embodiment, the threshold height can be established as the average height of the surface above the lowest point, although it should be understood that other factors may be used to determine the threshold height. For example, some peaks may be insufficiently high for properly characterizing the surface profile, and therefore should be discounted or ignored. In the example workpiece surface 210 of FIG. 2A, there are seven peaks, each identified at 212, at or above the threshold height.
FIG. 3 is a flow diagram of a surface measurement process 300, according to one embodiment. The system 100 of FIG. 1 can operate in at least two modes: Teach Mode 310 and Run Mode 312, each of which are described in further detail below with respect to FIGS. 4 and 5, respectively. Initially, once the sensing head 110 is installed and configured, the system 100 enters Teach Mode 310. During or prior to Teach Mode 310, the surface of the workpiece 116 is prepared, such as by blasting or peening, under controlled conditions prior to measurement by the system 100. The controls may include separate, periodic inspections (e.g., by an operator) to ensure that the process is continuously under control; if the process is out of control, the Teach Mode 310 may need to be restarted. During Teach Mode 310, the prepared surface is inspected, either automatically or by hand, as part of a configuration procedure that "teaches" the system 100 various measurement parameters that are subsequently used for real-time measurement of the workpiece 116. Once the system 100 has completed Teach Mode 310, the system 100 enters Run Mode 312.
FIG. 4 is a flow diagram of the surface measurement process 300, described above with respect to FIG. 3, while operating in Teach Mode 310, according to one embodiment. At block 320, a sample of the workpiece 116 surface is inspected. The sample should have substantially the same area as, or sufficiently represent, each subsequent sample. The camera 120 acquires an image of the sample, and sends it to the controller 112 for analysis. It should be understood that in some embodiments, the camera 120 can be part of a machine vision system, which may include a separate controller for analyzing the image; in some other embodiments, image processing can be performed by the controller 112, or by a similar processing device.
At block 322, the controller 112 identifies and counts each peak on the sample. Blocks
320 and 322 may be repeated for any number of different samples n (i.e., one or more samples), as indicated at block 324. For example, one hundred (100) samples may be analyzed during the Teach Mode 310 process. The number of samples that should be taken in Teach Mode 310 to achieve desirable results may be determined by statistical analysis.
At block 326, the standard deviation and the average of the number of peaks counted in each sample is calculated. At block 328, one or more control limits are calculated based on the standard deviation, the average, or both. For example, a statistical model may be derived based on the standard deviation, the average, or both, in which a +/- 3-sigma deviation may be calculated, as should be understood by one of skill in the art. The control limit(s) may be based on the corresponding +/- 3-sigma values. In another example, a statistical process control is applied around a known variable (e.g., related to a customer part number), which includes all known process variations associated with the variable. This enables a mathematical model of process control limits (e.g., that are used during a surface preparation process) to be built that accounts for the inherent variations of the prepared surface of a particular part. For example, a wide variation will cause the control points to be widely spread. The mathematical model also enables a user to determine how capable and repeatable the surface preparation process will be. The control limits will be used during the Run Mode 312, as described below.
FIG. 5 is a flow diagram of the surface measurement process 300, described above with respect to FIG. 3, while operating in Run Mode 312, according to one embodiment. At block 350, an area of the workpiece 116 surface is inspected. The area should have substantially the same area as or representative of each of the samples inspected during the Teach Mode process 310. The camera 120 acquires an image of the sample, and sends the image to the controller 112 (or other processor) for analysis. At block 352, each surface peak in the image is identified and counted. At block 354, the number of peaks counted within the inspection area is compared to one or more of the previously derived control limits, for example, +/- 3-sigma, which can be used to define a tolerance range. At block 356, if the number of peaks is outside of the control limits (e.g., it is above an upper limit or below a lower limit), the workpiece 116 is in an out-of- tolerance condition and an operator is alerted (block 358). Otherwise, the process 312 may continue at block 350 by inspecting a different area of the workpiece 116 surface. Process 312 may continue indefinitely or until a pre-defined condition occurs (e.g., operator intervention).
It should be understood that the operator alert described at block 350 may take on various different forms. For example, the operator may be alerted by a series of indicator lamps located proximate to the surface preparation machine, by a message on the HMI 130, by a display on the web browser 134, and/or by an e-mail message on the e-mail client 138. In one example, if the peak count for a single surface inspection area is outside of the control limits, a green indicator light will change to yellow. The lamp may also flash to attract the operator' s attention. In another example, if the peak count for several different surface inspection areas is outside of the control limits within 60 seconds of one another, the lamp will change from yellow to red.
Additionally, an audible alarm may sound to further attract the operator' s attention. Other contingency responses may be automatically performed, for example, to automatically stop the surface preparation machine. Once alerted, the operator may take subsequent action to determine whether the workpiece 116 surface is in an out-of-tolerance condition, such as by hand measurements using other techniques, or by re-calibration of the system 100. FIG. 6 illustrates a user interface 600 for displaying process data generated by the system 100 of FIG. 1, according to one embodiment. At least three types of data may be displayed, for example, along a timeline 602 that is updated in real-time while the process 300 of FIGS. 3-5 is executing. One type of data that may be displayed in the user interface 600 is peak count data 610, which represents the number of peaks that are counted in each surface inspection area. Also shown are an upper control limit 612 and a lower control limit 614. The upper and lower control limits 612 and 614 may, for example, represent +/- 3-sigma limits derived during the Teach Mode process 310.
Another type of data that may be displayed in the user interface 600 is a calculated average roughness 620, or Ra, that is a quantified representation of the surface profile of the workpiece 116. Also shown are an upper control limit 622 and a lower control limit 624. The upper and lower control limits 622 and 624 may, for example, represent +/- 3-sigma limits derived during the Teach Mode process 310.
Yet another type of data that may be displayed in the user interface 600 is a number of peaks per square inch 630. Also shown are an upper control limit 632 and a lower control limit 634. The upper and lower control limits 632 and 634 may, for example, represent +/- 3-sigma limits derived during the Teach Mode process 310.
FIGS. 7 A and 7B show one embodiment of a sensing head, such as the sensing head 110 of FIG. 1. FIG. 7 A is a side view and FIG. 7B is a bottom view. The sensing head 110 includes the frame 128 supporting the camera 120, lamps 122, the inclinometer 124, and the proximity sensor 126. The sensing head 110 may be positioned such that the workpiece 116 is
substantially parallel to the bottom of the sensing head such that the camera is aimed toward the surface 210 of the workpiece 116 and the proximity sensor is substantially parallel to the surface. The inclinometer 124 may be used to ensure that the above described orientations of the sensing head 110 correspond to the orientation of the workpiece 116.
Various aspects and functions in accordance with the present disclosure, including the user interface 600 shown in FIG. 6, may be implemented as specialized hardware or software executing in one or more special purpose or general purpose computer systems. Such computer systems may include a processor that performs a series of instructions that result in manipulated data. The processor may be a commercially available processor such as an Intel Pentium,
Motorola PowerPC, SGI MIPS, Sun UltraSPARC, or Hewlett-Packard PARISC processor, but may be any type of processor or controller as many other processors and controllers are available. The computer system may include specially-programmed, special-purpose hardware, for example, an application-specific integrated circuit (ASIC) tailored to perform a particular operation disclosed herein. Other embodiments may perform the same function using a general purpose computing device.
The computer system may include an operating system that manages at least a portion of the hardware elements included in the computer system. Usually, a processor or controller executes an operating system which may be, for example, a Windows-based operating system, such as, Windows NT, Windows 2000 (Windows ME), Windows XP or Windows Vista operating systems, available from the Microsoft Corporation, a MAC OS System X operating system available from Apple Computer, one of many Linux-based operating system
distributions, for example, the Enterprise Linux operating system available from Red Hat Inc., a Solaris operating system available from Sun Microsystems, or a UNIX operating system available from various sources. Many other operating systems may be used, and the
embodiments disclosed herein are not intended to be limited to any particular implementation.
The processor and operating system together define a computer platform for which application programs in high level programming languages may be written. These component applications may be executable, intermediate, for example, C-, bytecode or interpreted code, which communicates over a communication network, for example, the Internet, using a communication protocol, for example, TCP/IP. Similarly, aspects in accord with the present disclosure may be implemented using an object-oriented programming language, such as .Net, SmallTalk, Java, C++, Ada, or C# (C-Sharp). Other object-oriented programming languages may also be used. Alternatively, functional, scripting, or logical programming languages may be used.
Additionally, various aspects and functions in accordance with the present disclosure may be implemented in a non-programmed environment, for example, documents created in HTML, XML or other format that, when viewed in a window of a browser program, render aspects of a graphical-user interface or perform other functions. Further, various embodiments in accord with the present disclosure may be implemented as programmed or non-programmed elements, or any combination thereof. For example, a web page may be implemented using HTML while a data object called from within the web page may be written in C++. Thus, the disclosure is not limited to a specific programming language and any suitable programming language could also be used. Having thus described several aspects of at least one embodiment of this disclosure, it is to be appreciated various alterations, modifications, and improvements will readily occur to those skilled in the art. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the spirit and scope of the disclosure. Accordingly, the foregoing description and drawings are by way of example only.
What is claimed is:

Claims

1. A surface measurement method, comprising:
inspecting a plurality of samples of a first irregular workpiece surface in two dimensions, each of the plurality of samples having substantially the same size;
determining, based on the inspections of the samples, a total number of surface peaks on each of the plurality of samples;
deriving a control limit from a statistical variation of the total number of surface peaks on each of the plurality of samples, the control limit for specifying an out-of-tolerance condition for the total number of surface peaks on a second irregular workpiece surface;
inspecting a portion of the second irregular workpiece surface in two dimensions, the portion having substantially the same size as one of the plurality of samples;
determining, based on the inspection of the portion of the second irregular workpiece surface, a total number of surface peaks on the portion; and
comparing the total number of surface peaks on the portion to the control limit to determine whether the second irregular workpiece surface is in the out-of-tolerance condition.
2. The method of claim 1, wherein inspecting the plurality of samples of the first irregular workpiece surface in two dimensions includes optically inspecting the plurality of samples of the first irregular workpiece surface in two dimensions, and wherein inspecting the portion of the second irregular workpiece surface in two dimensions includes optically inspecting the portion of the second irregular workpiece surface in two dimensions.
3. The method of claim 1, further comprising alerting an operator in response to determining that the second irregular workpiece surface is in the out-of-tolerance condition.
4. The method of claim 3, wherein alerting the operator includes at least one of illuminating a lighted warning indicator, sounding an audible alarm, and displaying an alert message in a user interface of an operator terminal.
5. The method of claim 1, further comprising measuring an average height of the surface peaks on at least one of the plurality of samples, and correlating the total number of surface peaks on each of the plurality of samples to the average height such that a surface profile of the second irregular workpiece surface can be determined based on the total number of surface peaks on the portion of the second irregular workpiece surface.
6. The method of claim 5, further comprising determining the surface profile of the second irregular workpiece surface based on the correlation, wherein the surface profile represents an average roughness Ra of the second irregular workpiece surface.
7. The method of claim 1, wherein the plurality of samples includes at least 100 samples of the first irregular workpiece surface.
8. The method of claim 1, wherein the size of each of the samples is approximately 0.04 inches square.
9. The method of claim 1, wherein the control limit is derived from a +/-3 sigma statistical variation of the total number of surface peaks on each of the plurality of samples.
10. A surface measurement apparatus, comprising:
a sensing unit; and
a controller coupled to the sensing unit and configured to:
inspect, using the sensing unit, a plurality of samples of a first irregular workpiece surface in two dimensions, each of the plurality of samples having substantially the same size;
determine, based on the inspections of the samples, a total number of surface peaks on each of the plurality of samples;
derive a control limit from a statistical variation of the total number of surface peaks on each of the plurality of samples, the control limit for specifying an out-of- tolerance condition for the total number of surface peaks on a second irregular workpiece surface;
inspect, using the sensing unit, a portion of the second irregular workpiece surface in two dimensions, the portion having substantially the same size as one of the plurality of samples; determine, based on the inspection of the portion, a total number of surface peaks on the portion; and
compare the total number of surface peaks on the portion to the control limit to determine whether the second irregular workpiece surface is in the out-of-tolerance condition.
11. The apparatus of claim 10, wherein the sensing unit includes a camera configured to capture an image of the portion of the first irregular workpiece surface and an image of the portion of the second irregular workpiece surface.
12. The apparatus of claim 11, wherein the sensing unit includes an inclinometer configured to determine an angle of the camera with respect to the portion of the irregular workpiece surface.
13. The apparatus of claim 11, wherein the sensing unit includes a proximity sensor configured to determine a distance between the camera and at least one of the first irregular workpiece surface and the second irregular workpiece surface.
14. The apparatus of claim 11, wherein the sensing unit includes an illumination device configured to illuminate at least one of the first irregular workpiece surface and the second irregular workpiece surface.
15. The apparatus of claim 10, further comprising a user interface coupled to the controller configured to alert an operator in response to the controller determining that the second irregular workpiece surface is in the out-of-tolerance condition.
16. The apparatus of claim 16, wherein the user interface includes a lighted warning indicator and an audible alarm.
17. The apparatus of claim 10, wherein the controller is further configured to measure an average height of the surface peaks on at least one of the plurality of samples, and correlate the total number of surface peaks on each of the plurality of samples to the average height such that a surface profile of the second irregular workpiece surface can be determined based on the total number of surface peaks on the portion of the second irregular workpiece surface.
18. The apparatus of claim 17, wherein the controller is further configured to determine the surface profile of the second irregular workpiece surface based on the correlation, wherein the surface profile represents an average roughness Ra of the second irregular workpiece surface.
19. The apparatus of claim 10, wherein the size of each of the samples of the first irregular workpiece surface is approximately 0.04 inches square.
20. The apparatus of claim 10, wherein the sensing unit is configured to enable a measurement of a height of at least one of the surface peaks of the first irregular workpiece surface.
EP12804202.5A 2011-06-29 2012-06-22 Surface measurement system and method Not-in-force EP2726813B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13/171,516 US8620033B2 (en) 2011-06-29 2011-06-29 Surface measurement system and method
PCT/US2012/043723 WO2013003222A1 (en) 2011-06-29 2012-06-22 Surface measurement system and method

Publications (3)

Publication Number Publication Date
EP2726813A1 true EP2726813A1 (en) 2014-05-07
EP2726813A4 EP2726813A4 (en) 2015-01-28
EP2726813B1 EP2726813B1 (en) 2016-11-30

Family

ID=47390077

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12804202.5A Not-in-force EP2726813B1 (en) 2011-06-29 2012-06-22 Surface measurement system and method

Country Status (9)

Country Link
US (1) US8620033B2 (en)
EP (1) EP2726813B1 (en)
CN (1) CN103620338B (en)
BR (1) BR112013032808A2 (en)
CA (1) CA2834401C (en)
MX (1) MX2013013352A (en)
MY (1) MY173286A (en)
RU (1) RU2608326C2 (en)
WO (1) WO2013003222A1 (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8868231B2 (en) * 2011-08-10 2014-10-21 Xerox Corporation Flexible production collating system
JP5851813B2 (en) * 2011-12-05 2016-02-03 三菱重工業株式会社 Curvature holding device, curving method and curving method for plate-like workpiece
US9242251B2 (en) 2013-01-30 2016-01-26 Wheelabrator Group, Inc. Magnetic separator with dynamic baffle system
WO2014155727A1 (en) * 2013-03-29 2014-10-02 株式会社牧野フライス製作所 Workpiece worked surface evaluation method, control device, and working machine
KR102656197B1 (en) 2015-11-18 2024-04-09 삼성전자주식회사 A movable apparatus, a robot cleaner, a device for determining a condition of a surface of floor, a method for controlling the movable apparatus and a method for controlling the robot cleaner
CA3073313A1 (en) 2017-08-24 2019-02-28 Stepan Company Hardness and flatness tester
US11009485B2 (en) 2018-09-12 2021-05-18 Framatome Inc. Method of identifying and removing surface irregularities before ultrasonic inspection and device for identifying surface irregularities
US20220326111A1 (en) * 2021-04-08 2022-10-13 Pavement Recycling Systems Inc. Automated Analysis and Sorting of Milling Drum Tools
CN114535624B (en) * 2022-01-19 2023-08-29 南昌大学 Method for reducing influence of servo vibration on ultra-precision machining of micro lens array

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4126036A (en) * 1977-07-27 1978-11-21 United States Steel Corporation Surface roughness analyzer
US5814722A (en) * 1997-01-17 1998-09-29 Eastman Kodak Company System for measurement of peaks on surfaces
JP2006124824A (en) * 2004-09-28 2006-05-18 Nippon Steel Corp HIGHLY CORROSION RESISTANT Zn ALLOY PLATED STEEL MATERIAL HAVING HAIRLINE APPEARANCE
US20100136362A1 (en) * 2006-11-03 2010-06-03 Severstal Sparrows Point Llc Method and apparatus for polishing an aluminum-zinc alloy hot-dip coating and the product therefrom

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3123999A (en) 1964-03-10 Amplifier-
DE1245605B (en) 1964-10-20 1967-07-27 Janez Peklenik Dr Ing Method and device for determining parameters in stochastic processes
US4145140A (en) * 1977-06-15 1979-03-20 Hitoshi Fujii Measurement of surface roughness
US4914828A (en) * 1988-08-09 1990-04-10 Ppg Industries, Inc. Surface inspection device and method
US5581483A (en) 1994-09-19 1996-12-03 General Electric Company Measurement of shot peening coverage by correlation analysis of surface line data
US5873770A (en) * 1996-07-22 1999-02-23 The Timken Company Vibratory finishing process
US6249348B1 (en) 1998-11-23 2001-06-19 Lj Laboratories, L.L.C. Integrated spectrometer assembly and methods
US6507393B2 (en) * 1999-05-12 2003-01-14 John Samuel Batchelder Surface cleaning and particle counting
US6264198B1 (en) * 1999-06-29 2001-07-24 Rare Limited Method, system and computer-readable medium for a moving video image jigsaw puzzle game
MY118582A (en) 2000-05-12 2004-12-31 Kao Corp Polishing composition
JP2004128069A (en) * 2002-09-30 2004-04-22 Fujimi Inc Grinder composition and grinding method using it
CN100394162C (en) * 2003-01-23 2008-06-11 友达光电股份有限公司 Method for analyzing roughness on surface of film
US20080240510A1 (en) 2006-09-30 2008-10-02 Greg Dale Method and system for examining a surface
CN101819030B (en) * 2009-02-27 2012-05-30 北京京东方光电科技有限公司 Method and system for monitoring surface roughness of magnetic control spattering target

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4126036A (en) * 1977-07-27 1978-11-21 United States Steel Corporation Surface roughness analyzer
US5814722A (en) * 1997-01-17 1998-09-29 Eastman Kodak Company System for measurement of peaks on surfaces
JP2006124824A (en) * 2004-09-28 2006-05-18 Nippon Steel Corp HIGHLY CORROSION RESISTANT Zn ALLOY PLATED STEEL MATERIAL HAVING HAIRLINE APPEARANCE
US20100136362A1 (en) * 2006-11-03 2010-06-03 Severstal Sparrows Point Llc Method and apparatus for polishing an aluminum-zinc alloy hot-dip coating and the product therefrom

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO2013003222A1 *

Also Published As

Publication number Publication date
US20130002444A1 (en) 2013-01-03
CA2834401A1 (en) 2013-01-03
EP2726813B1 (en) 2016-11-30
WO2013003222A1 (en) 2013-01-03
RU2608326C2 (en) 2017-01-17
BR112013032808A2 (en) 2017-01-31
MY173286A (en) 2020-01-11
CA2834401C (en) 2019-04-30
CN103620338B (en) 2017-02-15
RU2014102719A (en) 2015-08-10
CN103620338A (en) 2014-03-05
US8620033B2 (en) 2013-12-31
MX2013013352A (en) 2014-11-26
EP2726813A4 (en) 2015-01-28

Similar Documents

Publication Publication Date Title
CA2834401C (en) Surface measurement system and method
US7495758B2 (en) Apparatus and methods for two-dimensional and three-dimensional inspection of a workpiece
KR20150038693A (en) Method and device for inspecting an object for the detection of surface damage
CN107389316B (en) Display panel testing device and display panel testing method
US10895450B2 (en) Systems and methods for gathering data and information on surface characteristics of an object
JP2019533158A (en) Method for monitoring the functional status of a system for computed tomography inspection of a workpiece
Baleani et al. Analysis of reproducibility and repeatability of a hand-held laser scanner for gap&flush measurement in car-assembly line
JP2021071480A (en) Method and device for inspecting surface defect, method for manufacturing steel plate, method for managing quality of steel plate, and steel plate manufacturing facility
CN106197515B (en) A kind of detecting system and its detection method of instrument pointer amount of jitter
CN109931882B (en) Heat exchange fin key parameter detection system and measurement method
CN117110309A (en) Glass film defect detection method, device and system
KR101024598B1 (en) Surface inspection method using algorithm calculating variance of variance on surface brightness
KR20210065677A (en) Method of detecting defect in welded joint using thermal imaging camera
CN206479141U (en) Weaving device and its bandwidth measurement mechanism with roller
CN110411379A (en) A kind of detection method of computer host box flatness detection
WO2005103608A1 (en) Continuous surface deformation measurement
CN118465220B (en) High-detection-precision nondestructive detection method and system for weld joint
CN112833817B (en) Detection method and detection device for oil casing physical section morphology
RU2777718C1 (en) Method for non-destructive optical and visual testing of items by means of computer vision
US20190227002A1 (en) Apparatus and methods for inspecting damage intensity
Wang et al. Quantitative post-processing module of online automatic image inspection for inner bubble defects in a tire
CN117781983A (en) Steel structure anti-corrosion paint quality detection system and method
JP2017101938A (en) Inner surface inspection device
Chen et al. Measurement of Circular Saw Blade Tooth Dimensions Based on Machine Vision
Berggold Leaving the Inspection Lab: Intelligent white‐light interferometry for maximum precision during production

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20131029

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

RIN1 Information on inventor provided before grant (corrected)

Inventor name: BITZEL, RAYMOND

Inventor name: JONES, JOHNATHAN

DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20150107

RIC1 Information provided on ipc code assigned before grant

Ipc: G01B 11/30 20060101ALI20141219BHEP

Ipc: G01B 7/34 20060101AFI20141219BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: G01B 11/30 20060101ALI20160513BHEP

Ipc: G01B 7/34 20060101AFI20160513BHEP

INTG Intention to grant announced

Effective date: 20160531

RIN1 Information on inventor provided before grant (corrected)

Inventor name: BITZEL, RAYMOND

Inventor name: JONES, JOHNATHAN

INTG Intention to grant announced

Effective date: 20160613

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 850222

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161215

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012026113

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20161130

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 850222

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170301

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170228

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170330

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170228

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012026113

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20170831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170630

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170622

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170630

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170622

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170622

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120622

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161130

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170330

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20200629

Year of fee payment: 9

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602012026113

Country of ref document: DE

Representative=s name: VENNER SHIPLEY GERMANY LLP, DE

Ref country code: DE

Ref legal event code: R082

Ref document number: 602012026113

Country of ref document: DE

Representative=s name: VENNER SHIPLEY LLP, DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210622

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210622

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20220627

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20220629

Year of fee payment: 11

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602012026113

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230630