WO2018060142A1 - Method for classifying objects - Google Patents

Method for classifying objects Download PDF

Info

Publication number
WO2018060142A1
WO2018060142A1 PCT/EP2017/074249 EP2017074249W WO2018060142A1 WO 2018060142 A1 WO2018060142 A1 WO 2018060142A1 EP 2017074249 W EP2017074249 W EP 2017074249W WO 2018060142 A1 WO2018060142 A1 WO 2018060142A1
Authority
WO
WIPO (PCT)
Prior art keywords
objects
clustering
digital image
features
classifying
Prior art date
Application number
PCT/EP2017/074249
Other languages
French (fr)
Inventor
Nicolas PIPARD
Original Assignee
Tata Steel Ijmuiden B.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tata Steel Ijmuiden B.V. filed Critical Tata Steel Ijmuiden B.V.
Publication of WO2018060142A1 publication Critical patent/WO2018060142A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G06V10/435—Computation of moments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30136—Metal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/06—Recognition of objects for industrial automation

Definitions

  • the invention relates to a method for classifying objects on, in or under the surface of a metal strip.
  • the objects on, in or under the surface of the metal strip often are defects such as scratches or area defects in the surface of the metal strip, but the objects can also be dirt or an oil spot on the surface, or a sub-surface defect such as non-metallic inclusions.
  • Such objects of the metal strip have to be detected so as to determine whether the strip meets the quality requirements posed by the use that is to be made of that metal strip. If a defect is present, the strip may be repaired, or a part of the strip may be cut out, or information about the defect can be supplied to the customer buying the strip.
  • the inspection system thus has to be such that the objects can be detected automatically.
  • the objects that are detected using the inspection system are classified and based on this classification it is decided what do to with the object.
  • a method for classifying objects on, in or under the surface of a metal strip comprising the following steps:
  • each clustered group is transformed into one single new digital image, and then each image can be analysed using image processing techniques to derive features such as shape, orientation, area and the like.
  • each clustered group of objects has its own features, and using these features it is possible to classify the clusters. Instead of classifying the objects itself, now the clusters of objects are classified. It will be clear that the number of clusters is much smaller then the number of objects.
  • the information provided by the classification method according to the invention is much more concise and pertinent than the information provided by the existing classification methods.
  • the invention also provides a classification method leading to a better balance between individual objects and objects that are similar and that are present in large numbers. Due to the clustering, the similar objects are allocated into a single group which adequately represents the set of individual objects. Therefore this resolves the problems created by the imbalance between detections of the different types of defects, an issue very common with current inspection systems. For example this imbalance has the consequence of resulting in low classification performances for some defect classes. A human is consequently needed in order to make use of the classification results.
  • the classification of the clusters can also make use of the values derived from the objects in a group forming a cluster. These values usually are statistical values, such as the mean value of the sizes of the objects in the cluster, or the mean value of the light reflexion (as measured with the pixel grey levels on the defect digital images) from each defect.
  • the metal strip is a steel strip.
  • many optical features can be introduced, so a classifying method with which to detect and classify the defects and non-defect objects is important.
  • the clustering of the objects is based on the proximity of the objects on, in or under the surface of the strip.
  • the size and shape of the resulting clusters can be influenced by the pattern the objects form together.
  • the clustering can also be based on other characteristics than the proximity of the objects, such as their shapes, their sizes or their light aspects (such as light reflexion).
  • the classification of the digital images is based on a learning algorithm.
  • the programme used to implement the classification method is using a learning algorithm of the type supervised learning (when the training examples of the clusters have known response labels).
  • learning algorithms can be for example a "decision tree learning", a “support vector machine” or an “artificial neural network”.
  • the clustering is based on a learning algorithm of the type unsupervised learning (when the training examples in this case have no response labels).
  • learning algorithms can be for example "density-based” (DBSCAN), model-based ("Gaussian mixture models”) or “grid-based” (STING or CLIQUE) which are clustering methods.
  • DBSCAN density-based
  • model-based Gaussian mixture models
  • STING or CLIQUE clustering methods.
  • a more complex clustering technique will then be necessary such as an ensemble of algorithms, a multi-step clustering (one clustering method following by an other) or multi-stage clustering (the resulting clusters from the first clustering stage can be clustered together when they have similarities and/or are closed to each others on the strip surface).
  • additional techniques can be used together with the clustering such as spectral clustering, subspace clustering or using constraints (“constrained clustering” is a class of semi- supervised learning algorithms).
  • Figure 1 shows the main steps of the invention.
  • the input is the multiple objects being clustered (here defects).
  • cluster object Based on the resulting "cluster object” and the objects contained in the cluster, pre-engineered features are being calculated.
  • This resulting dataset is being used for learning a classifier. This classifier will predict the cluster classes of new observed clusters.
  • Figure 2 shows two examples of these engineered features, orientation and curviness/convexity of the object for a cluster of objects.
  • Figure 1 shows the following main steps:
  • FIG. 1 shows two examples of engineering features:
  • An example of an application of the technique described in this invention is the clustering and cluster classification of different types of scratch type defects. These can have different root causes, and it might be important to differentiate between the different types (in case of root cause analysis for example).
  • the current inspection systems will classify all the different types of scratch defects in a single class, because small subtle differences cannot be measured in the defect images. For example most scratch defects are perfectly parallel to the strip rolling direction while others can have a very small angle offset that cannot be measured on the defect digital images. It is only when we group all these defects together that we can start measuring it. In order to do so, the detected scratch defects (by a surface inspection system) can be clustered together using the defect positions (as described in claim 1 and as shown by F(a) in Figure 1).
  • the clustering is done using an unsupervised learning algorithm, and more specifically a density-based clustering. Some statistical values are then measured on the content of the resulting cluster and it is transformed into a new digital image (shown by B in Figure 1) from which features are derived (shown by F(b) in Figure 1 with some examples of these engineered features in Figure 2). All these values are therefore the cluster features (C in Figure 1), which can be used for the classification of the cluster.
  • Cluster examples are collected and labelled by experts in order to be used for the learning of a classification algorithm. Two future implementations will be with the detections of oxide defects on coils from hot strip mill, which is an area type defect with specific patterns on the coil surface and the detections of dross related defects on coils from a galvanising line.

Abstract

The invention relates to a method for classifying objects on, in or under the surface of a metal strip. According to the invention, the method comprises the following steps: Detecting the objects using a detection system; Determining similarities and dissimilarities between the detected objects; Clustering the objects into groups based on the determined similarities of the objects; Transforming each clustered group of objects into a single new digital image; Analysing the new digital image using image processing techniques to derive features of the digital image; Classifying the digital images using the features thereof.

Description

METHOD FOR CLASSIFYING OBJECTS
The invention relates to a method for classifying objects on, in or under the surface of a metal strip.
The objects on, in or under the surface of the metal strip often are defects such as scratches or area defects in the surface of the metal strip, but the objects can also be dirt or an oil spot on the surface, or a sub-surface defect such as non-metallic inclusions.
Such objects of the metal strip have to be detected so as to determine whether the strip meets the quality requirements posed by the use that is to be made of that metal strip. If a defect is present, the strip may be repaired, or a part of the strip may be cut out, or information about the defect can be supplied to the customer buying the strip.
Usually it is desirable to detect the objects during the manufacturing process, so real-time. The inspection system thus has to be such that the objects can be detected automatically. According to the known classification systems, the objects that are detected using the inspection system are classified and based on this classification it is decided what do to with the object.
However, it has been found that on a strip surface all types of objects can be present, and that some objects are only present occasionally whereas other objects are present in a multitude. The known existing detection systems therefore create a big difference in prevalence between the classes corresponding to these different types of objects, and consequently even with a small classification error many objects will be misclassified (and this especially for the objects with multitude detections), leading to a pollution of the classes with less detections.
It is a purpose of the invention to provide a classification method for objects on, in or under the surface of a metal strip, for instance based on position, size and density of said objects, providing an improved classification accuracy of those objects.
It is another purpose of the invention to provide a classification method for objects on, in or under the surface of a metal strip that is more reliable than the known systems.
According to the invention, a method for classifying objects on, in or under the surface of a metal strip is provided, comprising the following steps:
• Detecting the objects using a detection system; • Determining similarities and dissimilarities between the detected objects;
• Clustering the objects into groups based on the determined similarities of the objects;
• Transforming each clustered group of objects into a single new digital image; · Analysing the new digital image using image processing techniques to derive features of the digital image;
• Classifying the digital images using the features thereof.
The inventor has found that by first determining what the similarities and dissimilarities between the detected objects are, it is possible to cluster the objects into groups based on the determined similarities. After that, each clustered group is transformed into one single new digital image, and then each image can be analysed using image processing techniques to derive features such as shape, orientation, area and the like. Thus, each clustered group of objects has its own features, and using these features it is possible to classify the clusters. Instead of classifying the objects itself, now the clusters of objects are classified. It will be clear that the number of clusters is much smaller then the number of objects. Thus, the information provided by the classification method according to the invention is much more concise and pertinent than the information provided by the existing classification methods.
The invention also provides a classification method leading to a better balance between individual objects and objects that are similar and that are present in large numbers. Due to the clustering, the similar objects are allocated into a single group which adequately represents the set of individual objects. Therefore this resolves the problems created by the imbalance between detections of the different types of defects, an issue very common with current inspection systems. For example this imbalance has the consequence of resulting in low classification performances for some defect classes. A human is consequently needed in order to make use of the classification results.
The classification of the clusters can also make use of the values derived from the objects in a group forming a cluster. These values usually are statistical values, such as the mean value of the sizes of the objects in the cluster, or the mean value of the light reflexion (as measured with the pixel grey levels on the defect digital images) from each defect.
Preferably, the metal strip is a steel strip. During the manufacture of steel strip many optical features can be introduced, so a classifying method with which to detect and classify the defects and non-defect objects is important.
According to a preferred embodiment the clustering of the objects is based on the proximity of the objects on, in or under the surface of the strip. Thus, the size and shape of the resulting clusters can be influenced by the pattern the objects form together. However, the clustering can also be based on other characteristics than the proximity of the objects, such as their shapes, their sizes or their light aspects (such as light reflexion).
Preferably the classification of the digital images is based on a learning algorithm. The programme used to implement the classification method is using a learning algorithm of the type supervised learning (when the training examples of the clusters have known response labels). Such learning algorithms can be for example a "decision tree learning", a "support vector machine" or an "artificial neural network".
It is preferred when the clustering is based on a learning algorithm of the type unsupervised learning (when the training examples in this case have no response labels). Such learning algorithms can be for example "density-based" (DBSCAN), model-based ("Gaussian mixture models") or "grid-based" (STING or CLIQUE) which are clustering methods. In some application the use of a simple clustering algorithm method will not be enough. A more complex clustering technique will then be necessary such as an ensemble of algorithms, a multi-step clustering (one clustering method following by an other) or multi-stage clustering (the resulting clusters from the first clustering stage can be clustered together when they have similarities and/or are closed to each others on the strip surface). Also additional techniques can be used together with the clustering such as spectral clustering, subspace clustering or using constraints ("constrained clustering" is a class of semi- supervised learning algorithms).
Figure 1 shows the main steps of the invention. The input is the multiple objects being clustered (here defects). Based on the resulting "cluster object" and the objects contained in the cluster, pre-engineered features are being calculated. This resulting dataset is being used for learning a classifier. This classifier will predict the cluster classes of new observed clusters.
Figure 2 shows two examples of these engineered features, orientation and curviness/convexity of the object for a cluster of objects. Figure 1 shows the following main steps:
A: Multiple Objects (defects)
F(a): Clustering
B: Cluster object (1 object, which has been transformed into a digital image) F(b): Feature engineering (from digital image)
C: Cluster features (Data representing digital image and cluster content)
F(c): Classifier engine
D: Cluster classification
Figure 2 shows two examples of engineering features:
A: Orientation
B: Curviness/Convexity.
An example of an application of the technique described in this invention is the clustering and cluster classification of different types of scratch type defects. These can have different root causes, and it might be important to differentiate between the different types (in case of root cause analysis for example). The current inspection systems will classify all the different types of scratch defects in a single class, because small subtle differences cannot be measured in the defect images. For example most scratch defects are perfectly parallel to the strip rolling direction while others can have a very small angle offset that cannot be measured on the defect digital images. It is only when we group all these defects together that we can start measuring it. In order to do so, the detected scratch defects (by a surface inspection system) can be clustered together using the defect positions (as described in claim 1 and as shown by F(a) in Figure 1). The clustering is done using an unsupervised learning algorithm, and more specifically a density-based clustering. Some statistical values are then measured on the content of the resulting cluster and it is transformed into a new digital image (shown by B in Figure 1) from which features are derived (shown by F(b) in Figure 1 with some examples of these engineered features in Figure 2). All these values are therefore the cluster features (C in Figure 1), which can be used for the classification of the cluster. Cluster examples are collected and labelled by experts in order to be used for the learning of a classification algorithm. Two future implementations will be with the detections of oxide defects on coils from hot strip mill, which is an area type defect with specific patterns on the coil surface and the detections of dross related defects on coils from a galvanising line.

Claims

Method for classifying objects on, in or under the surface of a metal strip, comprising the following steps:
• Detecting the objects using a detection system;
• Determining similarities and dissimilarities between the detected objects;
• Clustering the objects into groups based on the determined similarities of the objects;
• Transforming each clustered group of objects into a single new digital image;
• Analysing the new digital image using image processing techniques to derive features of the digital image;
• Classifying the digital images using the features thereof.
Method according to claim 1, wherein the metal strip is a steel strip.
Method according to claim 1 or 2, wherein the clustering of the objects is based on the proximity of the objects on, in or under the surface of the strip.
Method according to any one of the preceding claims, wherein the classification of the digital images is based on a learning algorithm .
Method according to any one of the preceding claims, wherein the clustering is based on a learning algorithm.
PCT/EP2017/074249 2016-09-29 2017-09-25 Method for classifying objects WO2018060142A1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
EP16191559 2016-09-29
EP16191559.0 2016-09-29
EP17164604 2017-04-03
EP17164604.5 2017-04-03

Publications (1)

Publication Number Publication Date
WO2018060142A1 true WO2018060142A1 (en) 2018-04-05

Family

ID=60037565

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2017/074249 WO2018060142A1 (en) 2016-09-29 2017-09-25 Method for classifying objects

Country Status (1)

Country Link
WO (1) WO2018060142A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115841491A (en) * 2023-02-24 2023-03-24 杭州电子科技大学 Quality detection method of porous metal material

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5544256A (en) * 1993-10-22 1996-08-06 International Business Machines Corporation Automated defect classification system

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5544256A (en) * 1993-10-22 1996-08-06 International Business Machines Corporation Automated defect classification system

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115841491A (en) * 2023-02-24 2023-03-24 杭州电子科技大学 Quality detection method of porous metal material

Similar Documents

Publication Publication Date Title
CN114972357B (en) Roller surface defect detection method and system based on image processing
Ravikumar et al. Machine learning approach for automated visual inspection of machine components
Lieber et al. Quality prediction in interlinked manufacturing processes based on supervised & unsupervised machine learning
CN107610114B (en) optical satellite remote sensing image cloud and snow fog detection method based on support vector machine
EP2889618B1 (en) Image processing device, program, image processing method, computer-readable medium, and image processing system
CN110866476A (en) Dense stacking target detection method based on automatic labeling and transfer learning
Mokhtari et al. Comparison of supervised classification techniques for vision-based pavement crack detection
TW201407154A (en) Integration of automatic and manual defect classification
TW201350836A (en) Optimization of unknown defect rejection for automatic defect classification
US8705834B2 (en) Methods and apparatus for image analysis using threshold compactness features
CN110717540B (en) Method and device for identifying radar new radiation source individual
WO2021153633A1 (en) Metal structure phase classification method, metal structure phase classification device, metal structure phase learning method, metal structure phase learning device, material property prediction method for metal material, and material property prediction device for metal material
TW202127371A (en) Image-based defect detection method and computer readable medium thereof
CN102663422A (en) Floor layer classification method based on color characteristic
US20140233841A1 (en) Method of chekcing the appearance of the surface of a tyre
JP2022512292A (en) Classification of defects in semiconductor samples
CN116012291A (en) Industrial part image defect detection method and system, electronic equipment and storage medium
CN115511856A (en) Component defect detection method and device, electronic equipment and storage medium
KR101782364B1 (en) Vision inspection method based on learning data
JP2020177430A (en) Information processing apparatus, information processing method, and program
WO2018060142A1 (en) Method for classifying objects
CN112396580A (en) Circular part defect detection method
CN111860500A (en) Shoe print wear area detection and edge tracing method
Zhang et al. Determining statistical process control baseline periods in long historical data streams
JP2018018153A (en) Steel type discrimination device and steel type discrimination method

Legal Events

Date Code Title Description
DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17780661

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17780661

Country of ref document: EP

Kind code of ref document: A1