EP0661108B1 - Method for optically sorting bulk material - Google Patents

Method for optically sorting bulk material Download PDF

Info

Publication number
EP0661108B1
EP0661108B1 EP94250285A EP94250285A EP0661108B1 EP 0661108 B1 EP0661108 B1 EP 0661108B1 EP 94250285 A EP94250285 A EP 94250285A EP 94250285 A EP94250285 A EP 94250285A EP 0661108 B1 EP0661108 B1 EP 0661108B1
Authority
EP
European Patent Office
Prior art keywords
colour
examination material
values
process according
color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP94250285A
Other languages
German (de)
French (fr)
Other versions
EP0661108A3 (en
EP0661108A2 (en
Inventor
Wolfgang Dr. Graudejus
Eberhard Briem
Wilhelm Dr. Hättich
Heribert Dr. Geisselmann
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Reemtsma Cigarettenfabriken GmbH
Original Assignee
HF and PhF Reemtsma GmbH and Co
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by HF and PhF Reemtsma GmbH and Co filed Critical HF and PhF Reemtsma GmbH and Co
Publication of EP0661108A2 publication Critical patent/EP0661108A2/en
Publication of EP0661108A3 publication Critical patent/EP0661108A3/en
Application granted granted Critical
Publication of EP0661108B1 publication Critical patent/EP0661108B1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B07SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07CPOSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/36Sorting apparatus characterised by the means used for distribution
    • B07C5/363Sorting apparatus characterised by the means used for distribution by means of air
    • B07C5/365Sorting apparatus characterised by the means used for distribution by means of air using a single separation means
    • B07C5/366Sorting apparatus characterised by the means used for distribution by means of air using a single separation means during free fall of the articles
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B07SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07CPOSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34Sorting according to other particular properties
    • B07C5/342Sorting according to other particular properties according to optical properties, e.g. colour
    • B07C5/3422Sorting according to other particular properties according to optical properties, e.g. colour using video scanning devices, e.g. TV-cameras
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10STECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S209/00Classifying, separating, and assorting solids
    • Y10S209/939Video scanning

Definitions

  • the invention relates to a method for optical sorting of bulk goods according to the preamble of claim 1.
  • test material is conveyed on tapes and its image for testing with a diode line camera or TV camera is recorded.
  • the signal is preferably recorded in flight if e.g. the test material from one belt to one other band is translated.
  • recording signals in flight the test material can be viewed from several sides with a defined background be assessed.
  • the color is also used for image acquisition detected.
  • the color is used to identify striking areas in the Detect image.
  • the image of the test material keeps pace with the image scan evaluated so that immediately after passing through a test part this can be classified by the measuring station. So that is ejection of the parts in flight using flaps or air nozzles possible.
  • a disadvantage of the known methods is that the Detection rate for color heterogeneous products is low, if you focus on the detection of conspicuous pixels Detection of color values limited that are not included in the product because there are many different color values in the product occurrence. If one extends the detection to color values, the are also included in the product is generally already at Extensions to colors rarely found in the product are unbearable high proportion of the faultless product as rejects detected.
  • the object of the invention is the method for optical sorting of bulk goods to improve that in color heterogeneous bulk material to be detected with a foreign body very low error rate can be recognized.
  • the light from each pixel is transmitted Color filter in front of the detection elements of a line e.g. in the three color components red (R), green (G) and blue (B) disassembled.
  • RGB red
  • G green
  • B blue
  • This ensures that detection of conspicuous pixels (Points with color values that rarely occur in a faultless product) by evaluating those measured by the row elements Color values (intensities of the respective color components) possible is. Then an evaluation of the geometry with regard on local clusters of conspicuous pixels.
  • the color space is spanned by the different color components that be measured for each pixel.
  • red, green and blue are three-dimensional Color space formed.
  • Classifiers i.e. Means for evaluation of the measured values based on specified criteria a classification of the measured color values, whereby a Classifier focused on only one sub-area and included in this sub-area for the color heterogeneous product Detects detection areas, i.e. contiguous areas of eye-catching pixels.
  • the committee part as a relatively large area of pixels of the color values of the selected sub-area is detected and can be evaluated this detection area can be recognized by the classifier.
  • the faultless product within a such sub-area in general only in rare cases large areas of conspicuous pixels found, and the number of false detections remains small.
  • This improvement classification is used in practical application by dividing the committee into typical classes and a classifier is set up for each class, where the classifiers work in parallel during the test.
  • the bulk material moves preferentially in a color sorting machine in flight on an observation head with a light source and a product signal receiver arranged in the vicinity of the light source past.
  • the test material is made more adjacent by different color filters
  • Line elements of a camera line e.g. a CCD line, of the receiver in the three colors red (R), green (G) and blue (B) disassembled.
  • the row elements thus measure in their respective Spectral ranges the brightness of the pixels, also color values called. This results in a three-dimensional distribution of color values, the evaluation of which is based on one-dimensional examples is discussed.
  • test material is in a pre-learning process without Measure committee parts and the frequency distribution 1 of Color values determined.
  • the test material is also without rejects measured and in a first step a color value range determined for good test material by an experience Threshold 2 is placed over the frequency distribution 1 of the color values , the intersection between the threshold 2 and the curve of the frequency distribution 1 the limits of the test material color value range surrender.
  • the color value ranges of the product are divided into sub-ranges.
  • everyone is concentrating in this example of the classifiers A, B and C working in parallel only a sub-area.
  • the reject part is detected as a relatively large area and can be recognized by evaluating the detection areas become. Again, the distributions of the color values of this large areas measured and after their normalization as Thresholds introduced. All color values at which these thresholds 4, 5 and 6 exceed the color value distribution 1 of the test parts, are interpreted as rejects and lead to error detection.
  • the threshold 8 shows the Color value distribution of a reject part. Within the by the Threshold 8 of certain color value range becomes error-free test material classified as a committee. Through the learning process the color value distribution of this large-area detection area measured in error-free test material and according to a standardization as Threshold 7 introduced. All color values with threshold 7 the threshold 8 of the reject part is considered to be Test material is interpreted appropriately and does not lead to one Fault detection.
  • the classification system doubled.
  • a system takes over the test task while the other system the current color value distribution of the product measures.
  • the measurement of the current color value distribution is carried out by the testing classifier is monitored so that during this measurement no color values of the committee are recorded.
  • the learning classifier becomes a representative number of measured values with the newly measured distribution for the test task activated while the classifier currently set to check takes over the learning task.
  • the test item is, for example, of two Lamps illuminated from the direction of the line scan camera. Between Both lamps have the optical axis of the line scan camera. At this arrangement comes with the design of the background essential importance because of the background the color value distribution if possible, do not expand the faultless product should. An extension would lower the detection performance.
  • the background has the Color of the test material, which has the advantage that the contrast between Background and test material is low and therefore the color value distribution of the test material due to edge effects at the transition from the background is not significantly expanded to test material.
  • This variant provides in terms of color and spatial resolution the best results.
  • the disadvantage of pollution is avoided by the background running as a rotating roller, which deposits flung away immediately.
  • the shadow of the test specimen on the background becomes diffuse and, depending on the bulk density, harmless if the rotating roller at an adjusted distance from the test material is installed.
  • the background can be a cylindrical spotlight that is in the Color of the test specimen shines and from a transparent rotating Surrounded role, which throws away the deposits.
  • the background is on dark hole, which has the advantage that the test material from Background segmenting and no interference Pollution and shadow formation arise.
  • segmentation of the test material can, for example, be the form for separating Good sharing and committee are used.
  • the line scan camera looks through a slit into this container.
  • the slot is regarding its width at the aperture and focal length of the camera lens and adapted to the distance to the focus plane.
  • the light of each pixel is reflected in the three colors red (R), green (G) and blue (B) disassembled.
  • the color sensors are in common color cameras even side by side so that the color sensors with regard to a pixel different local areas of the See the target. 4 are the color sensors (R, G, B) arranged horizontally while the measurement object is from moved up and down this horizontal line.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Sorting Of Articles (AREA)
  • Spectrometry And Color Measurement (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)
  • Epoxy Compounds (AREA)
  • Crystals, And After-Treatments Of Crystals (AREA)
  • Treatment Of Sludge (AREA)

Abstract

The invention relates to a method for optically sorting bulk material in a colour sorting machine, the material being conveyed by way of a conveyor belt and moving past an observation head having a light source and a product signal receiver arranged in the vicinity of the light source, the reflected light from the image points of the material to be tested being decomposed by means of various coloured filters of adjacent detection elements of a row of the receiver into a plurality of spectral regions and the material to be tested being sorted on the basis of the colour values (measured value of the intensity in the respective colour). According to the invention, to improve the detection rate, it is provided that in the case of material to be tested which is mixed with reject parts, in each case the colour values of the product are investigated in a plurality of selected subregions, in that in each subregion a classifier determines associated areas of image points having colour values falling into the respective subregion and carries out a classification according to predetermined criteria from the geometry and the size of these detection areas. <IMAGE>

Description

Die Erfindung betrifft eine Verfahren zum optischen Sortieren von Schüttgut gemäß Oberbegriff des Patentanspruchs 1.The invention relates to a method for optical sorting of bulk goods according to the preamble of claim 1.

Einderartiges Verfahren ist aus der US-A-5,085,325 bekannt.Such a method is known from US-A-5,085,325.

Es ist bereits bekannt, daß Prüfgut auf Bändern gefördert und dessen Bild zur Prüfung mit einer Diodenzeilenkamera oder einer Fernsehkamera aufgenommen wird. Die Signalaufnahme erfolgt vorzugsweise im Flug, wenn z.B. das Prüfgut von einem Band auf ein anderes Band übergesetzt wird. Bei der Signalaufnahme im Flug kann das Prüfgut von mehreren Seiten bei definiertem Hintergrund begutachtet werden.It is already known that test material is conveyed on tapes and its image for testing with a diode line camera or TV camera is recorded. The signal is preferably recorded in flight if e.g. the test material from one belt to one other band is translated. When recording signals in flight the test material can be viewed from several sides with a defined background be assessed.

Bei der Bildaufnahme wird bei modernen Anlagen auch die Farbe erfaßt. Dabei wird die Farbe dazu genutzt, auffällige Gebiete im Bild zu detektieren.In modern systems, the color is also used for image acquisition detected. The color is used to identify striking areas in the Detect image.

Das Bild des Prüfgutes wird schritthaltend mit der Bildabtastung ausgewertet, so daß unmittelbar nach Durchlauf eines Prüfteils durch die Meßstation dieses klassifiziert werden kann. Damit ist eine Ausschleusung der Teile im Flug mittels Klappen oder Luftdüsen möglich. The image of the test material keeps pace with the image scan evaluated so that immediately after passing through a test part this can be classified by the measuring station. So that is ejection of the parts in flight using flaps or air nozzles possible.

Ein Nachteil der bekannten Verfahren besteht darin, daß die Detektionsrate bei farblich heterogenen Produkten gering ist, wenn man sich bei der Detektion auffälliger Bildpunkte auf die Detektion von Farbwerten beschränkt, die nicht im Produkt enthalten sind, weil sehr viele unterschiedliche Farbwerte im Produkt vorkommen. Erweitert man die Detektion auf Farbwerte, die auch im Produkt enthalten sind, wird im allgemeinen schon bei Erweiterungen auf selten im Produkt vorkommende Farben ein unerträglich hoher Anteil des fehlerfreien Produktes als Ausschuß detektiert.A disadvantage of the known methods is that the Detection rate for color heterogeneous products is low, if you focus on the detection of conspicuous pixels Detection of color values limited that are not included in the product because there are many different color values in the product occurrence. If one extends the detection to color values, the are also included in the product is generally already at Extensions to colors rarely found in the product are unbearable high proportion of the faultless product as rejects detected.

Es ist Aufgabe der Erfindung, das Verfahren zum optischen Sortieren von Schüttgut dahingehend zu verbessern, daß bei farblich heterogenem Schüttgut zu detektierende Fremdkörper mit einer sehr geringen Fehlerquote erkannt werden.The object of the invention is the method for optical sorting of bulk goods to improve that in color heterogeneous bulk material to be detected with a foreign body very low error rate can be recognized.

Zur Lösung dieser Aufgabe dient das Verfahren mit den kennzeichnenden Merkmalen des Patentanspruchs 1.The method with the characteristic is used to solve this task Features of claim 1.

Bei der Bildaufnahme wird das Licht jedes Bildpunktes durch Farbfilter vor den Nachweis-Elementen einer Zeile z.B. in die drei Farbkomponenten rot (R), grün (G) und blau (B) zerlegt. Dadurch wird erreicht, daß eine Detektion auffälliger Bildpunkte (Punkte mit Farbwerten, die selten im fehlerfreien Produkt vorkommen) durch Auswertung der von den Zeilenelementen gemessenen Farbwerte (Intensitäten der jeweiligen Farbkomponenten) möglich ist. Anschließend wird eine Auswertung der Geometrie in Hinblick auf lokale Anhäufungen von auffälligen Bildpunkten durchgeführt.When capturing an image, the light from each pixel is transmitted Color filter in front of the detection elements of a line e.g. in the three color components red (R), green (G) and blue (B) disassembled. This ensures that detection of conspicuous pixels (Points with color values that rarely occur in a faultless product) by evaluating those measured by the row elements Color values (intensities of the respective color components) possible is. Then an evaluation of the geometry with regard on local clusters of conspicuous pixels.

Zunächst wird die gesamte Bandbreite der möglichen Farbwerte im Farbraum in mehrere Unterbereiche unterteilt, wobei der Farbraum durch die verschiedenen Farbkomponenten aufgespannt wird, die für jeden Bildpunkt gemessen werden. Beispielsweise wird durch die drei Farbkomponenten rot, grün und blau ein dreidimensionaler Farbraum gebildet. Klassifikatoren, d.h. Mittel zum Auswerten der Meßwerte aufgrund vorgegebener Kriterien, erlauben eine Klassifizierung der gemessenen Farbwerte, wobei sich ein Klassifikator nur auf einen Unterbereich konzentriert und dabei in diesem Unterbereich bei dem farblich heterogenen Produkt Detektionsflächen erkennt, also zusammenhängende Flächen von auffälligen Bildpunkten.First, the entire range of possible color values in the Color space divided into several sub-areas, the color space is spanned by the different color components that be measured for each pixel. For example, by the three color components red, green and blue are three-dimensional Color space formed. Classifiers, i.e. Means for evaluation of the measured values based on specified criteria a classification of the measured color values, whereby a Classifier focused on only one sub-area and included in this sub-area for the color heterogeneous product Detects detection areas, i.e. contiguous areas of eye-catching pixels.

Liegen die Farbwerte eines farblich homogenen Ausschußteils bevorzugt in dem ausgewählten Unterbereich, wird das Ausschußteil als relativ großflächiges Gebiet von Bildpunkten der Farbwerte des ausgewählten Unterbereichs detektiert und kann durch Auswertung dieser Detektionsfläche vom Klassifikator erkannt werden. Andererseits werden bei dem fehlerfreien Produkt innerhalb eines solchen Unterbereiches im allgemeinen nur in seltenen Fällen großflächige Gebiete von auffälligen Bildpunkten gefunden, und damit bleibt die Zahl der Fehldetektionen gering. Diese Verbesserung in der Klassifikation nutzt man in der praktischen Anwendung dadurch, daß der Ausschuß in typische Klassen unterteilt und für jede Klasse ein Klassifikator eingerichtet wird, wobei die Klassifikatoren bei der Prüfung parallel arbeiten.If the color values of a color-homogeneous reject part are preferred in the selected sub-area, the committee part as a relatively large area of pixels of the color values of the selected sub-area is detected and can be evaluated this detection area can be recognized by the classifier. On the other hand, the faultless product within a such sub-area in general only in rare cases large areas of conspicuous pixels found, and the number of false detections remains small. This improvement classification is used in practical application by dividing the committee into typical classes and a classifier is set up for each class, where the classifiers work in parallel during the test.

In einem bevorzugten Ausführungsbeispiel wird in den Unterbereichen, in denen Ausschußteile vermutet werden, durch Vorzeigen von Ausschußteilen die Verteilung derer Farbwerte gelernt.In a preferred embodiment, in the sub-areas, in which committee parts are suspected, by showing learned from committee parts the distribution of their color values.

Die Erfindung wird im folgenden anhand von Zeichnungen näher erläutert:

Fig. 1
zeigt beispielhaft eine eindimensionale Farbwertverteilung mit den Bereichen für gutes Prüfgut.
Fig. 2
zeigt ein eindimensionales Beispiel zur Klassifizierung mit parallelen Klassifikatoren bei der Erkennung von Ausschuß, dessen Farbwerte sich mit den Farbwerten des Produktes überlappen.
Fig. 3
zeigt ein eindimensionales Beispiel zur Korrektur eines Klassifikators durch das Nachlernen.
Fig. 4
zeigt ein Beispiel für die Farbverfälschung an Objektkanten bei Verwendung einer Kamera mit Bildpunkten, bei denen die Farbsensoren nebeneinander liegen.
The invention is explained in more detail below with reference to drawings:
Fig. 1
shows an example of a one-dimensional color value distribution with the areas for good test material.
Fig. 2
shows a one-dimensional example of classification with parallel classifiers in the detection of rejects, whose color values overlap with the color values of the product.
Fig. 3
shows a one-dimensional example for the correction of a classifier by re-learning.
Fig. 4
shows an example of the color distortion at object edges when using a camera with pixels in which the color sensors are next to each other.

In einer Farbsortiermaschine bewegt sich das Schüttgut bevorzugt im Flug an einem Beobachtungskopf mit einer Lichtquelle und einem in der Nähe der Lichtquelle angeordneten Produktsignalempfänger vorbei. Das reflektierte Licht jedes Bildpunktes des Prüfgutes wird durch verschiedene Farbfilter nebeneinanderliegender Zeilenelemente einer Kamerazeile, z.B. einer CCD-Zeile, des Empfängers in die drei Farben rot (R), grün (G) und blau (B) zerlegt. Die Zeilenelemente messen somit in ihren jeweiligen Spektralbereichen die Helligkeiten der Bildpunkte, auch Farbwerte genannt. Somit ergibt sich eine dreidimensionale Verteilung von Farbwerten, deren Auswertung im folgenden anhand von eindimensionalen Beispielen diskutiert wird.The bulk material moves preferentially in a color sorting machine in flight on an observation head with a light source and a product signal receiver arranged in the vicinity of the light source past. The reflected light of every pixel of the The test material is made more adjacent by different color filters Line elements of a camera line, e.g. a CCD line, of the receiver in the three colors red (R), green (G) and blue (B) disassembled. The row elements thus measure in their respective Spectral ranges the brightness of the pixels, also color values called. This results in a three-dimensional distribution of color values, the evaluation of which is based on one-dimensional examples is discussed.

Bezogen auf Fig. 1 wird in einem Vorlernprozeß das Prüfgut ohne Ausschußteile vermessen und die Häufigkeitsverteilung 1 der Farbwerte ermittelt.With reference to FIG. 1, the test material is in a pre-learning process without Measure committee parts and the frequency distribution 1 of Color values determined.

In einem Nachlernprozeß wird ebenfalls das Prüfgut ohne Ausschußteile vermessen und in einem ersten Schritt ein Farbwertbereich für gutes Prüfgut festgelegt, indem eine erfahrungsgemäße Schwelle 2 über die Häufigkeitsverteilung 1 der Farbwerte gelegt wird, wobei sich aus den Schnittpunkten zwischen der Schwelle 2 und der Kurve der Häufigkeitsverteilung 1 die Grenzen des Prüfgutfarbwertbereiches ergeben.In a re-learning process, the test material is also without rejects measured and in a first step a color value range determined for good test material by an experience Threshold 2 is placed over the frequency distribution 1 of the color values , the intersection between the threshold 2 and the curve of the frequency distribution 1 the limits of the test material color value range surrender.

Bei der gewählten Einstellung von Schwelle 2 werden auch beim fehlerfreien Prüfgut Bildpunkte vorkommen, die als auffällig eingestuft werden. Diese Bildpunkte würden aber, wenn sie sich zu großflächigen Gebieten zusammenballen, irrtümlich als Ausschuß klassifiziert. Die Erfahrung zeigt nun, daß eine solche Ballung wiederum bevorzugt in gewissen Farbwertbereichen vorkommt. Um diese Farbwertbereiche zu messen, werden in dem Nachlernprozeß ein im fehlerfreien Prüfgut detektiertes großflächiges Gebiet abgespeichert und die Verteilung dessen Farbwerte gemessen. Diese Verteilung wird nach einer Normierung als Schwelle 3 eingeführt. Alle Farbwerte, bei denen die Schwelle 3 die Farbwertverteilung 1 der Prüfgutteile übersteigt, d.h. die Farbwerte in dem Intervall zwischen den Schnittpunkten der Schwelle 3 mit der Kurve der Farbwertverteilung 1, werden als dem Prüfgut zugehörig interpretiert und führen damit nicht zu einer Fehlerdetektion.With the selected setting of threshold 2, also at error-free test sample pixels that appear as conspicuous get ranked. These pixels would, however, if they were agglomerate into large areas, mistakenly as a committee classified. Experience now shows that such Agglomeration in turn occurs preferentially in certain color value ranges. In order to measure these color value ranges, in the re-learning process a large area detected in the error-free test material Area saved and the distribution of its color values measured. This distribution is called a standardization Threshold 3 introduced. All color values where the threshold 3 the color value distribution exceeds 1 of the test parts, i.e. the Color values in the interval between the intersections of the Threshold 3 with the curve of the color value distribution 1 are called interpreted according to the test material and thus do not lead to an error detection.

Bei der Messung von mit Ausschußteilen versetztem Prüfgut werden die Farbwertbereiche des Produktes in Unterbereiche eingeteilt. Bezogen auf Fig. 2 konzentriert sich bei diesem Beispiel jeder der parallel arbeitenden Klassifikatoren A, B und C nur auf einen Unterbereich. Liegen die Farbanteile des farblich homogenen Ausschußteils bevorzugt in dem ausgewählten Unterbereich, wird das Ausschußteil als relativ großflächiges Gebiet detektiert und kann durch Auswertung der Detektionsflächen erkannt werden. Auch hier werden die Verteilungen der Farbwerte dieser großflächigen Gebiete gemessen und nach ihrer Normierung als Schwellen eingeführt. Alle Farbwerte, bei denen diese Schwellen 4, 5 und 6 die Farbwertverteilung 1 der Prüfgutteile übersteigen, werden als Ausschuß interpretiert und führen zu einer Fehlerdetektion.When measuring test material with rejects the color value ranges of the product are divided into sub-ranges. Referring to Fig. 2, everyone is concentrating in this example of the classifiers A, B and C working in parallel only a sub-area. Are the color components of the color homogeneous Committee part preferably in the selected sub-area, the reject part is detected as a relatively large area and can be recognized by evaluating the detection areas become. Again, the distributions of the color values of this large areas measured and after their normalization as Thresholds introduced. All color values at which these thresholds 4, 5 and 6 exceed the color value distribution 1 of the test parts, are interpreted as rejects and lead to error detection.

Es ist ebenfalls möglich, daß bei einem fehlerfreien Produkt großflächige Detektionsgebiete in einem durch einen Klassifikator abgedeckten Farbwertbereich detektiert werden und somit das fehlerfreie Produkt als Ausschuß klassifiziert wird. In einem weiteren Nachlernprozeß werden speziell diese Farbwerte, die zu großflächigen Detektionsgebieten im fehlerfreien Produktbereich führen, gelernt und durch Veränderung der Schwellen als gutes Prüfgut erkannt. Bezogen auf Fig. 3 zeigt die Schwelle 8 die Farbwertverteilung eines Ausschußteils. Innerhalb des durch die Schwelle 8 bestimmten Farbwertbereiches wird fehlerfreies Prüfgut als Ausschuß klassifiziert. Durch den Nachlernprozeß wird die Farbwertverteilung dieses großflächigen Detektionsgebietes im fehlerfreien Prüfgut gemessen und nach einer Normierung als Schwelle 7 eingeführt. Alle Farbwerte, bei denen die Schwelle 7 die Schwelle 8 des Ausschußteils übersteigt, werden als dem Prüfgut zugehörig interpretiert und führen damit nicht zu einer Fehlerdetektion.It is also possible that with a faultless product large-scale detection areas in one by a classifier covered color value range are detected and thus the faultless product is classified as a committee. In one further learning processes are specifically these color values, which too large detection areas in the flawless product area lead, learned and by changing the thresholds as good Test material recognized. 3, the threshold 8 shows the Color value distribution of a reject part. Within the by the Threshold 8 of certain color value range becomes error-free test material classified as a committee. Through the learning process the color value distribution of this large-area detection area measured in error-free test material and according to a standardization as Threshold 7 introduced. All color values with threshold 7 the threshold 8 of the reject part is considered to be Test material is interpreted appropriately and does not lead to one Fault detection.

Nach dem Lernen wird zur automatischen Prüfung des Produktes übergegangen.After learning, the product is automatically checked passed over.

Bei der Prüfung, die sich über Tage hinziehen kann, ist mit systematischen driftartigen Veränderungen des Produktes zu rechnen. Diese Änderungen führen zu einer mit der Zeit nachlassenden Systemleistung. Um dies zu vermeiden, wird das Klassifikationssystem verdoppelt. Ein System übernimmt die Prüfaufgabe, während das andere System die aktuelle Farbwertverteilung des Produktes mißt. Die Messung der aktuellen Farbwertverteilung wird durch den prüfenden Klassifikator überwacht, damit bei dieser Messung keine Farbwerte des Ausschusses erfaßt werden. Nach Erfassung einer repräsentativen Zahl von Meßwerten wird der lernende Klassifikator mit der neu gemessenen Verteilung für die Prüfaufgabe aktiviert, während der bis jetzt auf Prüfen eingestellte Klassifikator die Lernaufgabe übernimmt.With the test, which can drag on for days, is with systematic drift-like changes in the product. These changes result in a deterioration over time System performance. To avoid this, the classification system doubled. A system takes over the test task while the other system the current color value distribution of the product measures. The measurement of the current color value distribution is carried out by the testing classifier is monitored so that during this measurement no color values of the committee are recorded. After capture The learning classifier becomes a representative number of measured values with the newly measured distribution for the test task activated while the classifier currently set to check takes over the learning task.

Diese Anpassung ist nur möglich, wenn ein detektierter, als auffällig eingestufter Farbpunkt nicht in jedem Fall zu einer Ausschußentscheidung führt. Würde ein detektierter Farbpunkt immer zu einer Ausschußentscheidung führen, könnte der lernende Klassifikator keine neuen Farbwerte übernehmen, da bei einer Ausschußentscheidung die neu gelernte Farbwertverteilung verworfen wird. Da bei dem System aber detektierte Farbpunkte nur dann als Ausschuß klassifiziert werden, wenn sie eine größere zusammenhängende Fläche bilden, kann die gemessene Häufigkeit auch bei detektierten Farbwerten angepaßt werden. Umgekehrt kann das System mit dieser Anpassung zum Ausschuß gehörende Farbwerte detektieren, die bei einer früheren Messung in der Farbwertverteilung des Produktes vertreten waren und bei der aktuell gemessenen Verteilung nicht mehr enthalten sind.This adjustment is only possible if a detected as noticeably classified color point does not always become one Committee decision leads. Would be a detected color point the learner could always lead to a committee decision Classifier do not take on new color values, because with one Committee decision rejected the newly learned color value distribution becomes. Since the system only detected color dots then be classified as a committee if they are a larger one form a contiguous area, the measured frequency can also be adjusted for detected color values. Conversely, can the system with this adjustment color values belonging to the committee detect that in a previous measurement in the color value distribution of the product were represented and at the currently measured Distribution is no longer included.

Bei der Signalaufnahme wird das Prüfgut beispielsweise von zwei Lampen aus Richtung der Zeilenkamera beleuchtet. Zwischen den beiden Lampen liegt die optische Achse der Zeilenkamera. Bei dieser Anordnung kommt der Gestaltung des Hintergrundes eine wesentliche Bedeutung zu, weil der Hintergrund die Farbwertverteilung des fehlerfreien Produktes möglichst nicht erweitern sollte. Eine Erweiterung würde die Detektionsleistung senken.When the signal is recorded, the test item is, for example, of two Lamps illuminated from the direction of the line scan camera. Between Both lamps have the optical axis of the line scan camera. At this arrangement comes with the design of the background essential importance because of the background the color value distribution if possible, do not expand the faultless product should. An extension would lower the detection performance.

Diese Forderung läßt sich nicht verwirklichen, wenn das Prüfgut auf dem Transportband liegend aufgenommen wird. Wegen Verschmutzung und Abnutzung hat das Band keine einheitliche Farbe. Zusätzlich bilden sich Schatten auf dem Transportband aus, was insgesamt zu einer wesentlichen Erweiterung der Farbwertverteilung beim Messen des fehlerfreien Prüfgutes führt. Aus diesem Grund wird das Prüfgut im Flug beobachtet.This requirement cannot be met if the test item is recorded lying on the conveyor belt. Because of pollution and wear, the tape has no uniform color. In addition shadows form on the conveyor belt what overall to a substantial expansion of the color value distribution leads when measuring the error-free test material. For this The test material is therefore observed in flight.

In einer ersten Ausführungsvariante hat der Hintergrund die Farbe des Prüfgutes, was den Vorteil hat, daß der Kontrast zwischen Hintergrund und Prüfgut gering ist und daher die Farbwertverteilung des Prüfgutes durch Randeffekte am Übergang vom Hintergrund zu Prüfgut nicht wesentlich erweitert wird. Diese Ausführungsvariante liefert hinsichtlich Farb- und Ortsauflösung die besten Ergebnisse.In a first embodiment, the background has the Color of the test material, which has the advantage that the contrast between Background and test material is low and therefore the color value distribution of the test material due to edge effects at the transition from the background is not significantly expanded to test material. This variant provides in terms of color and spatial resolution the best results.

Der Nachteil der Verschmutzung wird vermieden, indem der Hintergrund als rotierende Rolle ausgeführt wird, welche Ablagerungen sofort wegschleudert. Der Schatten des Prüfgutes auf dem Hintergrund wird diffus und je nach Schüttungsdichte unschädlich, wenn die rotierende Rolle in einem angepaßten Abstand zum Prüfgut installiert wird. Bei großer Schüttungdichte des Prüfgutes wird eine zu starke Abdunkelung des Hintergrundes durch eine zusätzliche Beleuchtung des Hintergrundes vermieden. Alternativ kann der Hintergrund ein zylindrischer Strahler sein, der in der Farbe des Prüfgutes strahlt und von einer transparenten rotierenden Rolle umgeben ist, welche die Ablagerungen wegschleudert.The disadvantage of pollution is avoided by the background running as a rotating roller, which deposits flung away immediately. The shadow of the test specimen on the background becomes diffuse and, depending on the bulk density, harmless if the rotating roller at an adjusted distance from the test material is installed. When the bulk density of the test material is high an excessive darkening of the background by an additional Backlighting avoided. Alternatively, you can the background can be a cylindrical spotlight that is in the Color of the test specimen shines and from a transparent rotating Surrounded role, which throws away the deposits.

In einer zweiten Ausführungsvariante ist der Hintergrund ein dunkles Loch, was den Vorteil hat, daß sich das Prüfgut vom Hintergrund segmentieren läßt und keine Beinträchtigung durch Verschmutzung und Schattenbildung entsteht. Bei einer Segmentierung des Prüfgutes kann zum Beispiel die Form zur Trennung von Gutteilen und Ausschuß genutzt werden.In a second embodiment, the background is on dark hole, which has the advantage that the test material from Background segmenting and no interference Pollution and shadow formation arise. With segmentation of the test material can, for example, be the form for separating Good sharing and committee are used.

Zur Realisierung des dunklen Loches wird ein möglichst großer Behälter mit reflektionsarmen Wandungen gebaut. Die Zeilenkamera blickt durch einen Schlitz in diesen Behälter. Der Schlitz ist hinsichtlich seiner Breite an Blende und Brennweite des Kameraobjektivs sowie an den Abstand zur Schärfeebene angepaßt.In order to realize the dark hole, one is as large as possible Containers built with low-reflection walls. The line scan camera looks through a slit into this container. The slot is regarding its width at the aperture and focal length of the camera lens and adapted to the distance to the focus plane.

Bei der Bildaufnahme wird das Licht jedes Bildpunktes in die drei Farben rot (R), grün (G) und blau (B) zerlegt. Abhängig von dem gewählten Abtastprinzip und der Justage der Kamera werden die Farbkomponenten nicht idealerweise am gleichen Ort, sondern ortsversetzt gemessen. Bei gängigen Farbkameras liegen die Farbsensoren sogar örtlich nebeneinander, so daß die Farbsensoren hinsichtlich eines Bildpunktes unterschiedliche Ortsbereiche des Meßobjektes sehen. Bezogen auf Fig. 4 sind die Farbsensoren (R, G, B) waagerecht angeordnet, während sich das Meßobjekt von oben nach unten an dieser waagerechten Zeile vorbeibewegt. Der Hintergrund erzeugt in diesem Beispiel bei den jeweiligen Farbsensoren die Signalpegel R = 0, G = 0 und B = 0, während das Meßobjekt die Signalpegel R = 100, G = 50 und B = 20 bewirkt. In Fig. 4 mißt hier nur das Sensor-Tripel Xn, Yn die richtige Farbe des Meßobjektes. Bei allen anderen Tripeln werden Farbwerte gemessen, die mindestens einen Farbwert enthalten, der dunkler als der entsprechende Farbwert des Prüfgutes ist. So mißt zum Beispiel das Tripel Xn, Yn-1 die Pegel R = 50, G = 25 und B = 10. Um diese Störungen zu vermeiden, werden Bildpunkte, deren Farbwerte um einen einstellbaren Faktor dunkler als die des entsprechenden Nachbarpunktes sind, unterdrückt, indem die Signalpegel gespeichert werden und ein Vergleich der horizontalen und vertikalen Nachbarpunkte durchgeführt wird. When capturing an image, the light of each pixel is reflected in the three colors red (R), green (G) and blue (B) disassembled. Depending on the selected scanning principle and the adjustment of the camera the color components not ideally in the same place, but measured at different locations. The color sensors are in common color cameras even side by side so that the color sensors with regard to a pixel different local areas of the See the target. 4 are the color sensors (R, G, B) arranged horizontally while the measurement object is from moved up and down this horizontal line. Of the In this example, the background is generated by the respective color sensors the signal levels R = 0, G = 0 and B = 0, while the Object causes the signal levels R = 100, G = 50 and B = 20. In Fig. 4 only measures the sensor triple Xn, Yn the correct color of the measurement object. Color values are used for all other triples measured that contain at least one color value, the darker than the corresponding color value of the test material. So measure to Example the triple Xn, Yn-1 the levels R = 50, G = 25 and B = 10. To avoid these disturbances, pixels, whose color values are darker than that by an adjustable factor of the corresponding neighboring point are suppressed by the Signal levels are stored and a comparison of the horizontal and vertical neighboring points is performed.

Key to the FiguresKey to the figures Figure 1Figure 1

Besetzungoccupation
densitydensity
fehlerfreies Prüfguterror-free test material
fault-free examination materialfault-free examination material
MeßwertReading
measured valuemeasured value
Figure 2Figure 2

Besetzungoccupation
densitydensity
KlassifikatorClassifier
classifierclassifier
detektiertdetected
detectsdetects
AusschußCommittee
rejectreject
MeßwertReading
measured valuemeasured value
Figure 3Figure 3

Besetzungoccupation
densitydensity
Detektion vonDetection of
detection of rejectdetection of reject
AusschußCommittee
MeßwertReading
measured valuemeasured value
Figure 4Figure 4

BildpunkrasterPixel grid
image point gridimage point grid
Hintergrundbackground
backgroundbackground
PrüfobjektTest object
examination subjectexamination subject

Claims (10)

  1. A process for the optical sorting of bulk material mixed with reject parts, such as agricultural products, drugs, ores, etc. in a colour sorting machine by this bulk material being conveyed over a transport belt and being moved past an observation head with a light source and a product signal receiver arranged in the vicinity of the light source, whereby the reflected light of the image points of the examination material is broken down by various colour filters of detection elements lying next to one another in a line of the receiver into several colour components and the examination material is sorted on the basis of the colour values corresponding to the measured intensities of the respective colour components,
    characterised in that in each case the colour values of the product are examined in several selected sub-regions of the colour space by, in each sub-region, a classifier ascertaining connected areas of image points with colour values falling into the respective sub-region and carrying out a classification according to preset criteria from the geometry and size of these detected areas.
  2. A process according to Claim 1,
    characterised in that
    the examination material is surveyed in a prelearning process without reject parts and its colour value frequency distribution is determined dependent on the colour;
    in a relearning process, in a first step using examination material without reject parts, a colour value region is defined for good examination material by laying a threshold based on experience over the frequency distribution of the colour values, wherein the limits of the colour value region of the examination material result from the intersection points between the threshold and the curve of the frequency distribution;
    ascertained in the relearning process, using examination material without reject parts, are measured values suspected of containing foreign objects in dependence on colour, which according to the limits of the colour value region of the examination material from the first step of the relearning process would mistakenly be classified as reject, and the size of the local accumulation of these measured values is determined; and
    in the relearning process, using examination material without reject parts, when a preset magnitude of this local accumulation of measured values suspected of containing foreign objects is exceeded, the threshold value decision of the first step of the relearning process is changed dependent on colour so that for these measured values a decision is made for good examination material.
  3. A process according to one of the preceding Claims,
    characterised in that during examination of the examination material mixed with reject parts, the classifiers operating in parallel only analyse sub-regions of the colour space in which reject parts are suspected.
  4. A process according to Claim 3,
    characterised in that in the sub-regions of the colour space in which reject parts are suspected, the colour value distribution of reject parts is learnt by displaying them.
  5. A process according to one of the preceding Claims,
    characterised in that the examination of the examination material mixed with reject parts takes place with a first classification system, while the current frequency distribution of the colour values of the examination material is measured for adaptation to systematic drift-like changes of the examination material with a second classification system, the measurement by the examining first classifier being monitored so that no reject values are detected during the measurement.
  6. A process according to Claim 5,
    characterised in that both classification systems alternate in their function.
  7. A process according to one of the preceding Claims,
    characterised in that by comparing colour signals of adjacent image points, large gradients and these generally disrupted colour values are not taken into account during the measurement.
  8. A process according to one of the preceding Claims,
    characterised in that the sorting of the material takes place in flight, when the material passes from one belt to another, for example.
  9. A process according to one of the preceding Claims,
    characterised in that the measurement background is a dark hole.
  10. A process according to one of Claims 1 to 8,
    characterised in that the measurement background is designed as a cylindrical radiator with a rotating transparent roller surrounding the radiator, wherein the radiator transmits light in a colour matched to the examination material.
EP94250285A 1993-12-28 1994-11-25 Method for optically sorting bulk material Expired - Lifetime EP0661108B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE4345106A DE4345106C2 (en) 1993-12-28 1993-12-28 Process for the optical sorting of bulk goods
DE4345106 1993-12-28

Publications (3)

Publication Number Publication Date
EP0661108A2 EP0661108A2 (en) 1995-07-05
EP0661108A3 EP0661108A3 (en) 1997-02-12
EP0661108B1 true EP0661108B1 (en) 1999-11-03

Family

ID=6506597

Family Applications (1)

Application Number Title Priority Date Filing Date
EP94250285A Expired - Lifetime EP0661108B1 (en) 1993-12-28 1994-11-25 Method for optically sorting bulk material

Country Status (8)

Country Link
US (1) US5586663A (en)
EP (1) EP0661108B1 (en)
JP (1) JP3517292B2 (en)
AT (1) ATE186242T1 (en)
BR (1) BR9405268A (en)
CA (1) CA2136779C (en)
DE (2) DE4345106C2 (en)
HK (1) HK1013038A1 (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5577733A (en) * 1994-04-08 1996-11-26 Downing; Dennis L. Targeting system
DE19511534C2 (en) * 1995-03-29 1998-01-22 Fraunhofer Ges Forschung Method and device for detecting 3D defects in the automatic inspection of surfaces with the aid of color-capable image evaluation systems
DE19511901A1 (en) * 1995-03-31 1996-10-02 Commodas Gmbh Device and method for sorting bulk goods
JPH0943058A (en) * 1995-05-23 1997-02-14 Olympus Optical Co Ltd Apparatus for classifying color and apparatus for examining unevenness of color
EP0775533A3 (en) * 1995-11-24 1998-06-17 Elpatronic Ag Sorting method
DE19609916A1 (en) * 1996-03-14 1997-09-18 Robert Prof Dr Ing Massen Optical process for identifying materials, especially recycled plastics
ATE291969T1 (en) 1999-04-30 2005-04-15 Binder Co Ag METHOD AND DEVICE FOR SORTING WASTE PAPER
US6936784B2 (en) * 2002-05-28 2005-08-30 Satake Usa, Inc. Illumination source for sorting machine
MX2008012726A (en) * 2006-04-04 2009-01-16 6511660 Canada Inc System and method for identifying and sorting material.
US20110068051A1 (en) * 2009-05-22 2011-03-24 6358357 Canada Inc. Ballistic separator
US9234843B2 (en) * 2011-08-25 2016-01-12 Alliance For Sustainable Energy, Llc On-line, continuous monitoring in solar cell and fuel cell manufacturing using spectral reflectance imaging
US10480935B2 (en) 2016-12-02 2019-11-19 Alliance For Sustainable Energy, Llc Thickness mapping using multispectral imaging

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3560758A (en) * 1968-01-08 1971-02-02 Conductron Corp Color identification system taking into account the color and reflecting of the base material
DE2544703C3 (en) * 1975-10-07 1978-04-06 Dr.-Ing. Rudolf Hell Gmbh, 2300 Kiel Method and circuit arrangement for recognizing the colors of a colored surface
US4122951A (en) * 1977-02-28 1978-10-31 Alaminos Jose I L Machine for the automatic detection of blemishes in olives and other fruits
US4246098A (en) * 1978-06-21 1981-01-20 Sunkist Growers, Inc. Method and apparatus for detecting blemishes on the surface of an article
IT1205622B (en) * 1982-12-21 1989-03-23 Illycaffe Spa PROCEDURE TO MAKE A SELECTION IN A GRANULIFORM MATERIAL AND MACHINE TO IMPLEMENT THE PROCEDURE
GB2151018B (en) * 1983-12-06 1987-07-22 Gunsons Sortex Ltd Sorting machine and method
JPS63119887A (en) * 1986-11-06 1988-05-24 カネボウ株式会社 Selector
DE3802681C2 (en) * 1987-01-30 1997-01-09 Fuji Photo Film Co Ltd Method for classifying color templates and device therefor
US5085325A (en) * 1988-03-08 1992-02-04 Simco/Ramic Corporation Color sorting system and method
EP0342354A3 (en) * 1988-04-15 1992-01-08 Tecnostral S.A. Industria E Tecnologia Color sorting apparatus
JPH0670590B2 (en) * 1988-09-10 1994-09-07 倉敷紡績株式会社 Color order determination method
NL8803112A (en) * 1988-12-19 1990-07-16 Elbicon Nv METHOD AND APPARATUS FOR SORTING A FLOW OF ARTICLES DEPENDING ON OPTICAL PROPERTIES OF THE ARTICLES.
US5062714A (en) * 1990-02-12 1991-11-05 X-Rite, Incorporated Apparatus and method for pattern recognition
DE4210157C2 (en) * 1992-03-27 1994-12-22 Bodenseewerk Geraetetech Process for sorting broken glass

Also Published As

Publication number Publication date
CA2136779A1 (en) 1995-06-29
JP3517292B2 (en) 2004-04-12
CA2136779C (en) 2004-04-06
DE4345106A1 (en) 1995-06-29
JPH08206611A (en) 1996-08-13
DE4345106C2 (en) 1995-11-23
US5586663A (en) 1996-12-24
DE59408885D1 (en) 1999-12-09
BR9405268A (en) 1995-09-19
EP0661108A3 (en) 1997-02-12
HK1013038A1 (en) 1999-08-13
EP0661108A2 (en) 1995-07-05
ATE186242T1 (en) 1999-11-15

Similar Documents

Publication Publication Date Title
DE19519861C2 (en) Method and device for detecting and removing foreign objects
DE69731651T2 (en) SORTING DEVICE
EP0661108B1 (en) Method for optically sorting bulk material
DE60213133T2 (en) DEVICE AND METHOD FOR DETECTING OVERLAPS
DE69822608T2 (en) Binarization method in a character recognition system
EP1158460B1 (en) Image processing system and method
EP1030740A1 (en) Method and device for identifying and sorting objects conveyed on a belt
DE2820661C2 (en)
DE2549457A1 (en) DEVICE FOR AUTOMATIC MONITORING OF FINISHED PRODUCTS FOR FACTORY DEFECTS
DE3347472A1 (en) METHOD AND DEVICE FOR DETECTING RADIAL REFLECTIVE CONTAINER DEFECTS
DE2937335A1 (en) TEST SYSTEM FOR THE EXTERNAL APPEARANCE OF OBJECTS
DE69723997T3 (en) WAFER INSPECTION SYSTEM FOR THE DISTINCTION OF HOLES AND DUST PARTICLES
DE4340918C2 (en) Method and device for generating a uniform amplitude signal in a product stream to be sorted
WO1998012543A1 (en) Automatic, optical quality control process and device for flat, even products
DE19607258A1 (en) Object size and/or position determination device
EP0669527A1 (en) Method for extracting reusable bottles from their usage circuit
DE10110994B4 (en) Device for image scanning of an object
EP2144052A1 (en) Method and device for detecting and classifying defects
DE3514313C2 (en) Method and device for the inspection and sorting of glass containers
DD273505A5 (en) Method for the automatic detection of errors in moving webs
DE4434475C2 (en) Method and device for quality control of an object, in particular a compact disc
DE102018107305A1 (en) Device and method for optical position detection of transported objects
WO2008077680A1 (en) Method and device for optically testing objects
DE19504932A1 (en) Image processing controlled sorting of loose objects
EP1581909B1 (en) Method and arrangement for the automatic recognition of objects and structures in materials

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LI LU MC NL PT SE

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: H.F. & PH.F. REEMTSMA GMBH & CO

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LI LU MC NL PT SE

17P Request for examination filed

Effective date: 19970306

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: H.F. & PH.F. REEMTSMA GMBH

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

17Q First examination report despatched

Effective date: 19990121

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LI LU MC NL PT SE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: THE PATENT HAS BEEN ANNULLED BY A DECISION OF A NATIONAL AUTHORITY

Effective date: 19991103

Ref country code: GR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 19991103

Ref country code: ES

Free format text: THE PATENT HAS BEEN ANNULLED BY A DECISION OF A NATIONAL AUTHORITY

Effective date: 19991103

REF Corresponds to:

Ref document number: 186242

Country of ref document: AT

Date of ref document: 19991115

Kind code of ref document: T

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 19991103

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 19991125

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 19991125

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 19991130

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 19991130

ITF It: translation for a ep patent filed

Owner name: ING. A. GIAMBROCONO & C. S.R.L.

REF Corresponds to:

Ref document number: 59408885

Country of ref document: DE

Date of ref document: 19991209

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: THE PATENT HAS BEEN ANNULLED BY A DECISION OF A NATIONAL AUTHORITY

Effective date: 20000103

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: GERMAN

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20000203

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20000203

ET Fr: translation filed
REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed
REG Reference to a national code

Ref country code: GB

Ref legal event code: IF02

NLS Nl: assignments of ep-patents

Owner name: REEMTSMA CIGARETTENFABRIKEN GMBH

REG Reference to a national code

Ref country code: FR

Ref legal event code: CD

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

REG Reference to a national code

Ref country code: FR

Ref legal event code: CA

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20071104

Year of fee payment: 14

Ref country code: DE

Payment date: 20071122

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20071127

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20071121

Year of fee payment: 14

Ref country code: FR

Payment date: 20071108

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20071130

Year of fee payment: 14

BERE Be: lapsed

Owner name: *REEMTSMA CIGARETTENFABRIKEN G.M.B.H.

Effective date: 20081130

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20081125

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090601

NLV4 Nl: lapsed or anulled due to non-payment of the annual fee

Effective date: 20090601

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20081125

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20090731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20081130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090603

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20081125

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20081130