EP2671651A1 - Appareil et procédé dýinspection et de tri dýun flux de produits - Google Patents

Appareil et procédé dýinspection et de tri dýun flux de produits Download PDF

Info

Publication number
EP2671651A1
EP2671651A1 EP13170983.4A EP13170983A EP2671651A1 EP 2671651 A1 EP2671651 A1 EP 2671651A1 EP 13170983 A EP13170983 A EP 13170983A EP 2671651 A1 EP2671651 A1 EP 2671651A1
Authority
EP
European Patent Office
Prior art keywords
products
signal
light
background element
product
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP13170983.4A
Other languages
German (de)
English (en)
Other versions
EP2671651B1 (fr
Inventor
Dirk Adams
Pieter Op De Beeck
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Visys NV
Original Assignee
Visys NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/491,342 external-priority patent/US20130056398A1/en
Application filed by Visys NV filed Critical Visys NV
Priority to PL13170983T priority Critical patent/PL2671651T3/pl
Publication of EP2671651A1 publication Critical patent/EP2671651A1/fr
Application granted granted Critical
Publication of EP2671651B1 publication Critical patent/EP2671651B1/fr
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
    • B07C5/34—Sorting according to other particular properties
    • B07C5/342—Sorting according to other particular properties according to optical properties, e.g. colour
    • B07C5/3425—Sorting according to other particular properties according to optical properties, e.g. colour of granular material, e.g. ore particles, grain

Definitions

  • the current invention relates to a method and apparatus for inspecting a stream of products. This inspection can further result in a quality selection by means of a sorting step.
  • the invention is applicable when removing certain degraded products and foreign objects from an incoming stream of products.
  • the invention is particularly suited for sorting food products such as green beans, peas, nuts, raisins, cauliflowers, lettuce and such like for which non-food products such as wood, plastics, glass and others need to be removed from the stream of products.
  • the invention is furthermore extremely suitable for sorting non-food products such as plastics from recycling garbage, sorting of glass and such like.
  • the light is reflected by a background element that needs to be chosen product dependently. More specifically, the background needs to be chosen such that it shows the optical characteristics of good product. In other words, the good product is invisible against the background.
  • deviations in an analyzed product such as insect bytes, putrescent stains, foreign objects and such cause a deviation in the returned light signal.
  • a disadvantage of this method is that for each kind of product a specific background element needs to be available.
  • a background element for carrots should have an orange color, while for green beans it should have a green color.
  • a background element for carrots should have an orange color, while for green beans it should have a green color.
  • the background element is given additional optical properties.
  • Many fresh vegetables contain for example chlorophyll.
  • a frequency shift towards the infra-red spectrum occurs within these molecules when illuminated with light having a wavelength between 640 and 680 nanometer. This emission phenomenon is called fluorescence.
  • fluorescence By giving the background element the same fluorescent properties one can, in the state of the art, additionally sort based on the presence of chlorophyll in the analyzed products.
  • Other molecules, such as aflatoxin show fluorescent properties as well and could in principle be detected in the analyzed products according to a similar technique.
  • the object of the invention is a method and apparatus to carry out a selection between products in a large, continuous stream of products in a very effective, reliable and cost-effective way.
  • the invention aims at an inspection method and inspection means avoiding the replacement of the background element during a product switch.
  • the object of the invention is an inspection method and inspection means using a background reference which is at least product domain independent.
  • the invention is related to an apparatus for sorting a stream of products, comprising :
  • the background element is an optical fiber with a grooved surface for capturing the incoming light.
  • the background element is a manifold of detectors configured to convert the incident light into an electric signal.
  • the invention is also related to a method for sorting a stream of products in products to be accepted and products to be rejected comprising the steps of:
  • the background element is an optical fiber with a grooved surface for capturing the incoming light.
  • the background element is a manifold of detectors configured to convert the incident light into an electric signal.
  • the method of sorting products according to the invention is characterized at least by transporting the products to sort along a certain trajectory in the form of a product stream having a width; a background element extending along the width of the product stream; with a concentrated light beam illuminating along the width of the product stream, the products to sort and, in absence of said products, the background element; capturing the light reemitted by the products and the background element; based on said observed light carrying out a first selection between the background element on the one hand and all the products in the product stream on the other hand; carrying out a second selection between the products to sort on the one hand and the products to be rejected on the other hand and based on this second selection automatically carrying out a separation of the products in said product stream.
  • the background element comprises a surface extending along the width of the product stream, whereby said surface reflects the incoming light at least partially.
  • the background element has the shape of a cylindrical roller.
  • the background element comprises means to capture and redirect the incoming light towards an opto-electrical convertor.
  • a background element generates a signal having a progression from which the presence or absence of products in the scanning zone can be easily deduced.
  • a thus obtained Boolean signal is particularly useful in the method according to the invention.
  • the first selection is preferably done based on whether or not the intensity of the detected light or a derived signal thereof crosses a threshold value.
  • the first selection is done based on whether or not the intensity of the observed light emitted by the background element or a derived signal thereof falls within a zone, said zone further characterized by a maximum threshold value.
  • the second selection is preferably done based on whether or not the intensity of the detected light or a derived signal thereof crosses a threshold value.
  • said crossing of a threshold value is exclusively defined within those zones which are labeled during the first selection step as originating from product.
  • a new signal is generated after the first selection, further characterized by preserving the intensity of the observed light in those zones outlining product (the product zones) and subsequently changing the intensity in the zones where the background element can be observed to another level.
  • Said filter could for example be an adaptive filter specifically tuned to smooth the transition from product zone to background zone and vice versa.
  • the second selection is done on said filtered signal.
  • the selection of the background element will be such that it leads to at least one corresponding signal having a path according to which a first selection can be carried out between said background element on the one hand and all products in the product stream on the other hand.
  • the scanning is done using a rotating mirror, preferably a fast rotating polygonal mirror.
  • the scanning uses a laser beam.
  • the products are transported on a vibrating table, belt or suchlike, towards an inspection installation.
  • the products are further guided during their free fall by a free-fall plate. Furthermore, the products to separate are segregated by means of a manifold of air valves positioned along the width of the product stream and opened based on the second selection step.
  • the method could be combined with color sorting by sorting based on light reflections.
  • two signals are combined in a two-dimensional graph such that each point in this graph corresponds with a specific intensity level according to the path of the first signal combined with a specific intensity level according to the path of the second signal; the points corresponding with the product to be accepted are grouped in first zone; the points corresponding with products to reject are grouped in a second zone; the points corresponding to the background element are outlined by a third zone; adjusting the level of the background signal is realized by repositioning the third zone to a new location.
  • moving said third zone can be achieved by visualizing this zone in a graph displayed in a graphical user interface and subsequently dragging this zone to a new location.
  • this said new location is chosen such that a separation can be made between the first and the third zone on the one hand and the second zone on the other hand using a separation plane.
  • the invention also refers to an apparatus to sort products using this method and such that it comprises at least a transport device to transport a stream of products, extending over a width, into a specific direction; means to scan the products to sort along the width of the product stream, further comprising means to generate a concentrated light and direct it towards means to cast this light beam onto the products; means to capture the returning light; means to carry out a selection between the scanned products based on the observed light; means to separate the products based on this selection.
  • the means to generate the concentrated light is a laser generator.
  • the means to cast the light beam onto the products comprises optical means, more specifically a rotating polygonal mirror, moving the concentrated light transversely across the product stream.
  • optical means more specifically a rotating polygonal mirror, moving the concentrated light transversely across the product stream.
  • the current invention is however not limited to such a scanning arrangement. It could, by way of example, as well generate a row of concentrated light beams, possibly turned on and off in sequence.
  • the means to carry out a selection based on the returning light could be based on digital electronic components, more specifically Field Programmable Gate Arrays and microprocessors, or could be based on analogue electronic components such as operational amplifier circuits, or it could be a combination of analogue and digital processing units.
  • the means to make a separation between the products based on said selection are composed of a manifold of air valves, mounted transversely across the product stream.
  • the background element is composed of a surface across the width of the product stream, such that the incident light is at least partially reflected by said surface.
  • the background element is composed of means that capture and channel the incident light towards means to convert this light into an electric signal.
  • the means to capture the returning light are composed of an optical filter making the detection arrangement sensitive for a specific light spectrum, in operable communication with a spatial filter making the detection arrangement sensitive for a specific zone of the returning light, and in operable communication with both filters an opto-electrical converter transforming the light into a corresponding electric signal.
  • a method for sorting a stream of products in products to be accepted and products to reject comprising the steps of moving through a scanning zone the products to sort, supplied in a product stream spanning a certain width and having a thickness of substantially a single layer of products, in this scanning zone linearly scanning one or more concentrated light beams across the width of this product stream, illuminating, in the absence of products, a background element positioned behind this product stream that extends over the width of it, whereby this light beam produces light signals at these scanned products and at this scanned background element, detecting these light signals whereby these light signals are converted in electric signals, generating one or more control signals on the basis of these converted signals whereby these control signals allow making a selection between the scanned products to be accepted on the one hand and the scanned products to be rejected on the other hand, and sorting the product stream by means of these one or more control signals, characterized in that this method further comprises; choosing this background element) such that the corresponding detected light signal differs in at least 1 parameter from the light signals of the products to sort and
  • This method can further comprise, after moving the background level of the converted signals, comparing the thus obtained signal with one or more threshold values to generate in this way the one or more control signals.
  • Moving the background level according to this method can further comprise generating a signal which is indicative of the location of the scanned products in the detected and converted signals and shifting the level of the converted signals to locations other than these of the scanned products such as have been indicated by this location signal.
  • Generating the location signal according to this method can further comprise detecting and converting the light signals originating from the scanned background element and from the scanned products, in these converted signals separating the signal originating from the scanned background element from the signals of the scanned products such that a signal is obtained indicative of the location of the scanned products.
  • Distinguishing the signal originating from the scanned background element according to this method can further comprise comparing the converted signals with one or more threshold values.
  • generating the location signal can further comprise detecting and converting that zone of the concentrated scanning light beam passed by the products, as such obtaining a signal that is indicative of the location of the scanned products.
  • the location signal in the different embodiments of this method can be an analogue or digital signal, whereby comparing the thus obtained signal with one or more threshold values happens only on the location of the scanned products as indicated by the location signal.
  • the background level of the converted signals can be shifted towards a signal level according to that of a product to be accepted.
  • the location signal in any embodiment can be generated on the basis of one or more first detected signals and afterwards used to indicate the location of the products in one or more second detected signals.
  • the background element in any of the embodiments can consist of a surface that extends over the width of the product stream, whereby said surface at least partially reflects the incident light.
  • this background element has the shape of a cylindrical roller.
  • this background element consists of means to capture and redirect the incident light towards a opto-electrical convertor.
  • the background element can be an optical fiber with a grooved surface to capture the incoming light.
  • Such optical fiber can consist of segments, whereby for each segment the orientation of these grooves with respect to the longitudinal direction of the optical fiber is chosen in function of the position of this segment along the width of the product stream.
  • the thus obtained signal can be filtered prior such that the high-frequency transitions near the product zones are flattened, generating a new filtered signal.
  • This filter can be an adaptive filter which is adjusted specifically to smooth the transitions from a product zone to a background zone and vice versa.
  • the linear scanning of the product stream can happen by means of a moving mirror, preferably a rapidly rotating polygonal mirror.
  • the product stream can be scanned by several concentrated light beams by means of this moving mirror, whereby every light beam has a separate frequency.
  • this concentrated light beam is a laser beam.
  • the products to sort can be scanned from both edges of the product stream.
  • the supply of the product stream can happen by means of a vibrating table, a conveyor belt or suchlike.
  • the products can further be supplied by means of a free-fall plate which guides these products during their free fall towards the scanning zone.
  • the sorting of the product stream by means of these one or more control signals can happen by controlling a manifold of air valves positioned across the width of the product stream by means of these one or more control signals.
  • sorting the product stream can also happen on color, i.e., the frequency of the detected light signals.
  • Moving said third zone can happen by visualizing this zone in a graph displayed in a graphical user interface and subsequently dragging this zone to a new location.
  • This said new location can be chosen such that a separation can be made between the first and third zone on the one hand and the second zone on the other hand.
  • This two-dimensional graph can have an additional dimension showing the histogram of appearing signal combinations.
  • more than two detected signals can be combined with each other in more-dimensional graph so that, for every location in the scanning zone, as much as possible optical information is collected and depicted, allowing to make a better distinction between the location of the products to be sorted and these of the background on the one hand and between the products to be accepted and products to be rejected on the other hand.
  • All or at least one of said first, second and third zones in such two or more-dimensional graph can be inferred by automatic clustering algorithms.
  • An apparatus for sorting products characterized by at least consisting of a supply system transporting a single layer of the products to be sorted along a certain trajectory in the form of a product stream extended over a width; means to scan to products to be sorted over the width of this product stream, whereby these scanning means further comprise; means to generate a concentrated light beam and direct it towards the products via optical means; means to detect the returning light and convert it to an electric signal; means to generate control signals enabling to carry out a selection between the scanned products based on said detected light; and means to sort the product stream based on said selection by means of said one or more control signals, characterized in that the sorting apparatus further comprises; a background element chosen such that the corresponding detected light signal differs in at least 1 parameter from the light signals of the products to be sorted and whereby the selection means comprise means to generate one or more control signals by shifting the background level of the light signals towards a signal level chosen such that, in the thus obtained signal, the signal level of a scanned product
  • the selection means of such sorting device can further comprise means to generate a location signal based on one or more detected signals, means to obtain a signal based on this location signal and based on these or other one or more detected signals, such that the background level in these last signals is shifted to a new level enabling to differentiate products to be accepted from products to be rejected in said obtained signal.
  • These selection means can further comprise means to compare the obtained signal with one or more threshold values, in this way generating the one or more control signals.
  • These selection means can further comprise filters to prefilter the thus obtained signal so that the high-frequency transitions close to the product zones are flattened and thus obtain a new filtered signal.
  • This filtering operation can be achieved by means of an adaptive filter specifically adjusted to smoothing the transitions from a product zone to a background zone and vice versa.
  • this background element can consist of a surface that extends over the width of the product stream, whereby said surface reflects the light at least partially.
  • this background element has the shape of a cylindrical roller.
  • this background element can consist of means to capture and redirect the incident light towards an opto-electrical convertor.
  • Such background element can be an optical fiber with a grooved surface to capture the incident light.
  • Such optical fiber can consist of segments, whereby for each segment the orientation of these grooves with respect to the longitudinal direction of the optical fiber is chosen in function of the position of this segment along the width of the product stream.
  • a laser can be used in such sorting apparatuses to capture the concentrated light beam.
  • This laser can be moved across the width of the product stream by means of a rotating polygonal mirror.
  • the means to make a selection between the scanned products in function of the observed light consist of a signal processing platform based on digital electronic components, more specifically Field Programmable Gate Arrays and microcomputer processors, or based on analogue electronic circuits, such as op-amp circuits, or a combination of analogue and digital processing units.
  • the means to sort the product stream in function of said selection by means of these one or more control signals consist of a manifold of air valves, positioned transversely across the product stream.
  • the means to capture the light can consist of an optical filter making the detector sensitive to a particular light spectrum; in operational communication with a spatial filter making the detector sensitive to a particular zone of the returning light; in operational communication with both said filters an opto-electrical convertor converting the light to a corresponding electric signal.
  • Figure 1 illustrates schematically the fundamental operation of a sorting apparatus according to the invention
  • Figure 2 illustrates schematically a possible embodiment of a scanning device
  • Figures 3a and 3b show alternative embodiments of the background element.
  • Figure 4 illustrates schematically a detection unit
  • FIG. 5 illustrates schematically an apparatus with several detection units
  • Figures 6a-e illustrate the method in several steps according to the current invention which results in a better or at least more advantageous inspection
  • Figures 7a-b illustrate this method in a two-dimensional representation
  • Figure 8 shows a sorting apparatus according to the invention in viewing perspective.
  • Figures 9a-f illustrate schematically the signal processing in an inspection device when the background element generates a signal that deviates from a good product
  • FIGS 10a-d illustrate schematically the processing of the signal according to an embodiment of the invention
  • FIGS 11a-d illustrate schematically the processing of the signal according to an embodiment of the invention
  • FIGS 12a-d illustrate schematically the processing of the signal according to an embodiment of the invention
  • FIGS 13a-c illustrate schematically the processing of the signal according to an embodiment of the invention
  • FIG. 14 illustrates schematically the processing of the signal according to an embodiment of the invention
  • FIGS 15a-c illustrate schematically the processing of the signal according to an embodiment of the invention
  • the current invention teaches a method and a mechanism for sorting products 2, 3, to be precise, granular products like raisins, beans, berries, but also plastic grains, that are conveyed in large quantities and in a continuous stream.
  • the method according to the invention is also suitable for inspecting larger products like broad beans, cauliflower, lettuce, etc.
  • FIG. 1 shows schematically a sorting apparatus 14 according to the current invention.
  • This sorting apparatus contains a supply system 1, at least one inspection unit 9,10, a reject system 11 and possibly a free fall glide 4 that guides the product stream in free fall to the inspection unit 9,10 and the reject system 11.
  • the glide can also be a conveyor belt that conveys the products 2, 3.
  • the supply system 1 with width W can be a vibrating table, or any other conveyor system known in the current state of the art.
  • a free fall glide 4 can be superfluous, as is well known by the person skilled in the art.
  • the inspection unit watches the falling product 2,3 by analyzing the returning light.
  • a reject system 11 is being controlled. This results in a separation of the product stream in an accept stream 13 and a reject stream 12.
  • a background element 5 is illuminated and observed while no product is present 2,3 in that zone 28.
  • the optical properties of the background element 5 are chosen in such a way that a proper distinction can be made between all products 2,3 in the product stream on the one hand and the background element on the other hand. These optical properties can refer to the frequency or to spatial properties of the background element 5.
  • the background element can generate a light signal with another frequency or reflect the incident light 34 in another way, or even scatter it. This method differs from the current state of the art, which tries to make a distinction between all the products to be rejected 3 from the product stream on the one hand and the background 5 together with the products to be accepted 2 on the other hand.
  • the choice of the background element 5 becomes considerably less complicated and independent of the product domain.
  • the background element 5 will be the same for green beans and orange carrots.
  • the current state of the art in which a specific background element for every product has to be available which has optical properties identical to those of the products to be accepted 2.
  • the current state of the art would need two different background elements: one with a green color, and another with an orange color.
  • the background element 5 is a cylindrical roller, to be precise, a rotating roller that cleans itself by means of a scraper that is placed against the roller.
  • the background element 5 is implemented as a white, strongly scattering, not fluorescing, cylindrical roller.
  • a signal 39 measured by a detection unit 44, only sensible to the scattered light, will in this case show a path B along which a proper distinction can be made between the background signal 19 on the one hand, and the peaks 20,21 originating from green beans, peas, carrots, wood, plastic, metal, glass etc. on the other hand.
  • the background element 5 is an intrinsic component of the mechanical construction and contains, besides the function of optical background, an additional function as mechanical support element. As a result, it can, as such, not be removed.
  • Such greatly simplified mechanical embodiment according to the current invention is impossible to implement in the current state of the art.
  • the inspection unit 9,10 consists of a scanning unit 43 and a detection unit 44 ( figures 4 and 5 ).
  • the scanning unit illuminates the product stream along the entire width W.
  • the detection unit 44 captures a zone of the returning light and will convert this returning light by at least one opto-electrical converter 38 to an electric signal that is afterwards analyzed in a processing unit 41.
  • FIG. 2 illustrates a possible embodiment of a scanning unit 43.
  • a concentrated light beam 45 preferably originating from a laser 29, is directed towards a fast rotating, reflecting polygon wheel 30. During the rotation, this polygon 30 generates a fast moving concentrated point of light directed towards the product stream.
  • the scanning is performed along the entire width W with a scanning angle determined by the extreme light beams 31,31'.
  • the concentrated light beam 32,33,34 will be reflected directly and/or scattered, as is elaborately described in the American patent US4634881 . This makes structure sorting possible. Additionally, the presence of fluorescing molecules in the product 2,3 will cause a frequency shift in the reflected light, making it possible to sort on the presence of those molecules, like chlorophyll and aflatoxin.
  • various light beams with different wavelengths can be bundled, preferably by combining various lasers of different wavelengths by means of mirrors and optical filters.
  • this element consists of means to capture the incident light 34 and to direct it to a detector 40 that converts this light into an electric signal 39.
  • the background element 5 can consist of an optical fiber 56, on which, by means of a closer specified mechanism 57, the incident light 34 can be further channeled to a detector 40 that generates an electric signal 39.
  • the element 57 can for example consist of grooves or little mirrors attached to the fiber 56, so that the light gets bended towards the abovementioned filter 56.
  • figure 5 of WO 2007/062219 or figure 2 of the SPIE article width, pitch d, angle of the groove in relation to the normal direction perpendicular on the longitudinal direction, angle of the groove in relation to the longitudinal direction) and of the optical parameters (wavelength of the light source, refraction index of the optical fiber), the direction and the degree of the redirection can be determined.
  • the grooves can be applied by means of a printer that uses laser light or UV light (cf. chapter 2, second paragraph of the SPIE article, p.9 third paragraph illustrated by figure 5 of WO 2007/062219 ). By varying the relative orientation of the optical fiber in relation to the printer, the angle under which the grooves are formed in the side of the optical fiber can be determined.
  • Such optical fiber can, however, also be used to capture light originating from a linearly moving concentrated light beam 34 and to redirect it in the longitudinal direction of the optical fiber to an exit as illustrated in figure 3a .
  • a grooved optical fiber of arbitrary length can be obtained by coupling separate segments (cf. SPIE article, chapter 2, last paragraph). Therefore, an optical fiber can be made that extends over de entire width W of the product stream. Accordingly, for every immediate position of the concentrated light beam, such a grooved optical fiber can capture the, in the absence of products, uninterrupted light 34 and redirect it to a detector 40.
  • the angle, formed by the uninterrupted light and the optical fiber used as background element 5 in figure 3a depends on the position according to the width W, it can be necessary to vary the orientation of the grooves along the length of the optical fiber.
  • the uninterrupted light bundle 34 will enter the fiber substantially perpendicularly in the middle of the product stream, if the inspection unit 10 is set up symmetrically in relation to the width of the product stream. Moving to the edges of the product stream, the uninterrupted light beam 34 will, however, incide on the optical fiber 5 under a certain angle.
  • the optical fiber can consist of coupled segments. For every segment of the optical fiber, the orientation of the grooves can be kept the same. This can lead to an efficient production of these segments. Every segment in the optical fiber 5 can be placed under another angle, depending on the position according to the width of the product stream. A segment in the middle of the product stream will be placed in a substantially parallel position in relation to the product stream, while segments on the edges of the product stream are placed under an angle in relation to the product stream, correspondent to the angle formed by the light beam 34 and the product stream. Because of this variable orientation of the segments throughout the width of the product stream, the uninterrupted light 34 will always incide on the grooves under substantially the same angle and will be captured and redirected in the same way. Preferably, the segments are placed on an arc described by the concentrated light beam 45 when scanning the product stream.
  • the optical fiber can consist of coupled segments.
  • the orientation of the grooves can be changed. After all, the orientation of the grooves can be chosen separately for every segment by setting the relative orientation of the printer accordingly.
  • the orientation of the grooves of a segment can therefore be adapted according to its position in relation to width of the product stream and to the angle formed by the inciding light beam 34 and the product stream at that point.
  • all segments of the optical fiber will be placed substantially parallel to the product stream.
  • the orientation of the grooves of a segment will, however, depend on the position along the width of the product stream according to the angle formed by the inciding uninterrupted light bundle 34 and this position. This way, the uninterrupted light 34 will each time form substantially the same angle with the grooves. The light will also be captured and redirected in the same way.
  • an alternative as illustrated in figure 3b consists of implementing the background element 5 as a manifold of little detectors 40' that convert the incident light 34 in an electric signal 39.
  • the background element can be constructed as a linear array or line of light sensitive elements, such as photodiodes or Photo Multiplier Tube (PMT) or other elements known to the person skilled in the art. Because such linear array is usually constructed as a line of separate light sensitive elements 40', i.e. separated from each other, it is possible that an uninterrupted light bundle 34 does not incide on a light sensitive element 40'. One could wrongly conclude that a product 2,3 was present in the product stream that obstructed the concentrated light bundle.
  • PMT Photo Multiplier Tube
  • This light scattering coating can for example be a milky plastic layer or a glass plate.
  • This limited light scattering one can make sure that two or more nearby light sensitive elements 40' get illuminated, even if the uninterrupted light beam 34 should reach the background element 5 between two such light sensitive elements 40'. After all, it is important to know whether an uninterrupted light beam 34 reaches the background element 5, rather than the position where the background element 5 is illuminated. This position can be deduced from the known immediate position of the scanning light bundle and by correlating the time period of the signal 47, coming from the background element 5, with the time period of the moving light bundle.
  • the electric ( figure 3b ) or optical ( figure 3a ) signal at the exit of such a background detection element 5, can be filtered to withhold only the signal components coming from the uninterrupted light beams 34, while the ambient light is being filtered away.
  • the signal components will typically have a higher frequency than signal components coming from background light.
  • a DC-filter or high-pass filter can usually be enough for only allowing the wanted higher-frequency signal components, characteristic for the presence of a product 2,3, passage for subsequent signal processing as discussed in this description.
  • Figure 4 shows schematically a detection unit 44, struck by the incident light or light cone 46, 47, 48, and that subsequently converts said light, or a particular zone of it, into an electric signal by means of an opto-electrical convertor 38.
  • This electric signal 39 is given as input to a processing unit 41, which by means of an analytic method generates a control signal 42 that controls a reject system 11.
  • optical filters 36 can be used to render the detection unit 44 sensitive to one specific wavelength by placing this filter 36 into operational communication with abovementioned opto-electrical convertor 38.
  • a spatial filter 37 can be used to block or to let through certain zones of the returning light 46,47,48.
  • a spatial filter 36 can be used, which only lets through the scattered light.
  • Such spatial filters are described in the American patents US4634881 and US4723659 .
  • the spatial filter 36 consists of a diaphragm that is placed right before the opto-electrical convertor 38.
  • more detection units 44, 44' can be set up according to the invention.
  • every detection unit 44, 44' uses a different combination of optical 36 and spatial 37 filters. Because of this, every detection unit 44,44' is sensitive to a specific zone of the returning light 46,47,48 having a specific wavelength.
  • the output signals 39, 39' are representative of a specific zone of the returning light 46,47,48 on a specific wavelength.
  • the first detection unit 44 generates a first electric signal 39 with a level determined by the abovementioned optical and spatial filters chosen for that detection unit.
  • the second detection unit 44' generates a second electric signal 39' with a level determined by the abovementioned optical and spatial filters chosen for that detection unit.
  • the detection units 44,44' are in operational communication with the processing unit 41 via the signals 39,39'.
  • the processing unit 41 will perform a selection between the scanned products 2,3 and the background element 5, in function of the returning light 46,47,48, more specifically based on the electric signals 39,39'.
  • the processing unit 41 is a digital processing platform based on Field Programmable Gate Arrays or microprocessors.
  • the processing unit 41 could, however, also consist of analog op-amp circuits or a combination of analog and digital components as is known by the person skilled in the art.
  • An apparatus described in the previous paragraphs by combining a number of the above-described features is an apparatus 14 for sorting a stream of products 2, 3, comprising :
  • the detection unit 44 may consist of one or more separate units 44/44' as shown in figures 4 and 5 , arranged to receive light beams 46,47,48 reflected off the products 2,3 and the background element 5. The received light may be converted into electric signals and processed further according to the methods described elsewhere in this description.
  • the detection unit 44 is a camera, configured to capture images of the falling products as they are passing the scanning zone 28 in front of the background element 5.
  • the camera captures an image of the scanning zone 28.
  • the background element 5 consists of means to capture incident light from the scanning unit and to direct it to a detector 40, or the background element consists of an array of detectors 40' configured to convert said light into an electric signal (i.e. the background element is produced according to figure 3a or 3b ).
  • the camera can be sensitive to (i.e. capable of detecting) the reflected laser light beams 46,47,48 reflected off the products and the background element.
  • the camera acts as a detection unit that is completely equivalent to the detection unit 44 shown in figures 4 and 5 .
  • the scanned laser light 46,47,48 that is reflected off the products 2,3 and the background element is not used directly for performing the selection.
  • the camera is equipped with filters that make the camera insensitive to said reflected laser light. Instead, light from the environment or from a separate light source (i.e. separate from the scanning means), e.g. a row of LEDs emitting a pulsed LED illumination, is reflected by the products in the scanning zone and captured by the camera.
  • the camera is sensitive both to the reflected laser light 46,47,48 and to the light from the light source or the environment reflected off the products and the background.
  • the separate light source e.g. the row of LEDs, may be included in the apparatus according to the invention.
  • the camera may be a focal plane array camera, for instance a solid state based camera or a CCD based camera, mounted suitably in relation to the product stream.
  • a possible camera setup that is suitable for an apparatus according to this embodiment is shown in patent document WO2008/116924A2 , incorporated herein by reference.
  • the difference between an apparatus according to the latter embodiment and the apparatus of WO2008/116924A2 is that the background element in an apparatus of the invention consists of means to capture incident light from the scanning unit and to direct it to a detector means 40,40' that is configured to convert said light into an electric signal 39.
  • the detector means is either a detector 40 arranged to receive the light 56 from a grooved fiber as in figure 3a , or it is a manifold of detectors 40' arranged along the surface of the background element 5 as in figure 3b .
  • an apparatus equipped with a camera then comprises means for producing a location signal that is indicative of the location of products along the width W of the scanning zone, wherein said location signal is based on the detected laser light falling on the background element and thus passing by the products 2,3, i.e. the light that is not blocked by products 2,3.
  • the method for deriving such a location signal is described further in this description, in relation to figure 13 .
  • the processing of said location signal together with the captured camera image allows a straightforward selection procedure to be established, by verifying whether or not electric signals related to the products 2,3 and obtained by processing the camera image are passing a pre-defined threshold.
  • the method according to one embodiment of the invention consists of the light 45 having at least one wavelength, being sent out towards an inspection zone 28.
  • This zone 28 is being scanned and when the abovementioned light 45 strikes the product 2,3 or the background element 5, the emission 46, 47, 48 will be captured by at least two detection units 44,44'.
  • factors m and p are equal to zero. This means in principle that no combination is made.
  • detection unit 44 generates the signal A, and detection unit 44' the signal B.
  • a detection unit 44 having an optical filter 36 set to the light spectrum between 690 and 740 nanometers, more particularly the fluorescence spectrum of product 2 containing chlorophyll when illuminated between 540 and 680 nanometer.
  • the signal A shows a possible path of such a set-up where a peak 16 is perceptible at the location of said product 2 containing chlorophyll.
  • signal A illustrates a path on which an emission phenomenon, that can only be attributed to the product to be accepted, is measurable.
  • no selection is possible on such a signal because of the above mentioned problem.
  • Zones 20,21 of the signal are the result of the emission that occurs on products 2,3, particularly as a consequence of the emission of both the products to be accepted 2 and the products to be rejected 3.
  • Zone 19 is the result of emission of the background element 5.
  • a zone 49 is defined in which the background signal 19 is situated. All zones 19 that are located within this zone 49, are labeled by the processing unit 41 as coming from the background element 5.
  • the zone 49 is determined by a maximum threshold value t max and a minimum threshold value t min . In an advantageous embodiment, these threshold values t max , t min can be adjusted by a user.
  • a Boolean signal C is introduced, whereby the value 0 is adopted at the locations of the background signal 54 and where the value 1 is adopted at the locations 18 outside the zone 49 on the signal B.
  • the Boolean signal C is effectively being generated or is directly available in the processing unit 41, for example in case of an embodiment as described in figures 3a en 3b, wherein the background element 5 generates a signal 39 according to the path of signal B, which can be transformed in a manner as described above to said signal C.
  • the selection between the products 2,3 and the background element 5 can take place using one threshold value t min only because the peaks 20,21 where the products 2,3 are located, are all on the same side of the background signal 19, all below (or all above) the background signal 19 to be precise.
  • the new signal D is an exact copy of the abovementioned signal A on places 20,21 of product 2,3.
  • a new value s is being established so that the background signal 22 clearly separates itself from the products to be rejected 3.
  • the path of signal A was interpreted as coming from the emission peak of a product 2 containing chlorophyll, against a non-fluorescing background.
  • the background element 5 together with the products 2 containing chlorophyll on the one hand, and the products to be rejected 3 on the other hand, making it possible to automatically remove the latter products.
  • the signal D will be additionally filtered by, for instance, a low-pass filter, generating a new signal E.
  • the low-pass filter is constructed according to known principles, for instance by means of digital multi-tap FIR filter.
  • the cut-off frequency will be chosen in a way that the high frequency transitions on the edges of the zones 20,21 in the signal D are flattened sufficiently, without losing the actual signal content. In this way, one obtains the zones 24, 25, where the products 2,3 are, and zone 23, where the background element 5 is.
  • the filtering is done by means of an adaptive filter that is tuned to apply a filtering to mainly only said transitions.
  • the cut-off frequency chosen can be much smaller.
  • the current invention is not restricted to the use of low-pass or adaptive filters to flatten out said transitions. To be precise, all methods to realize such a flattening fall within the scope of this invention.
  • an automatic detection can be carried out at the locations 25 wherein the products to be rejected 3 are located.
  • an automatic detection can be carried out at those locations 17 wherein the products to be rejected are located, based on whether or not the signal A crosses a threshold value g, by merely analyzing said crossing in those zones wherein the products 2,3 are located according to signal C.
  • the signals A, A', B, C, D, E are synchronized with each other. After all, these signals originate from the concentrated light beams 45, scanning the product stream in temporal movement. Every immediate value of one of these signals can therefore be correlated to the corresponding immediate value of the other signals. This synchronization allows applying one signal to another signal or combining both, because at every single moment the signals are coming from the same scanned position.
  • air valves 11 are opened so that, at the locations where these products 3 were detected, each such product 3 will be blown out of the product stream.
  • the operation of the reject system 11 is carried out by comparing control signals generated by the signals D, E, to one or more threshold values g i . These control signals only contain information about either the product to be rejected 3, if this has to removed, or the product 2 to be accepted, if it has to be withheld.
  • This method according to the current invention is, of course, not restricted to the use of two detection units.
  • the corresponding signals 42 can for instance be algebraically combined to abovementioned signals A and B.
  • the zone 49, in which the background signal is located is defined on various signals B, to be precise on all the output signals 39 of all the present detection units 44.
  • the final Boolean signal C that determines where the background 19 is located and where the products 2,3 are located, is obtained by performing a logical-OR operation to all the separate signals C that are obtained according to the abovementioned method.
  • the output signals A, A' of two detection units 44,44' are combined in a two-dimensional graph. Both signals A, A' originate from the same position in the product stream, but can differ in one or more signal parameters, so that different optical properties of the scanned position at that moment can be analyzed. These signals A, A' can be obtained by filtering them out of a same detected light signal using spatial and/or frequency filters. Every point in this graph corresponds to a specific intensity level according to the path of the first signal A, combined with a specific intensity level according to the path of the second signal A' at a specific moment in time, or, in other words, for a known immediate position of the concentrated scanning light beam 45.
  • Specific signal combinations can occur several times if products 2,3 with the same optical properties are being scanned or every time the background element 5 is being scanned.
  • a two dimensional histogram can be created.
  • a color can be attributed to every histogram value.
  • a two-dimensional intensity map 55 can be created. On this map 55 contours of equal intensity can be drawn. As shown in figure 7a , zones can be determined in which products with similar optical properties are clustered.
  • Every point in the two-dimensional diagram of figure 7a is being characterized by the intensity levels of the respective signals A, A' and by the corresponding value of the location signal C. Depending on the value of this location signal, the signal levels of a point in the histogram correspond to those of a product 2,3 or to the background element 5. Based on the information of the signal C for the individual signal combination A, A', it can be determined whether or not a product is concerned.
  • the zone 51 is being defined, in which the products to be accepted 2 are located, and the zone 52, in which the products to be rejected 3 are located.
  • the zone 50 determined by the respective threshold values t max , t min and t' max , t' min , is represented by a square in said intensity map 55 containing the background as is illustrated in figure 7a .
  • the signal combinations of a product to be rejected 3 can be located in between the threshold values T' max , T' min and T max , T min containing the points corresponding to the scanned positions on the background element 5.
  • Figure 6b illustrates how the signal level of background signal 47 is located within a strip 49, limited by one or both threshold values T max , T min .
  • the background level When the background element 5 would not show variation in the corresponding signal 47, the background level would be one single line in figure 6b and one single point in figure 7a . In a realistic embodiment this background level can vary such that one obtains a matching set of points within the rectangle T' max , T' min and T max , T min that defines the strip 49 in figure 7a . As demonstrated in figure 7a , some points belonging to a product to be rejected 3, can have signal levels that cannot be distinguished from the background element 5. This is illustrated in figure 7a by the overlap between strip 50 and zone 52. This is also illustrated in figure 6a wherein the signal on position 17 of a product 3 that is to be rejected, cannot be distinguished from the signal on position 15 coming from the background element 5.
  • a graphical user interface one can, by means of a simple operation as is illustrated in figure 7b , for example by a dragging move, move the square 50 to another location with coordinates n'n, such that the zone 52 that matches the products 3 to be rejected, can be isolated by means of a separation plane g'.
  • the positioning signal On the basis of the corresponding value of the positioning signal one can define for each point whether it refers to a product 2,3 or to the background element 5, regardless of the corresponding levels of signals A, A'.
  • this positioning signal C can be obtained starting from one or more observed light signals A, B whereby one can use this positioning signal C to indicate respectively, within this or other light signals, the position of the background 5 orthe products 2,3.
  • the zones 50, 51 and 52 can be automatically calculated by means of known clustering algorithms, for example K-means.
  • the invention is not restricted to one and two-dimensional presentations, but can be easily extended to three and more dimensional presentations, be it by means of one, two or three-dimensional projections in those cases.
  • Figure 8 displays a complete sorting apparatus 26 in viewing perspective.
  • This apparatus consists of a supply system 1, more specifically a vibrating table transporting a stream of products 2,3 in a certain direction 27 through the sorting apparatus. During its free fall the product is additionally guided by a free-fall plate 4.
  • the structure 26 is furthermore equipped with 2 inspection devices 9, 10. These inspection devices 9, 10 inspect an inspection zone 28 by means of a concentrated light beam that sweeps across the entire width W of the product stream. In the absence of products a background element is scanned that, according to the invention, may hold the optical characteristics of the products to be rejected.
  • the products 3 to be rejected are blown out of the product stream via the air valves.
  • the accepted products 2 are guided through a shaft 53 towards possible further production steps.
  • Figure 9b schematically reflects this combined signal B, obtained for a scan wherein two good 2 and two bad 3 products were scanned as in figure 9a .
  • the desired products 2 are to be retained in the product stream, while the undesired products 3,3' are to be rejected by the reject system 11.
  • the reject system 11 When neither a good product 2 nor a bad product 3, 3' is scanned, the reject system 11 must remain inactive. Although at that time no single product to be rejected is present in the system, the unnecessary activation of the reject system 11 could cause an undesired disturbance of the product stream.
  • the reference level 70 in this combined signal B is situated on the level of the background signal 47.
  • the bad products 3 have a signal peak 48 that stays underneath the signal peaks 46 of the good products and past the first threshold value g1, one can distinguish the good products 2 from the bad products 3 based on these signals 46, 48, which ultimately allows sorting the products in a stream of good products 13 and a stream of bad products 12.
  • the second threshold value g2 is chosen in such a way that the signal peaks 46 of the good products 2 would reach past this second threshold value g2, whereas the signal peaks 48' of such bad products 3' would not reach past the second threshold value g2, as indicated by the hatched area in figure 9d . Signals that are situated below this threshold value g2 then match products 3' that are to be rejected from the product stream. Not only the signal peaks of the good products 2 but also the signal peaks 48 of the other, bad products 3 reach past the second threshold value g2 and past the first threshold value g1 as described above.
  • a problem that can occur regarding the choice of the threshold value g2 in figure 9d is that all the signals, those of the good products 2 as well, pass partially underneath and partially above the threshold value g2 and could in this way be wrongly interpreted as originating from a bad product.
  • the signal peaks from the good products 2 will pass through such a strip and are possibly erroneously considered as originating from a product to be rejected 3'.
  • threshold values g1 and g2 By a suitable choice of the threshold values g1 and g2, one can identify from the combined signal as indicated in figure 9e , bad products 3, 3', characterized respectively by a high signal peak 48 and a low signal peak 48'. Only the signals 46 within the strip constituted by both threshold values g1-g2 are considered as originating from a good product 2. One could also describe these threshold values as respectively an upper limit g1 and a lower limit g2, together limiting a signal strip locating the signal peaks to which the reject system 11 may not react.
  • One way of solving this problem is by physically constructing the background element 5 in such a way that it provides a signal 47 that is comparable, for the detected optical parameter, to the signal 46 originating from a good product 2 and is thus situated within the strip constituted by both threshold values g1, g2.
  • the problem remains that one has to install for each product 2 a matching background element 5.
  • Another way of solving the problem is by shifting the reference level 70 of the combined signal B using signal processing techniques to a new value 71 that preferably matches the signal level 46 of a good product 2.
  • the signal peaks 48, 48' of the bad products 3,3' are being referenced in the processed signal D to the new reference level 71.
  • this new reference 71 is adjusted to the signal level 46 of a good product 2. This is illustrated in figure 10c .
  • the reference level 70 was situated on the background signal 47, as is indicated for signal B in figure 10b
  • the reference level 71 for the new signal D is substantially equal to the signal level 46 of the good products 2.
  • the matching signal peaks 48 for some bad products 3 will still stretch out from the new reference level 71 downwards.
  • These undesired products 3 still give rise to a negative signal peak even in the new reference framework 71 of the signal D.
  • the corresponding signal peaks 48' will now stretch out from the new reference level 71 upwards.
  • These are the undesired products that in the original reference framework 70 illustrated in figure 10b had a signal peak 48' that did not stick out past this 46 of the desired products 2.
  • these undesired products 3' provide a positive signal peak 48' in the new reference framework 71.
  • the level 71 By shifting the level 71 with respect to whatever the different product signals were being referred to, it is now as if the given background signal 47 with level 70 is removed from the detected signal B and replaced by a new signal with level 71 in the signal D.
  • a new signal D is obtained, as indicated in figure 10c , now containing information about the products 2, 3, 3' in such a manner that these products can be distinguished from each other without the problems present in the state of the art.
  • the good products 2 by defining an appropriate threshold value(s) for the new signal D, as demonstrated in figure 10d , while one must distinguish between 2 types of bad products 3,3'.
  • these threshold values are referred to the new reference level 71, which, in this example, has been chosen such that it is substantially equal to the signal level 46 of the good products 2.
  • a first, negative threshold value g1 has been chosen in such a manner that the undesired products 3, provide a signal peak 48 till past this first threshold value g1. These products 3 may be rejected from the product stream by activating the reject system 11.
  • a second, now positive threshold value g2 is chosen in such a manner that the undesired products 3' provide a signal peak 48' past the second threshold value g2.
  • This threshold value is indicated as positive because in the given reference framework this threshold value g2 is situated above the reference level 71.
  • These products 3' may be rejected from the product stream by activating the reject system 11.
  • the signal level 70 of the background signal 47 according to the background element 5 is shifted to the new reference level 71, in this case matching the signal from the good products 2, the variations on the signal level of the background signal 47 will be situated within both threshold values g1, g2. Therefore the reject system is not activated.
  • the signal levels 46 of the good products 2 may show small variations, provided that these variations are situated around the reference level 71 within both threshold values g1, g2.
  • the reject system is not activated in that case.
  • threshold values g1 and g2 By appropriately choosing the threshold values g1 and g2, one can identify from a combined signal as indicated in figure 10d , bad products 3,3' characterized by respectively a high signal peak 48 and a low signal peak 48'. All signals 48, 48' outside the strip defined by both threshold values g1-g2 compared to the reference level 71 are considered as originating from a bad product 3, 3'. Although in the embodiment illustrated by figures 10a-d a signal is shown with 2 types of bad products 3,3' the invention is not limited to such signals. Also in signals B whereby only one bad product type, either 3, or 3' occurs, one can apply the signal processing technique as illustrated in the different embodiments of the invention.
  • one shifts the reference level 70 of the background 47 to a new level 71 in the embodiments according to the invention using signal processing techniques one picks a background signal 47 that differs , preferably considerably, from any product signal 46, 48, 48'.
  • the background reference level 47 via signal processing, preferably to the signal level 46 of a good product 2, the accurate value of this background level 47 is not important, as long as the background element 5 provides a signal 47 that differs, preferably considerably, from the signals 46, 48, 48' of any product 2, 3, 3'.
  • This choice of background signal 47 enables to clearly distinguish between the original reference level 70 and the product signals 46, 48, 48' and, as such, refer these product signals to the new level 71.
  • the level 70 of this background signal 47 there are different ways to determine the level 70 of this background signal 47.
  • the selected signal B will substantially match the signal 47 of the background element 5.
  • the immediate signal being detected, i.e. on the given moment t(s) matches a given immediate position of the scanning beam of light and thus with a given position x(mm) in the product stream.
  • the above static tuning procedure one can also proceed in a more flexible way.
  • a sorting system can operate in which the product stream is scanned and signals B are detected. Initially, all products are either accepted or rejected. By gradually adjusting the threshold values g1 and/or g2 a selection within the product stream will be made. One can continue adjusting the threshold values until clearly only the undesired products 3, 3' are rejected.
  • Figure 7 illustrates such a dynamic specification of the different signal levels and the corresponding adjustment of the threshold value(s).
  • the reference level 70 of the detected signal B via signal processing in such a way that a new signal D is obtained that permits the products 2, 3 and/or 3' to be distinguished from each other without the problems present in the state of the art.
  • the signal 46 coming from the good products 2 and the signal 47 coming from the background element 5 will be shifted to a substantially equal signal level 71, at least for the detected optical parameter.
  • This location signal C thus contains information about both the desired products 2 and the undesired products 3, 3'.
  • Figures 11a-d illustrate a first embodiment.
  • the detected signal B one can distinguish zones corresponding to the background signal 47 from the zones corresponding to product signals 46, 48 and 48'.
  • the signal 47 originating from the background element 5 is chosen in such a way that it differs from the signals 46, 48 and 48' of the products 2, 3, 3', one can define one or more threshold values g3 and g4 in the signal B, such that the signal levels 46, 48 and 48' from the products 2, 3, 3' are located outside these threshold values, e.g. on one side of this threshold value g3 and the signal level 47 of the background element 5 is located within these threshold values, e.g. on the other side of this threshold value g3.
  • Figure 6 illustrates the embodiment wherein one has determined two threshold values g3/tmin and g4/tmax containing the background signal 47.
  • Figure 11b illustrates an embodiment wherein one uses merely one threshold value g3 to define the background signal 47 against the product signals 46, 48 and 48'.
  • the signal peaks 46, 48 and 48' and the threshold value g3 are determined with respect to the original level 70 from the background signal 47.
  • Each peak 46, 48, 48' that stretches past this threshold value g3 thus indicates a presence of a product 2, 3, 3'.
  • FIG 11c such a signal C is shown, whereby the product pulses 46, 48 and 48' are already inverted compared to the original signal B.
  • This signal C can also be a binary signal as shown in figure 11d .
  • the same detected signal B is first used to generate a product location signal C, after that, as shown above, this detected signal B is combined with the signal C to shift the reference level as illustrated in figure 10a-d .
  • Figures 12a-d give a schematic representation of different embodiments of this signal processing process.
  • the detected signal B and threshold value g3 are compared in a signal processing unit 60 to generate the product location signal.
  • This signal C is then combined in a signal processing unit 61 with the detected signal B in order to determine which zones of this detected signal B match the signal 47 originating from the background element 5.
  • the reference level 70 of these background signals is then shifted to the desired level 71, preferably the level 46 of the good products 2, which thus generates a signal D with an adjusted reference level 71 that allows distinguishing signal peaks of the good products 2 from these of the undesired products 3, 3' in the product stream. Due to the choice of the threshold values g1 and g2 one can distinguish between the signals 48, 48' of the undesired elements in the product stream and the signals of the desired elements 2.
  • Figure 12b illustrates a signal processing process in which a signal C is obtained based on a first signal B. This signal C is then being used for determining the position of the signal peaks 46, 48, 48' within the second signal B'.
  • the location signal C is obtained based on one or more first detected signals B after which this location signal C is being used to indicate the position of the products in one or more detected second signals B'.
  • Both signals B, B' are detected during the same line scan because the light returning from the product stream is converted via suitable spatial and/or frequency filters into the distinct signals B, B'.
  • the product location signal C can also originate from different signals Bi.
  • the product location signal C can also originate from different signals Bi.
  • the product stream By scanning the product stream in different ways, for instance with beams of light 45 that have different frequencies, or by analyzing the light that returns from the product stream in different ways, for instance by suitable spatial and/or frequency filters, one can obtain a more complete image of the product stream and avoid that a product remains unnoticed.
  • the signal processing unit 62 shall be able to combine these detected signals Bi in any possible manner: addition, subtraction, multiplication,... as shown in figure 12c .
  • the signal processing unit 62 will combine the different independent positioning signals Ci via an "OR" function so that no product location information is lost.
  • FIGS 13a-c illustrate another embodiment to obtain the product location signal C.
  • a signal 47 is generated during scanning by the background element 5.
  • This background signal can be a result of reflections of the incident optical signal 34, or by fluorescence of this background element 5 due to this exposure.
  • the background signal 47' was captured in this embodiment together with the signals 46, 48, 48' originating from the products 2, 3, 3'. This line-up is also illustrated in figure 2 .
  • the background element 5 will not reemit any signal, but will only detect the immediate incidence of light on this background element 5.
  • the light 45 generated by the light source 29, is being blocked by the products 2, 3, 3' that are located between the background element 5 and the light bundle 45.
  • the products when they are scanned, they will emit a light signal 46, 48, 48', for example by reflection or by fluorescence. Only the light 34 that was not blocked by these products 2, 3, 3' will strike at the posteriorly positioned background element 5.
  • a signal C is obtained, that is indicative for the presence of products 2, 3, 3' in the product stream for a given position of the scanning light bundle, or in other words, on a specific moment t(s) or yet in other words, for a specific position x (mm) according to the line scan.
  • the position of the light bundle 45 during the scan of the product stream is known, and it will therefore be no problem to correlate the time path of the signal 47', captured by the background element 5, with the time path of the signals 46, 48, 48', originating from the products 2, 3, 3' and thus with the position x (mm) of a product 2, 3, 3' in the product stream.
  • Figure 13c illustrates the obtained signal C', this time only showing peaks where there is no product 2, 3, 3' present in the scanned product stream.
  • these peaks are labeled by background reference 5, namely the incidence of the light bundle 34 on the background element 5.
  • This analogue signal C' can also be transformed into a digital signal, thus a pulse train C of "1" and "0" signals is obtained, wherein "1” indicates the absence of a product 2, 3, 3' and "0" indicates the presence of a product 2, 3, 3'.
  • this digital signal C' can easily be transformed, using signal processing techniques, into a pulse train C, where "1" indicates the presence of a product 2, 3, 3' and the "0" indicates the absence of a product 2, 3, 3', so that a signal as depicted in figure 11d is obtained.
  • An advantage of the embodiment illustrated by figures 13a-c en figures 3a and 3b is that the background element 5 can be placed at a larger distance d from the product stream. This prevents the background element 5 from being polluted by the product stream.
  • the only essential issue is whether or not, for every immediate position of the moving light bundle, a signal on the background element 5 is obtained. Because one uses a concentrated light bundle 45, such as a laser beam, the light will not substantially diverge once it has passed the product stream, even if the background element 5 is not being placed in direct proximity of the scanned product stream. Consequently, the solid angle at which the concentrated light bundle 34, for an immediate position of the scanning light beam 45, strikes the background element 5 is small enough to enable distinguishing, with sufficient accuracy, between the different positions of the moving light bundle in the scan line.
  • Figure 14 gives a schematic presentation of this signal processing process, starting from the signal obtained as illustrated in figures 13a-c .
  • the product location signal C originating from the background element 5, is being combined 61 with the detected signal B so as to determine which zones of this detected signal B correspond to a signal 47, coming from the background element 5.
  • the level 70 of these background signals is shifted to the desired level 71, preferably the level 46 of the good products 2, thus generating a signal D with an adjusted reference level that allows distinguishing between the signal peaks of the good products 2 and those of the undesired elements 3, 3' in the product stream.
  • the signals 48,48' of the undesired elements in the product stream can be distinguished from the signals of the desired elements 2, as illustrated in figures 10c-d .
  • Figures 3a and 3b illustrate different embodiments of such a background element 5 that is capable of detecting uninterrupted light beams 34.
  • the processing unit 41 comprises, according to the different embodiments, means 60, 62 for generating a location signal C based on one or more detected and converted signals B. Furthermore, this processing unit 41 comprises means 61 for generating, based on this location signal C, a signal D based on the same or other one or more detected and converted signals B, whereby the background level (70) of these latter signals is shifted to a new level (71) that allows for a clearer and more efficient distinction between the good products (2) from the bad products (3) in this signal D.
  • a method described in the previous paragraphs through the combination of a number of the above-described features is the method for sorting a stream of products 2,3 in products to be accepted 2 and products to be rejected 3 comprising the steps of:
  • the location signal is preferably obtained by detecting the light that is falling directly on the background element and thus passes by the products 2,3, i.e. the light that is not blocked by products 2,3, as described above in relation to figure 13 .
  • an automatic detection is carried out at those locations where the products to be rejected are located, based on whether or not the signal A crosses a threshold value, by merely analyzing said crossing in those zones where the products 2,3 are located according to the location signal C.
  • An apparatus for sorting products according to the method of the latter embodiment comprises :
  • the method of obtaining the location signal illustrated in figures 13a to 13d may be combined with a method of sorting based on the analysis of a camera image of the products that are present in the scanning zone.
  • the method comprises the steps of :
  • the image may be taken by any suitable camera system, such as for example a system described in WO2008/116924A2 , incorporated herein by reference.

Landscapes

  • Sorting Of Articles (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
EP20130170983 2012-06-07 2013-06-07 Appareil et procédé d'inspection et de tri d'un flux de produits Not-in-force EP2671651B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PL13170983T PL2671651T3 (pl) 2012-06-07 2013-06-07 Urządzenie i sposób dla kontrolowania i sortowania strumienia produktów

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/491,342 US20130056398A1 (en) 2006-12-08 2012-06-07 Apparatus and method for inspecting and sorting a stream of products

Publications (2)

Publication Number Publication Date
EP2671651A1 true EP2671651A1 (fr) 2013-12-11
EP2671651B1 EP2671651B1 (fr) 2015-04-29

Family

ID=48740816

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20130170983 Not-in-force EP2671651B1 (fr) 2012-06-07 2013-06-07 Appareil et procédé d'inspection et de tri d'un flux de produits

Country Status (3)

Country Link
EP (1) EP2671651B1 (fr)
ES (1) ES2540236T3 (fr)
PL (1) PL2671651T3 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9006599B2 (en) 2006-12-08 2015-04-14 Visys Method and device for sorting products
EP3263233A1 (fr) 2016-06-28 2018-01-03 Buhler Sortex Ltd. Dispositifs d'illumination
WO2019011809A1 (fr) 2017-07-10 2019-01-17 Arlanxeo Deutschland Gmbh Appareil d'inspection et procédé d'inspection visuelle de particules élastiques
US10302598B2 (en) 2016-10-24 2019-05-28 General Electric Company Corrosion and crack detection for fastener nuts
CN113182203A (zh) * 2021-05-17 2021-07-30 柳州市诚明科技有限公司 蚕茧智能分选方法
EP3962668A4 (fr) * 2019-05-01 2022-10-26 Inspect Technologies Ltd. Inspection de grains automatisée

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4634881A (en) 1982-11-09 1987-01-06 Supernova Systems, Inc. Apparatus for detecting impurities in translucent bodies
US4723659A (en) 1985-06-28 1988-02-09 Supernova Systems, Inc. Apparatus for detecting impurities in translucent bodies
EP0443469A2 (fr) 1990-02-23 1991-08-28 APPARATEBAU BIERSDORF Walter Krämer Gesellschaft mit beschränkter Haftung, Rain am Lech Filtre-presse de rotor avec plaques de filtre divisées
EP1012582A1 (fr) 1997-03-28 2000-06-28 Marc Ruymen Procede et dispositif servant a detecter des irregularites dans un produit
WO2001000333A1 (fr) 1999-06-28 2001-01-04 Barco Elbicon, Naamloze Vennootschap Procede et dispositif pour trier des produits
EP1724030A2 (fr) * 2005-05-17 2006-11-22 Visys NV Système de détection pour appareil de tri, méthode pour déterminer la dérive de ce système de détection et appareil de tri avec un tel système de détection
WO2007062219A1 (fr) 2005-11-23 2007-05-31 Princeton Lightwave, Inc. Systemes et procedes de generation d'une source de lumiere distribuee comprenant des reseaux de bragg fibres
WO2008116924A2 (fr) 2007-03-27 2008-10-02 Visys Nv Procédé et système permettant de contrôler et/ou de retirer des objets inappropriés d'une chaîne de produits et appareil de triage appliquant ledit procédé
WO2008142496A2 (fr) * 2006-12-08 2008-11-27 Visys Procédé et appareil pour inspecter et trier un flux de produits

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4634881A (en) 1982-11-09 1987-01-06 Supernova Systems, Inc. Apparatus for detecting impurities in translucent bodies
US4723659A (en) 1985-06-28 1988-02-09 Supernova Systems, Inc. Apparatus for detecting impurities in translucent bodies
EP0443469A2 (fr) 1990-02-23 1991-08-28 APPARATEBAU BIERSDORF Walter Krämer Gesellschaft mit beschränkter Haftung, Rain am Lech Filtre-presse de rotor avec plaques de filtre divisées
EP1012582A1 (fr) 1997-03-28 2000-06-28 Marc Ruymen Procede et dispositif servant a detecter des irregularites dans un produit
WO2001000333A1 (fr) 1999-06-28 2001-01-04 Barco Elbicon, Naamloze Vennootschap Procede et dispositif pour trier des produits
EP1724030A2 (fr) * 2005-05-17 2006-11-22 Visys NV Système de détection pour appareil de tri, méthode pour déterminer la dérive de ce système de détection et appareil de tri avec un tel système de détection
WO2007062219A1 (fr) 2005-11-23 2007-05-31 Princeton Lightwave, Inc. Systemes et procedes de generation d'une source de lumiere distribuee comprenant des reseaux de bragg fibres
WO2008142496A2 (fr) * 2006-12-08 2008-11-27 Visys Procédé et appareil pour inspecter et trier un flux de produits
WO2008116924A2 (fr) 2007-03-27 2008-10-02 Visys Nv Procédé et système permettant de contrôler et/ou de retirer des objets inappropriés d'une chaîne de produits et appareil de triage appliquant ledit procédé

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
"SPIE"
"SPIE", pages: 9
G.E. CARVER: "A fiber grating based distributed light source", PROC. OF SPIE, vol. 6371, 2006, pages 63710H - 2

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9006599B2 (en) 2006-12-08 2015-04-14 Visys Method and device for sorting products
EP3263233A1 (fr) 2016-06-28 2018-01-03 Buhler Sortex Ltd. Dispositifs d'illumination
WO2018002185A1 (fr) 2016-06-28 2018-01-04 Buhler Sortex Ltd Appareil de tri ou de classification
US10987701B2 (en) 2016-06-28 2021-04-27 Bühler UK Limited Sorting or classifying apparatus
US10302598B2 (en) 2016-10-24 2019-05-28 General Electric Company Corrosion and crack detection for fastener nuts
WO2019011809A1 (fr) 2017-07-10 2019-01-17 Arlanxeo Deutschland Gmbh Appareil d'inspection et procédé d'inspection visuelle de particules élastiques
US11358178B2 (en) 2017-07-10 2022-06-14 Arlanxeo Deutschland Gmbh Inspection apparatus and method for visual inspecting elastic particles
EP3962668A4 (fr) * 2019-05-01 2022-10-26 Inspect Technologies Ltd. Inspection de grains automatisée
CN113182203A (zh) * 2021-05-17 2021-07-30 柳州市诚明科技有限公司 蚕茧智能分选方法
CN113182203B (zh) * 2021-05-17 2023-11-28 柳州市诚明科技有限公司 蚕茧智能分选方法

Also Published As

Publication number Publication date
ES2540236T3 (es) 2015-07-09
EP2671651B1 (fr) 2015-04-29
PL2671651T3 (pl) 2015-10-30

Similar Documents

Publication Publication Date Title
EP2468426B1 (fr) Appareil et procédé d'inspection et de tri d'un flux de produits
US20130056398A1 (en) Apparatus and method for inspecting and sorting a stream of products
EP2671651B1 (fr) Appareil et procédé d'inspection et de tri d'un flux de produits
US8285029B2 (en) Device and method for optically sorting bulk material
US9924105B2 (en) System and method for individually inspecting objects in a stream of products and a sorting apparatus comprising such system
KR101936457B1 (ko) 제조 스트림 내의 결함들을 검출하기 위한 동적 디지털 이미징 시스템의 교정
US20110317001A1 (en) Multisensor array for the optical inspection and sorting of bulk materials
JP2009115613A (ja) 異物検査装置
JP2001356097A (ja) 透明容器を光学的に検査するための方法および装置
US9156065B2 (en) Sorting and inspection apparatus and method with determination of product velocity
JP2023529580A (ja) 物質を検出するための装置
US20140226163A1 (en) Inspection apparatus with alternate side illumination
US20140152808A1 (en) Method and device for the reliable detection of material defects in transparent material
EP0517950A1 (fr) Dispositif de tri
DE10297337T5 (de) Automatischer Inspektionsapparat und Verfahren zur Erkennung von Anomalien in einem 3-dimensionalen transluzenten Objekt
US5352888A (en) Method and apparatus for detecting and utilizing frame fill information in a sorting machine having a background and a color sorting band of light
WO2013025320A1 (fr) Appareil d'imagerie
CN118742801A (zh) 用于照射物质的设备
EP0968772A2 (fr) Une machine de tri
EP2186576B1 (fr) Procédé et dispositif de tri de produits
US20230152240A1 (en) Inspection system for quality analysis of a product to be inspected
ITPD20150094A1 (it) Dispositivo perfezionato per il controllo della denocciolatura di frutti essiccati o parzialmente essiccati
EP3047918A1 (fr) Appareil de tri de produit électro-optique et procédé de triage
JP2005233724A (ja) 異物検出方法および異物検出装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20140605

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20140829

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20141120

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 724146

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150515

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013001603

Country of ref document: DE

Effective date: 20150611

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2540236

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20150709

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 724146

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150429

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150831

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150729

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

REG Reference to a national code

Ref country code: PL

Ref legal event code: T3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150829

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150730

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013001603

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150429

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150607

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

26N No opposition filed

Effective date: 20160201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20160607

Year of fee payment: 4

Ref country code: ES

Payment date: 20160617

Year of fee payment: 4

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20160623

Year of fee payment: 4

Ref country code: PL

Payment date: 20160607

Year of fee payment: 4

Ref country code: IT

Payment date: 20160630

Year of fee payment: 4

Ref country code: BE

Payment date: 20160615

Year of fee payment: 4

Ref country code: NL

Payment date: 20160626

Year of fee payment: 4

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160630

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20130607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602013001603

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20170701

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20170607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170701

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180103

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170630

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170607

REG Reference to a national code

Ref country code: BE

Ref legal event code: FP

Effective date: 20150626

Ref country code: BE

Ref legal event code: MM

Effective date: 20170630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170630

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20181017

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150429

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170608

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170607