EP3468727B1 - Sorting device and corresponding sorting method - Google Patents

Description

The present invention relates to a sorting device (hereinafter also referred to as an optical sorting system) according to the preamble of claim 1 (and a corresponding method).

In the optical sorting of bulk materials in material streams, which can be easily separated using optical features in the visible spectrum such as color, shape and / or texture, or material properties in the near infrared range, plant operators are interested in maximizing material throughput. This results in better profitability. A higher material throughput can be achieved through higher bulk densities, larger sorting widths and / or higher material speeds.

For the most accurate discharge possible and thus the best possible sorting result, it is necessary to arrange the sensors and the separation technology as close together as possible. This allows the location of the Objects in the material flow at the level of the separation mechanism are better predicted and they can be ejected with corresponding precision, for example using pneumatic valves. In addition, bycatch (i.e. the unwanted reject of objects) close to an object to be ejected can be kept as low as possible.

For many known features, e.g. geometric descriptors, the calculation time is directly dependent on the object. In the case of very high bulk densities or occupancy densities in the material flow, from the point of view of image processing, object agglomerates (object clusters) often arise, which are difficult to separate algorithmically (in particular: segmented). As a result, even when sorting homogeneous products or objects, very different computing efforts for classification can result.

The above-mentioned aspects pose great challenges for evaluation methods, since they always involve the requirement for real-time evaluations. If real-time barriers for certain objects contained in the material flow are violated, the evaluation system does not make a timely decision about those objects, i.e. a sorting decision for a corresponding object (in particular: should this object be ejected or not) cannot be made in time. This potentially (or actually) leads to incorrect sorting.

• DE 10 2009 007 481 A1 ,
• DE 10 2010 046 438 A1 ,
• DE 10 2011103 253 A1 sowie
• DE 10 2012 001868 A1 .

Sorting systems, in particular for sorting bulk goods, are known from the prior art from the following laid-open documents:

• DE 10 2009 007 481 A1 ,
• DE 10 2010 046 438 A1 ,
• DE 10 2011 103 253 A1 as
• DE 10 2012 001868 A1 .

See also "State of the Art of Sensor-Aided Sorting" by H.

Wotruba, BHM, Volume 153 (2008), Issue 6, Pages 221 - 224 . document GB 2 273 154 A discloses a sorting device with which objects of a material flow are sorted, comprising an image recording unit with which the material flow can be optically recorded and with which image data thereof can be generated, an evaluation unit with which objects in the material flow can be identified and classified, and a sorting unit with which classified objects of the material flow are sortable. The device is designed to sort grains as document objects. The device itself sets a color value threshold, which serves as an evaluation parameter, on the basis of an average color value of normal grains, which is determined by the device.

The object of the present invention is to improve the performance of optical sorting systems for sorting objects in material flows, in particular to improve the real-time capability of the systems, that is to say the To increase the probability that a sorting decision is actually made for each object in the material flow at the time of discharge. In addition, the present invention is intended to provide more efficient processing of sensor data in order to enable real-time capability (and consequently to increase the sorting performance) even with short distances between the image recording unit (sensor system) and the sorting unit (unit for discharging objects). To this end, the present invention is intended in particular to provide new approaches with regard to sensor data processing.

This object is achieved by an optical sorting system according to claim 1 and a sorting method according to claim 13. Variants that can advantageously be implemented can be found in the dependent claims.

• Verringerung der Materialzufuhr: Liegen wenige(r) Objekte zur Auswertung vor, kann mehr Zeit pro Objekt für Berechnungen genutzt werden. Allerdings würde sich dies negativ auf die Wirtschaftlichkeit der Anlage auswirken.
• Vergrößerung des Abstands zwischen der sensorischen Erfassung und der Separation: Hierdurch kann der Auswerteeinheit (bei konstanter Materialgeschwindigkeit) eine größere Latenz zur Bildauswertung gewährt werden. Dies würde jedoch den Nachteil mit sich bringen, dass die Position von Objekten bei Erreichen der Sortiereinheit (Separationsmechanismus) schlechter prognostiziert werden kann. Durch größere Ausblasfenster könnte dem entgegengewirkt werden, jedoch würde dadurch der Beifang (ungewollter Ausschuss von Objekten nahe an einem auszuschleusenden Objekt) erhöht.
• Implementierung der Bildauswertung in Hardware: Hierdurch würden zwar sehr hohe Verarbeitungsgeschwindigkeiten erreicht werden, dies hätte jedoch den Nachteil, dass die entsprechende Realisierung auf eine ganz bestimmte Sortieraufgabe zugeschnitten werden müsste. Es würde dann an Flexibilität hinsichtlich der zugrundeliegenden Sortieraufgabe mangeln (in vielen Anlagen müssen aber unterschiedliche Sortieraufgaben gelöst werden).

The present invention is initially based on the following basic considerations: In conventional systems there is the problem that there is no sorting decision for some objects when the separation mechanism (sorting unit) is reached, since the necessary calculations by the evaluation system (evaluation unit) have not yet been completed. A subset of systems known from the prior art does not deal with this state at all, which inevitably results in losses in terms of sorting performance and / or sorting quality. In order to solve these problems, the following approaches seem to offer:

• Reduction of the material supply: If there are few objects for evaluation, more time can be used for calculations per object. However, this would have a negative effect on the profitability of the system.
• Increasing the distance between the sensory detection and the separation: This allows the evaluation unit (with constant material speed) to be granted a greater latency for image evaluation. However, this would have the disadvantage that the position of objects when they reach the sorting unit (separation mechanism) cannot be predicted more difficult. Larger blow-out windows could counteract this, but this would increase bycatch (unwanted rejects of objects close to an object to be discharged).
• Implementation of the image evaluation in hardware: This would indeed achieve very high processing speeds, but this would have the disadvantage that the corresponding implementation would have to be tailored to a very specific sorting task. There would then be a lack of flexibility with regard to the underlying sorting task (in many systems, however, different sorting tasks have to be solved).

All these approaches have in common that they either support high processing speeds or increase the tolerable latency. From these basic considerations, the present invention therefore draws the conclusion that an adaptive behavior of the sorting system is necessary in order to ensure or improve the real-time capability of the system. In other words, a basic idea of the present invention consists in appropriately responding to the calculation times required, which fluctuate depending on the sensor data or the specific conditions in the material flow and / or in its objects.

The basic implementation of this basic idea describes the optical sorting system according to claim 1.

The system according to the invention can be implemented as a belt sorting system, but slide sorting systems are also conceivable. The optical detection of the material flow usually means the recording of a large number of individual images of the material flow or of excerpts thereof per unit of time. For example, while the sorting system is working, video images (rapid individual image sequences) of a neutral background, over which the material flow is transported, can be recorded and evaluated in real time by means of the evaluation unit (with regard to the objects in the material flow recorded against said background).

A color line or color area camera system (array), for example based on CCD or CMOS, can be used as the imaging sensor system (image recording unit). The individual recordings or video images of the image recording unit can be evaluated by the evaluation unit using image processing algorithms in order to obtain the six-dimensional pose (That is, the three-dimensional position and the three-dimensional orientation) or at least the three-dimensional position of each individual object of the material flow at defined times.

The setting of the evaluation parameter (s) takes place according to the invention by the or in the evaluation unit, in particular by a microcontroller or a computer unit thereof.

The sorting of “the classified” objects does not exclude the fact that non-classified objects are also sorted (namely, for safety's sake, for example, they are simply always separated out or discharged as “bad objects”). The sorting can in particular be a mechanical separation of the classified objects. In the simplest case, the objects are classified into two classes, namely "good objects" and "bad objects". The objects of the two classes can be collected in separate containers during sorting. For this purpose (which is known in principle to the person skilled in the art, cf. also the aforementioned prior art) rapid air valves can be used to blow out the bad objects.

The first advantageously realizable features can be found in claim 2.

In other words, a sorting decision can be made by the evaluation unit with said minimum probability for any object in the material flow, so that the sorting unit can react to this object in accordance with the sorting decision made. If the (minimum) probability is equal to 100%, the evaluation unit makes a decision of identifying and classifying, i.e. a sorting decision, with absolute certainty for all objects in the material flow. In this case, the sorting decision can also be wrong (especially with very high belt speeds of the conveyor belt of a belt sorting system and / or with very high occupancy densities of the objects in the material flow). It can also happen that, due to an incorrect effect of the sorting unit on the material flow, a certain, classified object is still not discharged despite a (correct) sorting decision by the evaluation unit to "discharge". It can also happen that a certain, classified object is ejected even though the (correct) sorting decision for this object is "not eject".

Further advantageously realizable features can be found in claim 3.

The accuracy GK of one or more process step (s) (e.g. the process step of segmentation or the process step of using a classifier based on a decision tree) can therefore be specified or set. The process step (s) is / are then carried out with the (each) set GK. Setting a low GK can mean, for example, that the image data or the data resulting from the image data are only processed as input data of the process step in a rough raster manner in order to keep the number of computer-aided calculations as low as possible. In this case, the process step carried out with this accuracy GK is ended in any case.

However, in the case of iterative calculations or in the case of recursive calculations, this can also mean that after a defined (e.g. low) number of repetitions of the iterative loop (for iterative calculations) or after reaching a defined (e.g. low) recursion depth (for recursive calculations), i.e. after fulfilling a defined termination criterion, the calculation of the process step is terminated.

As a result of such a specification or setting of different accuracies, varying calculation times and / or numbers of repetitions (in the case of iterative calculations) or recursion depths (in the case of recursive calculations) result.

According to the invention, algorithms that refine the accuracy step by step (in particular: algorithms that refine recursively, iteratively and / or incrementally) can be used. However, this does not have to be the case; the algorithms cannot be repetitive and / or analytical either.

Further features that can advantageously be implemented can be found in claim 4.

The calculation time BZ of one or more process step (s) can therefore be specified or set at least approximately. The process step (s) can then be carried out with the (respectively) set BZ or until the BZ has expired (termination criterion).

As a result of specifying or setting different calculation times, varying accuracies and / or numbers of repetitions (in the case of iterative calculations) or recursion depths (in the case of recursive calculations) result.

Further advantageously realizable features can be found in claim 5.

Frequencies with which the computation sequences of process steps are repeated can be specified or set (in the case of recursive algorithms, the frequency corresponds to the recursion depth). Setting a lower frequency generally leads to a lower accuracy with which a process step is carried out, or to a lower calculation time for the process step. After the calculation sequences have been repeated frequently (or the selected recursion depth has been reached), the process step is ended. The termination criterion here is therefore a frequency. As a result of such a specification or setting of different repetition frequencies (hereinafter also abbreviated as WH), varying accuracies GK and / or calculation times BZ result for the corresponding process steps.

According to claims 3 to 5, process steps that refine accuracy can be implemented or realized step by step. Any two or all three of the named evaluation parameter types (GK, BZ and / or WH) can be used together by the evaluation unit to identify and classify the objects of the material flow. For example, the GK can be set in one process step (for example the segmentation in the image data), while the BZ is set in another process step (for example using a classifier based on a decision tree).

The algorithms used are advantageously implemented in such a way that the conditions according to claim 2 are met for all evaluation parameters or types of evaluation parameters used for each process step at any point in time during the implementation of such a process step (or at the time when the calculations of such a process step are aborted) are.

In other words, all calculations for all objects in the material flow are preferably carried out and completed (if necessary, for example by premature termination of the calculations with a still very low level of accuracy) that the objects each need after they have been captured by the image recording unit (for example on the Conveyor belt) to the last possible place where they can still be sorted out by the sorting unit. According to the invention, it can therefore be ensured that for each object or at least for 90%, 95% or 99% of the objects there is always a momentary (possibly still very rough) classification decision (which can then be followed by a corresponding sorting decision).

Further features that can advantageously be implemented can be found in claim 6.

The identification and classification or the evaluation can therefore be carried out in several process steps (hereinafter also alternatively referred to as components) of the evaluation unit. For each process step (each component) a type, preferably exactly one type, of evaluation parameters according to claims 3 to 5 can be defined.

Further features that can advantageously be implemented can be found in claim 7.

This means that the resulting or the selected calculation time (s), cf. also the preceding dependent claims are such that even with changes in the material flow or in the objects of the same, the occupancy parameter (s) is / are determined so quickly and proceeding therefrom the evaluation parameter (s) can be adapted so quickly that the conditions of claim 2 can also be met in the changed material flow or for the changed objects. This can be achieved by means of suitable hardware and / or software and / or also by means of suitable, sufficiently low set accuracies.

Further features that can advantageously be implemented can be found in claim 8.

According to the first variant of this claim, evaluation parameters can thus be set which are used directly in the evaluation unit in order to control the identification and classification (or the process steps of the same) directly, ie immediately, by the evaluation unit.

According to the second variant of this claim, evaluation parameters can be set which control the image recording unit 1 and which thus indirectly or indirectly, i.e. as a result of this control, influence the identification and classification carried out by the evaluation unit (or the process steps of the same). An example of such an evaluation parameter is the image resolution (or the number of pixels per unit area) in the image recording unit: This can be reduced (e.g. by combining several pixels). The image data then have a lower image resolution, the occupancy parameters thus become less precise / coarser and / or are available in fewer numbers (and can thus be determined more quickly). The latter then also indirectly simplifies or accelerates the identification and classification by or in the evaluation unit. The latter can also be achieved by changing one or more evaluation parameters of the evaluation unit with regard to the setting on the basis of the evaluation parameter (s) of the image recording unit.

Further features that can advantageously be implemented can be found in claim 9.

Such a sorting parameter can preferably be set by the sorting unit. One possible sorting parameter is the interconnection of several neighboring blow-off nozzles of the sorting unit to form a nozzle cluster. For example, with such a nozzle cluster, large objects can be sorted out more reliably or (with a low occupancy density) larger blow-out areas can be created around objects to be sorted out. The probability of the actual sorting out objects to be sorted out by the sorting unit can thus be further increased.

The occupancy density can in particular be defined as the mean number of objects per unit area of the material flow (for example as the mean number of objects with which a unit area of the conveyor belt of a sorting system of the belt type is occupied). In the case of sorting systems of the slide type or of the fall type, the occupancy density can mean the average number of objects per fall section or per fall area unit.

The occupancy distribution can record or describe whether the objects in the material flow are all separated or the probability of any single object being present on, for example, the conveyor belt (or whether objects still overlap or the probability of clustering).

Further features that can advantageously be implemented can be found in claim 10.

If several of these process steps are carried out, they are preferably carried out in a chronological order according to their order in the list of this claim (example: segmentation followed by context analysis followed by classification).

The context analysis as part of the identification process can Connected Component Analysis ", as it is, for example, in" Topological Algorithms for Digital Image Processing "by TY Kong, A. Rosenfeld, North Holland, Amsterdam, NL, 1996 is described.

The segmentation as part of the identification can be carried out, for example, as described in " Digital image processing and image acquisition "by B. Jähne, Springer, Heidelberg, Germany, 2012 is described.

The identification and classification, in particular the identification, can include further process steps such as an image preprocessing step (before segmenting and before the correlation analysis) and / or a feature calculation step (after segmenting and after the correlation analysis). See for example " Digital image processing and image acquisition "by B. Jähne, Springer, Heidelberg, Germany, 2012 and " Automatic visual inspection: Basics, methods and practice of image acquisition and evaluation ", J. Beyerer, FP León, C. Frese, Springer-Verlag Berlin Heidelberg, Germany, 2012 .

After several identification process steps, a classification process step and then a sorting decision process step can take place (the classification process step can also already contain the sorting decision).

Further advantageously realizable features can be found in claims 11 and 12.

According to claim 12, the implementation of individual process steps can take place both in software and in hardware, but also only in software or only in hardware. A central processing unit (server system) is preferably used, with which all units of the sorting system are connected via bidirectional data lines. This central processing unit can carry out all necessary data processing measures, calculations and / or process steps with the aid of a computer.

A sorting method according to the invention can be found in claim 13.

The present invention thus describes a procedure in which, in principle, the best possible sorting decision can be made within the available computing time, in order to enable real-time limits to be adhered to in principle. The Associated Upgrading the evaluation system (evaluation unit) can thus directly support qualitative advances in optical bulk material sorting, since time limits can be defined more closely. According to the invention, this is implemented through the use of algorithms which incrementally refine a sorting decision or which, alternatively, align their calculations with an allocated time budget.

A number of implementations, which differ in accuracy GK and calculation effort BZ, can be used for partial tasks of the evaluation system. During the individual process steps (especially during the evaluation), a runtime system can select specific implementations in order to adhere to real-time limits and to achieve the best possible result during the time available. The available time budget can be propagated to evaluation algorithms. The algorithms can be expanded to include intelligence so that they make the best possible use of the time available.

According to the invention, interruptible sub-processes (the process steps) can be implemented. The evaluation algorithm can be interrupted at any time by a control system and the best process step result up to this point in time can be queried. By using the best possible decision up to this point in time, information about an object to be sorted can always be evaluated for a sorting decision. So it does not come to the case that if the time limit is exceeded, an object for a classification or sorting is ignored at all. This ultimately results in an increase in the sorting performance and the sorting quality.

According to the invention, it can be ensured that a sorting decision is made for each object contained in the material flow when the separation mechanism (sorting unit) is reached. This decision is generally based on information that has been recorded about the object by the sensor (image recording unit), the evaluation quality differing depending on the time available. This means that a better decision can be made than is the case with conventional prior art systems. In addition, in the present invention a lower latency between the sensory detection and the separation can be made possible, whereby the spatial separation is minimized, an increase in the sorting quality is achieved, and a more compact sorting system is realized.

The invention makes it possible to achieve improved separation in the automated sorting of any bulk goods or material flows. In addition, smaller outlet windows can be used compared to the prior art. The invention can be used for sorting complex bulk goods which are classified on the basis of many complex characteristics.

Optical sorting systems as in the present invention can be used if the materials or the objects can be distinguished on the basis of optical features, preferably in the visible or also in the near infrared range, and can thus be classified for sorting. The invention can be used for the raw material processing industry, in which, in addition to cost-effective production, consistently high quality must always be ensured. This includes, for example, the sorting of industrial minerals, for example to reduce the iron content in a raw material. The invention can also be used in the food or luxury goods sector, where impurities in products (for example: dried peppers, dried grapes ...) have to be removed. Another important area of application is the recycling of products (e.g. sorting of old glass).

The present invention can make a significant contribution to better economic efficiency in optical sorting systems.

As explained in more detail in the following exemplary embodiment, according to the invention, with the aid of the acquired sensor data (image data), a material occupancy (eg occupancy density) and a distribution of the objects in the material flow can be determined by a monitoring unit. This can be achieved through learned models as well as through certain metrics. Based on this knowledge, individual links (components or process steps) of the processing chain can, for example, be made possible by a control component regarding the accuracy. A shorter calculation time for these elements can thus be achieved. For example, in the case of a high occupancy density and / or an unfavorable distribution in the occupancy (for example, if there are object clusters), the accuracy of at least one element or a process step can be set to "coarser" or "less", so that the real-time conditions are always met (even if this reduces the accuracy of the classification or sorting decisions or if this increases the error rate, ie the probability of an incorrect sorting decision or classification decision for an object being viewed). According to the invention, this greatly increases the probability of compliance with all real-time conditions in the sorting system. This results in better sorting decisions, since a classification can be carried out at all for more objects in the material flow by increasing compliance. (In the extreme case, a classification decision and thus also a sorting decision is made in every case, ie for each individual object in the material flow.)

According to the invention, the control component (evaluation unit) can set the evaluation parameters, in particular the accuracy, the calculation time and / or the repetition frequency, on the basis of relationships determined with known material flows (with known object occupancy, with known object types, object sizes, object weights, etc.). These relationships can reflect dependencies between the material allocation or the allocation parameters on the one hand and the settings to be made for the evaluation parameters on the other hand. The sorting quality and sorting performance result according to the set evaluation parameters.

Figures 1 to 5 show an exemplary embodiment for an optical sorting system according to the invention (and a corresponding sorting method) as follows.

• Figur 1 einen optischen Bandsortierer gemäß der Erfindung.
• Figur 2 den Verarbeitungsablauf im Sortiersystem gemäß Figur 1 aus Sicht der digitalen Daten (insbesondere: der Bilddaten) und des Materialstroms bzw. des Weges von dessen Objekten.
• Figur 3 den Ablauf des Sortierprozesses gemäß der Figuren 1 und 2, insbesondere die Prozessschritte der Auswertung (Identifizieren und Klassifizieren) in der Auswerteeinheit des Systems.
• Figur 4 ein Beispiel für einen gemäß der Figuren 1 bis 3 bestimmten Belegungsparameter (hier: Materialbelegungsdichte) sowie einen unter Verwendung desselben eingestellten Auswertungsparameter (hier: Genauigkeit GK), der das Identifizieren und Klassifizieren der Objekte, also die Auswertung, durch die Auswerteeinheit steuert.
• Figur 5 ein Beispiel für den Prozessschritt der Klassifikation in der Auswertung gemäß Figur 3.

It shows:

• Figure 1 an optical belt sorter according to the invention.
• Figure 2 the processing sequence in the sorting system according to Figure 1 from the point of view of the digital data (in particular: the image data) and the material flow or the path of its objects.
• Figure 3 the sequence of the sorting process according to the Figures 1 and 2 , in particular the process steps of the evaluation (identification and classification) in the evaluation unit of the system.
• Figure 4 an example of one according to the Figures 1 to 3 certain occupancy parameters (here: material occupancy density) and an evaluation parameter set using the same (here: accuracy GK), which controls the identification and classification of the objects, i.e. the evaluation, by the evaluation unit.
• Figure 5 an example of the process step of the classification in the evaluation according to Figure 3 .

Figure 1 shows an optical band sorter which basically follows the structure according to the prior art, the special features according to the invention being in particular in the evaluation unit 2 or the evaluation of the image data 4 by the same. In the belt sorter, a bulk material flow or material flow M is transported by means of a conveyor belt 11 in a manner known per se to the person skilled in the art past an image recording unit 1 to a sorting unit 3 arranged at a defined distance from the image recording unit 1. The bulk material flow M comprises a large number of individual objects O, which are to be classified or sorted into only two classes, namely into good objects (to be sorted into the collecting container 13b of the sorting unit 3) and bad objects (to be sorted into the further collecting container 13a the sorting unit 3). For this purpose, the objects O of the material flow M must first be recorded by the image recording unit, then classified into good objects and bad objects by evaluating the image data 4 recorded by this unit 1 in the evaluation unit 2 and finally sorted. Sorting into or dividing into the two containers 13a, 13b in accordance with the sorting or classification result 13 is carried out by the compressed air valves of the sorting unit 3, which uses the evaluation results 10 of the evaluation unit 2 to identify bad objects Remove from the material flow M by blowing out.

The bad objects blown out fall into the container 13a for the bad objects, the good objects remain in the material flow M, that is, are not blown out, and fall into the container 13b for good objects. This describes the ideal state of sorting result 13.

The image recording unit 1 comprises an imaging sensor system, here a CCD color line camera 1a, which detects the material flow M or the objects O of the same in the discharge area of the conveyor belt 11 against a background 12 or records a rapid sequence of images of the material flow M against the background 12 . For this purpose, an illumination 1b of the unit 1 illuminates the material flow M against the background 12 of the unit 1 in order to ensure optimal image recording conditions for the camera 1a.

The recorded image data or video data 4 are transmitted to the latter via a data line connection between camera 1a and evaluation unit 2. The evaluation unit 2 then carries out the process steps of identifying and classifying the objects O of the material flow M in the image data 4, which are described in detail below, and transmits the evaluation results 10 of the process steps 7a-7g carried out (cf. Figure 3 ) via a data connection to the sorting unit 3.The latter finally discharges bad objects at a distance from the discharge area of the conveyor belt 11 or from the pick-up unit 1, whereby the separation into good objects (container 13b) and bad objects (container 13a) results in 13 in the sorting result.

Figure 2 shows the material flow (solid arrows) as well as the data flow (dashed arrows), in particular the data flow during image data acquisition and evaluation, in the system according to FIG Figure 1 . The material feed onto the conveyor belt 11 is first followed by a separation and stabilization of the objects O on the conveyor belt before the in Figure 1 The transport state shown on the left in the picture or the state of the objects O in the material flow M on the conveyor belt 11 is present. Finally, in the dropping area, the image recording or sensor recording of the objects O in the material flow M takes place by means of the image recording unit 1 or the camera 1a of the same. The image data 4 are sent to Evaluation in the evaluation unit 2, which carries out the classification or makes the sorting decision for the individual objects O at the end of the evaluation. During the evaluation with the evaluation unit 2 (i.e. the implementation of all necessary process steps 7, cf. Figure 3 , or calculation processes), the material flow M falls downstream of the discharge area of the conveyor belt 11, viewed along the discharge parabola. The time span of the free fall of the objects O along the ejection parabola (until reaching the discharge area of the unit 3) is defined by the distance between the end of the conveyor belt on the one hand and the area of action of the compressed air valves of the sorting unit 3 on the other hand and corresponds to the latency or the time required for a meeting a sorting decision for an object O is available.

According to the invention, for each object in the material flow M, cf. also Figure 4 , a classification and sorting decision is made during this latency period. This takes place in the evaluation unit 2. The sorting decision (in a good object or a bad object) is finally implemented by the sorting unit 3 (separation) after the latency period has ended for each object.

Figure 3 shows, in simplified form, the sequence in the sorting system according to Figures 1 and 2 : After the material feed 30 onto and the material transport 31 on the conveyor belt 11, the image acquisition 32 takes place by means of the camera 1a of the image recording unit 1. The evaluation 33 then begins by the evaluation unit 2, which is followed by the separation 34 by the sorting unit 3.

According to Figure 3 comprises the evaluation, i.e. the identification and classification of the objects O by the evaluation unit 2, here a total of seven individual process steps 7.The input data of the evaluation are the image data 4. On the basis of the determined occupancy parameter 5 (cf. Figure 4 : Material occupancy density), exactly one evaluation parameter 6 controlling this process step is set in the evaluation unit for each individual process step 7. The evaluation of the occupancy density required for this to calculate the evaluation parameters 6a-6g of the individual process steps 7a-7g of identification and classification by evaluation unit 2 takes place in the computer system of evaluation unit 2 (not shown).

In the example shown, the Figure 3 an image data preprocessing accuracy level GK 6a is set by the evaluation unit 2 for the first process step 7a of the image data preprocessing of the image data 4 as the evaluation parameter controlling this preprocessing. The accuracy level can be, for example, "low", "medium" or "high" (see also Figure 4 ).

In the process step 7b of image cleaning that follows step 7a in terms of time, the evaluation unit 2 sets a fixed calculation time BZ as evaluation parameter 6b. Step 7b of the evaluation is thus controlled using a BZ. The subsequent third process step 7c of the segmentation is in turn carried out or controlled with an accuracy level GK set by means of the unit 2 as evaluation parameter 6c. For the correlation analysis 7d, one of the three accuracy level values “low”, “medium” or “high” is also assigned or set as evaluation parameter 6d.

The fifth process step of the feature calculation 7e in terms of time is carried out or controlled on the basis of a set repetition frequency WH as evaluation parameter 6e. The feature calculation here comprises a calculation sequence to be carried out recursively, the repetition frequency WH 6e e.g. can assume a value between 3 and 7 (ie the recursion depth of the calculations can be selected between 3 and 7, whereby 3 requires little computational effort and thus enables step 7e to be carried out quickly, and a value of 7 requires a high calculation effort, so that Step 7e takes a long time to be carried out, but can be done with high accuracy - the latter is therefore only sensible or possible if there is a low occupancy density 5 if sorting decisions have to be made for all objects O).

Finally, the sixth step 7f is the classification, with an accuracy GK again being set as the evaluation parameter 6f for this process step 7f. The final sorting decision 7g is also made on the basis of setting an accuracy value GK 6g.

The result of the total of seven process steps 7a - 7g with the respective With the evaluation parameters 6a-6g set, the sorting result 13 is off for each individual object O of the material flow M Figure 1 .

Figure 4 shows how the Figures 1 to 3 Evaluation parameters 6 can be set on the basis of occupancy parameters 5 determined beforehand in the image data 4. This is shown on the basis of an individual occupancy parameter 5, here the material occupancy density, which is determined with the evaluation unit 2 from the image data recorded with the image recording unit 1 or the camera 1a of the same (individual images 4 are shown here).

From the here exactly one occupancy parameter material occupancy density 5, which characterizes the material flow M with regard to the number of objects O per unit area, by means of the evaluation unit 2 exactly one evaluation parameter 6, here an accuracy (e.g. - here simplified - exactly one calculation accuracy controlling all individual process steps 7 that assumes one of the values "high", "medium" or "low") is determined and set. This accuracy 6 is an evaluation parameter of the evaluation unit 2, which is set in the evaluation unit 2 in order to control the identification and classification of the objects O by the evaluation unit 2. How Figure 4 shows, in the case of a low material occupancy density 5 for the identification and classification or for all individual process steps thereof, a high accuracy 6 is set as the evaluation parameter, so that the evaluation unit 2 carries out all the identification and classification process steps 7 with the accuracy "high", without the real-time conditions being violated during the evaluation (cf. Figure 4 , middle picture line).

If the material occupancy density 5 in the material flow M increases to "high", so that the real-time conditions would no longer be met if the high accuracy of the evaluation process steps were maintained, the evaluation unit 2 sets the accuracy 6 to "low" for all process steps 7 . From this point on, the identification and classification is carried out with low accuracy or controlled in such a way that the real-time conditions can be maintained again (cf. Figure 4 , lower picture line).

Figure 4 also shows that evaluation parameters 8 of the image recording unit 1 can also be set on the basis of the material covering density 5. For example, the only evaluation parameter 8 of the image recording unit 1 that can be set here is its image resolution. In the case of a low material occupancy density 5, the camera 1a can record the image data 4 with high image resolution 8, whereas in the case of a significant increase in the material occupancy density 5, as in the lower line of FIG Figure 4 shown, the image resolution 8 can be significantly reduced when recording the image data 4. The so adjustable evaluation parameter 8 of the image recording unit 1 (image resolution) can then indirectly also influence the determination of the occupancy parameter (s) 5 and thus also the setting of the evaluation parameter (s) 6 of the evaluation unit 2.

Figure 5 finally shows an example of a possible implementation of one of the process steps or subtasks Figure 3 , namely the classification 7f. The example shows an implementation of an interruptible classifier for making the best possible classification decision (or the sorting decision that follows this). A decision tree is used as the basis for the classifier. In each inner node of the decision tree, a feature of a found object is compared against a threshold value and the path to the right or to the left is followed according to the comparison result, as in FIG Figure 5 shown. The resulting classes are stored in the leaves of the decision tree. According to the Figures 1 to 4 there are only two classes here, namely the class of good objects and the class of bad objects.

According to the invention, various methods known per se to the person skilled in the art can be used for automatic learning of a corresponding decision tree on the basis of a given training set. See for example " Machine Learning "by TM Mitchell, McGraw-Hill, Boston, USA, 1997 .

According to the invention, these methods can be expanded in such a way that not only a class membership is stored in the leaves of the decision tree, but also a probability for the various class memberships in each inner decision tree node (based on the training amount) is stored. For example, if you distinguish between two classes, namely a class A for good objects and class B for bad objects, with the same number of examples in the training set for both classes, then the probabilities for class membership for A and B are in the top node f ₈ at 50% each. Here f describes the feature vector and the eighth contained feature is correspondingly described by f ₈ . It should be noted that a single feature in several nodes of the decision tree can be used as a test criterion.

An object O to be classified falls into the left or right subtree after the comparison with f ₈ in the first, topmost node. In the next node, for example on the right, a comparison is now made with f ₄ . The probabilities can be redistributed. For example, it is conceivable that in the latter node the probabilities for A are stored with 30% and for B with 70%. If the process step of classification is then interrupted at this point (or if the calculations have reached this point), the statement can be made with certainty that, according to the current state of knowledge, belonging to class B is more likely than to class A.

The process step of the correlation analysis 7d (cf. Figure 3 ) can be implemented according to the invention, for example, by a “connected component analysis”. See z. BMB Dillencourt et al. "A General Approach to Connected Component Labeling for Arbitrary Image Representations", Journal of the ACM 39 (2), 1992 .

Several interruption points can be granted here by undersampling the image with a shifted grid, for example. For example, only every fourth pixel can be viewed in a first pass: the same value can then be assumed for all pixels not viewed as for the pixel viewed last. If there is enough calculation time available, the grid can be shifted: In this way, the information about the image is refined with each cycle or each repetition. It is an iterative process that can deliver valid information after each run, but this information successively improved.

Interruptible sub-processes or process steps can also be implemented as follows, for example. The area of an object is determined in pixels. For this purpose, the pixels are simply counted in an algorithm. Such a process can be interrupted at any time and the pixels counted so far can be assumed as an area. Alternatively, the area can be estimated from other data, if already available. For example, if an axis-aligned "bounding box" is known, the area of this box can be used as an estimate instead of the real area (this can offer advantages through fewer memory accesses).

In general, within the scope of the invention, iterative methods or iteratively performed process steps can be interrupted with each loop pass. In the event of an interruption during a loop pass, the result of the last complete loop pass can be used.

Claims (13)

1. A sorting device by means of which objects (O) of a material flow (M) are sorted, comprising an image capturing unit (1) with which the material flow (M) is optically detectable and with which image data (4) of the material flow (M) can be generated,
   an evaluation unit (2) with which objects (O) in the material flow (M) can be identified and classified, and
   a sorting unit (3) with which classified objects (O) of the material flow (M) are sortable, wherein
   one or a plurality of occupancy parameters (5) which characterizes/characterize the material flow (M) with regard to its occupancy with the objects (O) is/are determinable by the evaluation unit (2) from the generated image data (4), and in that one or a plurality of evaluation parameters (6, 8) controlling the identification and classification of the objects (O) by the evaluation unit (2) is/are adjustable on the basis of the determined occupancy parameter(s) (5), wherein the adjustment of the evaluation parameter(s) takes place by the evaluation unit, and wherein the occupancy parameter(s) (5) is/are an occupancy density and/or an occupancy distribution of the objects (O) in the material flow (M).
2. A sorting device according to the preceding claim,
   characterised in that
   the evaluation parameter(s) (6, 8) is/are adjusted in such a manner that for each object (O) in the material flow (M), both an identification and a classification takes place by the evaluation unit (2) with a probability of ≥ 90%, preferably of ≥ 95%, preferably of ≥ 99%, particularly preferably of 100%.
3. A sorting device according to any one of the preceding claims,
   characterised in that one or a plurality of accuracy/accuracies GK, with which (a) process step(s) of identification and classification is/are carried out by the evaluation unit (2), is/are adjusted as evaluation parameters (6).
4. A sorting device according to any one of the preceding claims,
   characterised in that one or a plurality of calculation time/times BK, with which (a) process step(s) of identification and classification is/are carried out by the evaluation unit (2), is/are adjusted as evaluation parameters (6).
5. A sorting device according to any one of the preceding claims,
   characterised in that
   one or a plurality of repetition frequency/frequencies WH, with which (an) iterative, recursive or incremental calculation sequence(s) of (a) process step(s) of identification and classification to be carried out by the evaluation unit (2) is/are to be repeated, is/are adjusted as evaluation parameters (6).
6. A sorting device according to any one of the preceding claims,
   characterised in that
   the identification and classification of the objects (O) in the material flow (M) by the evaluation unit (2) is carried out in a plurality of process steps (7a, 7b,...) to be carried out in a chronologically successive or chronologically parallel manner, wherein for each of the process steps (7a, 7b,...) in each case one or a plurality of evaluation parameters (6a, 6b,...) is/are adjusted.
7. A sorting device according to any one of the preceding claims,
   characterised in that
   the determination of the occupancy parameter(s) (5) and/or the adjustment of the evaluation parameter(s) (6, 8) is/are carried out in real time.
8. A sorting device according to any one of the preceding claims,
   characterised in that on the basis of the determined occupancy parameter(s) (5), one or a plurality of evaluation parameters (6) of the evaluation unit (2) is/are adjusted, that is (an) evaluation parameter(s) with which one or a plurality of process steps(s) (7a, 7b,...) of identification and classification by the evaluation unit (2) to be carried out by the latter is/are controlled,
   and/or
   in that on the basis of the determined occupancy parameter(s) (5), one or a plurality of evaluation parameters (8) of the image capturing unit (1) is/are adjusted, therefore (an) evaluation parameter(s) with which one or a plurality of procedures in the image capturing unit (1) is/are controlled, which procedure(s) influence/influences the carrying out of one or a plurality of the process step(s) (7a, 7b,...) of identification and classification by the evaluation unit (2).
9. A sorting device according to any one of the preceding claims,
   characterised in that
   additionally (a) sorting parameter(s) controlling the sorting of classified objects (O) by the sorting unit (3) is/are adjusted on the basis of a/a plurality of determined occupancy parameter(s).
10. A sorting device according to any one of the preceding claims,
    characterised in that
    the identification and classification of the objects (O) by the evaluation unit (2) comprises as process step(s) one, a plurality or all of the following steps:

    • a segmentation step,

    • an interrelationship analysis step,
    and/or

    • a classification step by means of one or a plurality of training-set-based decision tree(s).
11. A sorting device according to the preceding claim,
    characterised in that
    upon classification, in each node of the decision tree(s) there is/are stored a probability/probabilities on the basis of a training set/training sets.
12. A sorting device according to any one of the preceding claims,
    characterised by
    one or a plurality of preferably computer-supported or microcontroller-supported arithmetic unit(s) by means of which the detection, generation, identification, classification, sorting, determination, control and/or adjustment is/are carried out.
13. An optical sorting procedure for the sorting of objects (O) of a material flow (M),
    characterised in that
    a sorting device according to any one of the preceding claims is used to carry out the procedure.

Images