EP3122479B1 - Conveyor system, device for sorting bulk material using such a con conveyor system, and method for transporting - Google Patents

Description

Automatic bulk material sorting enables the use of digital image acquisition and image processing to separate high-throughput solids into distinct fractions (such as good and bad fractions) using optically detectable features. For example, belt sorting systems are known which use line-shaped imaging sensors (eg line scan cameras) for image acquisition. The image acquisition by the line sensor takes place on the conveyor belt or in front of a problem-adapted background and synchronous to the tape movement. The material discharge of a fraction (ie the poor fraction or the bad objects) is usually carried out by a pneumatic blow-out unit or by a mechanical ejection device (cf., for example, H. Demmer "Optical Sorting Systems", BHM No. 12, Springer, 2003). As an alternative to a conveyor belt, material transport can also take place via free fall or in a controlled air flow.

Due to structural limitations as well as the required computing time for the image evaluation in a computer, the observation time of an object to be rejected, which is referred to below as t ₀ , do not coincide with the discharge or Ausblasus. The blow-out unit is therefore spatially separated from the line of sight of the line scan camera. For a correct discharge of a bad object, therefore, the blow-off time (hereinafter referred to as t _b or, as estimated, as t _b ) and also the position of the object to be ejected (hereinafter referred to as x _b (t _b ) and, respectively) estimated as x _b (t _b )). The bold " x " here indicates that it is usually a multi-dimensional spatial position, below for this alternative also the name x is used. In the estimates made in the prior art, however, it is assumed that the object to be bled has no proper movement relative to the belt and thus moves with the speed vector v _{band of} the conveyor belt. The discharge timing t _b and the discharge location x _b (t _b ) are then estimated by a linear prediction based on the speed vector of the conveyor belt v _band and the measured object position x (t ₀ ) at the observation time t ₀ .

However, many bulk materials prove to be uncooperative due to their geometry and weight and show an additional proper motion relative to the conveyor belt (eg peas, pepper, roundish granules). In addition, it may come due to a different air resistance (the weight or the density may play a role) of the individual objects to influence the trajectory. As a result, the constant linear movement assumption of the bulk material is violated in accordance with v _band and the object position x _b (t _b ) is incorrectly predicted at the estimated blow-out time t _b . As a result, there is no discharge of the detected bad objects or the bad fraction, and moreover, good material may be erroneously rejected. Thus, for non-cooperative bulk materials, optical sorting with the prior art linear sorting optical sorting systems is often impossible.

To circumvent this problem, are known from the prior art ( DE 10 2004 008 642 A1 and DE 69734198 T2 ) special systems for the sorting of uncooperative materials known, however, a lot of effort for operate a mechanical material calming (by mechanical means a proper movement of the bulk material is suppressed on the conveyor belt). However, these systems have a high technical implementation and maintenance (for example, the use of specially profiled conveyor belts and vibrating troughs). Further shows EP-A-0 719 598 a conveyor system or method, which takes into account the speed of the objects to be sorted by locating at two points. Based on the prior art, it is therefore the object of the present invention to provide a conveyor system for transporting a material stream comprising a large number of individual objects (and a plant for bulk material sorting based thereon) and a corresponding transport method with which a high-precision prediction of the spatial position is also possible Uncooperative bulk material objects and thus an optimized automatic bulk sorting is also possible for such objects.

This object is achieved by a conveyor system according to claim 1, by a plant for bulk sorting according to claim 13 and by a transport method according to claim 15. Advantageous embodiments can be found in each case the dependent claims.

In the following, the invention will be described first in general terms, then with reference to exemplary embodiments. The individual features of the invention shown in combination with each other in the embodiments need not be realized exactly in the combinations shown. In particular, individual features of the embodiments shown may also be omitted or, according to the structure of the dependent claims, combined with other features of the invention in other ways. Already some of the features shown may represent an improvement of the prior art.

An inventive conveyor system is described in claim 1.

For different objects, their positions can also be determined at different times. In general, however, the same points in time at which their respective location positions are determined are selected for all objects (the time points are, for example, the time of recording of camera images of an optical detection unit of the system specified). For different objects, the defined points in time, for each of which the location of the respective object is calculated (based on the location positions already associated with this object), may be different. However, the respective whereabouts can also be calculable or predictable for all detected objects for one and the same later point in time. According to the invention, a prediction is thus made possible in order to estimate the spatial position of each detected object at a point in time - as seen from the time of the last spatial position determination of this object - in the future.

The individual objects can be subjected to image processing methods known per se to the person skilled in the art (for example, a captured camera image of the objects can undergo image preprocessing, such as edge detection and segmentation is subsequently performed) in (generally digital) image recordings of the image generated during the optical detection Material stream or the objects are localized therein, identified and distinguished from each other to determine the spatial positions of a defined object at different times and thus to track the path of this object (object tracking). On the basis of the identification and the distinction of the individual objects in an optically recorded image series, a movement path for the object can be determined for each object from the spatial positions of this object determined at different times. For example, on the basis of this movement path, the future location can then be estimated or calculated (if appropriate, on the basis of a movement model determined or selected with the movement path or the individual object positions at different times for the object being viewed).

Thus, according to the invention, the individual objects in the material flow can be identified, and based on the position of an object determined several times at different times, its location can be determined highly accurately at a point in time in the near future (ie shortly after leaving the conveyor belt at the level of the blow-out unit, for example). To transport the material flow, the conveyor system according to the invention may have a conveyor unit, which may be a conveyor belt. Likewise, however, the invention in conveyor systems based on the free fall or a controlled airflow are used.

First advantageously realizable features of the invention is claimed in claim 2.

In the context of the invention, the determination of such movement paths is also referred to below as "object tracking". The determination of the movement paths is preferably computer-aided in a computer system of the conveyor system, that is microprocessor-based.

Further advantageously realizable features describes claim 3.

A motion model selected for an object can serve to model future object movements of this object. The movement models can be stored in a database in the memory of the computer system of the conveyor system. Such a motion model can comprise equations of motion whose parameters can be determined by regression methods (for example least-squares fit method, least-squares-fit) or by a Kalman filter extended by a parameter identification on the basis of the determined positional positions or the particular motion path of the respective object , It is possible to select the movement model only after the presence of all recorded and determined during the optical detection position positions of an object. Alternatively, the movement model can be selected or changed in real time during the recording of the individual images for successive determination of the individual location positions (ie, while the individual image recordings are still being performed, it is possible to switch to another movement model for the object being viewed if, for example, a fit method shows that this other motion model more accurately reflects the motion history of the object).

Further advantageously realizable features describes claim 4.

In this case, the classification need not be based on or using the spatial positions determined during the optical detection (in particular: from the successive camera shots) (even if the information can advantageously be included in the classification via the specific location positions, see also below). Thus, the classification of an object identified on the basis of its location positions at different points in time or movement path can also take place, for example, purely on the basis of geometric features (eg outline or shape) of this object, the geometric features being determined via suitable image processing methods (eg image preprocessing such as edge detection with subsequent segmentation) the images obtained in the optical detection can be determined.

In particular, the classification can take place in exactly two classes, a class of good objects and a class of bad objects (which are to be removed). The classification can thus take place on the basis of the optical recording of recorded images of the objects, in that these images are evaluated with suitable image processing methods and thus, for example, object shape, object position and / or object orientation is determined at different times.

In the following, the pose or spatial position of an object is understood as the combination of its spatial position (or the position of its center of gravity) and its orientation. This may be a two-dimensional position (for example relative to the plane of a conveyor belt of the conveyor system - the coordinate perpendicular to it is then ignored) but also a three-dimensional position, ie the spatial position and orientation of the three-dimensional object in space.

Further advantageously realizable features according to the invention can be found in claims 5 and 6.

The particular two-dimensional spatial position is preferably the position in the plane of a moving conveyor belt, but relative to the immovable elements of the conveyor system. Thus, a position determination can be made in the immobile world coordinate system in which not only the immovable elements of the conveyor system rest Also, for example, the optical detection device (camera).

Further advantageously realizable features are described in claim 7.

It is thus possible to determine not only the spatial position but also their orientation in space (and / or their shape), in total their pose, at the several different points in time for the individual objects to be detected. Also, this orientation information thus determined may be used to calculate the whereabouts at the defined time (s) after the latest of the different times.

In addition to determining these locations, it is not necessary to determine the orientation of the objects at the defined time or points) after the latest of the different points in time. According to claim 8, this is possible.

The specific orientations (in addition to the specific location positions) can also be included in the determination of the movement paths. The determination of the movement model (s) and / or the classification of the objects can also take place with the additional use of the determined orientation information.

Further advantageously realizable features are described in claim 9.

The surface sensor (s) may in particular be a camera (s). Preferably CCD cameras can be used, also the use of CMOS sensors is possible.

Further advantageously realizable features are described in claim 8.

In the individual recorded images, in each case the shape of this / these objects (s) can be determined via image processing measures (eg image preprocessing such as edge detection with subsequent segmentation and subsequent object tracking algorithm). The three-dimensional Image of an object can be identified by suitable algorithms (see eg R. Szeliski, Computer Vision: Algorithms and Applications, 1st Edition, Springer, 2010 ; or A. Blake, M. Isard "Active Contours", Springer, 1998 ) are generated from the forms of a single object identified, for example, by object tracking in the individual images.

Further advantageously realizable features of the conveyor system according to the invention can be found in claims 11 and 12. Advantageously realizable features of the plant according to the invention for bulk material sorting can be found in claim 14.

• Figur 1 einen grundlegenden Beispielaufbau einer erfindungsgemäßen Anlage zur Schüttgutsortierung unter Einsatz eines erfindungsgemäßen Fördersystems.
• Figuren 2 bis 4 die Arbeitsweise der in Figur 1 gezeigten Anlage zur Berechnung des zukünftigen Aufenthaltsortes von Objekten des Materialstroms.
• Figur 5 den prinzipiellen Aufbau einer weiteren Anlage zur Schüttgutsortierung gemäß der Erfindung.
• Figur 6 die Arbeitsweise dieser Anlage.

The invention will be described below with reference to exemplary embodiments. Showing:

• FIG. 1 a basic example structure of a plant according to the invention for bulk sorting using a conveyor system according to the invention.
• FIGS. 2 to 4 the way of working of in FIG. 1 shown installation for calculating the future location of objects of the material flow.
• FIG. 5 the basic structure of another plant for bulk sorting according to the invention.
• FIG. 6 the operation of this system.

In the FIG. 1 The plant for bulk material sorting shown comprises a conveyor system with a conveyor unit 2 designed as a conveyor belt, an area camera (CCD camera) 3 positioned above the unit 2 and at the discharge end thereof (and the discharge end) and areal illuminants 5 for illuminating the field of view of the area camera. The image acquisition by the surface sensor (camera) 3 takes place here on the conveyor belt 2 or in front of a problem-adapted background 7 and takes place synchronously to the belt movement.

The system also includes a sorting unit, of which only the Blow-out unit 4 is shown. Shown is also a computer system 6, with which all the calculations of the system or the conveyor system described below are performed.

The individual objects O1, O2, O3 ... in the material flow M are thus transported by means of the conveyor belt 2 through the detection range of the camera 3 and there recorded and evaluated in terms of their object positions by image evaluation algorithms in the computer system 6. Subsequently, the blow-out unit 4 separates into the bad fraction (poor objects SO1 and SO2) and into the good fraction (good objects GO1, GO2, GO3 ..).

According to the invention, an area sensor (area camera) 3 is thus used. The image is obtained on the bulk material or material flow M (or the individual objects O1, ... thereof) by the camera 3 on the conveyor belt 2 and / or in front of a problem-adapted background 7. The image acquisition rate is adapted to the speed of the conveyor belt 2 or synchronized by a position encoder (not shown).

According to the invention, the acquisition of an image sequence (instead of a snapshot) of the bulk material stream at different times (in quick succession) is aimed at by means of several surface scans or surface images of the material stream M through the area camera 3 as follows (cf. FIGS. 2 to 4 ).

In FIG. 2 the field of view 3 'of the camera 3 is shown on the conveyor belt 2 with the bulk material or the objects O thereof in plan view ( FIG. 1 outlines this field of view 3 'of the camera 3 in side view). The discharge takes place through the blow-out unit 4 (whose blow-out area 4 'in FIG FIG. 2 shown in supervision). As an alternative to the conveyor belt 2, the material transport could also take place in free fall or in a controlled air flow (not shown here) if the units 3 to 7 are repositioned accordingly.

In the context of the invention, the data acquisition can thus be based on one (or more) imaging surface sensors such as the area camera 3. This allows a position determination and also a measurement physical properties of the individual particles or objects O1,... of the bulk material M at the several different times, as shown in FIG FIG. 2 outlined. Imaging sensors outside the visible wavelength range and imaging hyperspectral sensors can also be used as imaging surface sensors. In addition, the position determination can also be carried out by one (or more) 3D sensor (s) which can / provide position measurements of the individual objects in space instead of in the plane of the conveyor belt 2.

As the FIGS. 2 to 4 In the present invention, predictive multi-object tracking can be used. By means of a clocking of the camera which is suitable with respect to v _band (ie the number of recorded camera images per unit time) or the image acquisition, the object positions x (t) = (x (t), y (t)) in the Cartesian coordinate system x, y , z (with the xy plane as the plane in which the conveyor belt 2 moves) at several different times t are measured. The measurement of the object positions x (t- ₅ ), x (t _-4 ), x (t _-3 ),... Takes place at the different, consecutive (usually consecutive at constant time intervals) times t _-5 , t _{- 4} , t _-3 ,..., Where t ₀ marks the last time of observation of the object having the marked movement path 1 before this object leaves the field of view 3 'of the camera 3. From these different measured spatial positions at the different time points, the individual optically captured objects result (suitable detection algorithms for image processing enable the tracking of the individual objects (cf. A. Yilmaz, O. Javed, and M. Shah, "Object tracking: A survey," ACM computing surveys (CSUR), vol. 38, no. 4, p. 13, 2006 or B. Jähne, digital image processing and image acquisition, 7th, newly edited edition 2012, Springer, 2012 ) each movement paths by juxtaposition of the individual detected and determined object positions x. This is in FIG. 2 with the movement path 1 for a single object O for its movement between the times t _-5 and t ₀ , between which this object has been detected in the detection area 3 'of the camera 3 by frame capturing. Thus, instead of a single measurement x (t ₀ ) of the location of an object O1, O2,..., The observed motion path 1 is x (t ₀ ), x (t _-1 ), x (t _-2 ), x (t _-3 ), ... of an object O1, O2, ... for the estimation or calculation of the location of this object at one (or even several) defined time (s) the latest time t ₀ at which the spatial position has been determined for this object in the series of recorded camera images. Thus, in particular for a later time t _b , to which this object is located in the blow-out area 4 'of the blow-out unit 4, the location can be estimated with high accuracy. With the exhaust position x _b (t _b ) calculated or estimated from the movement path 1, the exhaust unit 4 can purposely remove it from the material stream M at the blow-off time t _b on the basis of the highly precisely determined blow-out position of this object (if it is a bad object).

In addition, the applied predictive multi-object tracking method additionally provides an uncertainty indication of the estimated variables in the form of a variance (blow-out time) or covariance matrix (blow-out position).

Figures 3 and 4 show the inventively usable procedure for predictive multi-object tracking more accurate. In terms of time, this procedure can be subdivided into two phases, a tracking phase and a prediction phase (where the prediction phase takes place after the tracking phase in terms of time). FIG. 4 illustrates that the tracking phase is composed of filter and prediction steps and that the prediction phase is limited to prediction steps. As in FIG. 3 illustrates, the first phase (tracking phase) is assigned to the field of view 3 'of the area camera 3. While a certain object from the set of objects O1, O2,... In the material flow M passes through this area 3 ', it can be recorded in the individual times t _-5 , t _-4 , t _-3 ,.. Camera images are identified and it can be maintained in a positional positioning. In addition to determining the spatial position, it is also possible to determine the orientation of the object in the camera images so that not only the spatial position but also the orientation of the objects at several different points in time (ie the object pose) is determined in the first step of the invention in the second step according to the invention, the location at least one defined time (the blow-out time) after the acquisition of the last camera image can be calculated on the basis of the location positions for the individual objects determined at the different times.

FIG. 4 shows this approach of predictive multi-object tracking schematically. For tracking the objects in the tracking phase ( FIG. 4a ) recursive estimation methods can be used. Alternatively, however, non-recursive estimation methods can also be used for the tracking. The recursive methods (eg Kalman filter methods) are composed of a sequence of filter and prediction steps. Once an object is captured in the camera data for the first time, prediction and filtering steps follow. The current position estimate is updated by the prediction until the next image acquisition (eg by a linear movement prediction). In the subsequent filter step, the available position estimates are updated or corrected on the basis of the measured camera data (ie based on the recorded image data). For this purpose, a Kalman filter can be used. It is also possible for a plurality of prediction or filter steps to follow one another.

At the same time, parameters of equations of motion can be estimated in the tracking phase, wherein the equations of motion can describe a movement model for the movement of a single object. In this way, on the basis of the recorded, ie optically acquired, information (ie the movement path of the individual recorded location positions or, if the position is detected, of the movement and orientation change path resulting from the object poses recorded at the several different times) the future movement path of the object under consideration be estimated with high accuracy and thus also its whereabouts to the later, potential (if it is a bad object) Ausblaszeitpunkt t _b . Examples of parameters of the equations of motion that can be estimated on the basis of the image sequences are acceleration values in all spatial directions, rotational axes and directions. These parameters can be captured by tracking in the image sequences and define a motion model for each particle, which also includes, for example, rotational and lateral movements.

In the tracking phase (in which the object under consideration is located in the image capturing area of the camera 3, that is to say in the area 3 '), the prediction phase (during which the observed object has just left the imaging area of the camera 3, outside the area 3 'and in the area 3 "between this area 3' on the one hand and the blow-out area 4 'on the other hand, and thus can no longer be detected by the camera 3), the determined equations of motion can be used to for the currently considered object (ie at corresponding Computer power for each detected object in the material flow M) to predict an estimate or calculation of the subsequent spatial position (or the pose).

After the object to be tracked has left the field of view 3 'of the camera 3, the prediction phase thus follows. This second phase of the object tracking can consist of one or more prediction steps which are based on the motion models previously determined in the tracking phase (eg estimated rotational movements). The result of this prediction phase is an estimate of the whereabouts at a later point in time (such as, for example, the discharge time t _b and the whereabouts at this point in time, ie the discharge position x _b (t _b )). Tracing the objects thus takes place in two phases. The tracking phase consists of sequences of filtering and prediction steps. Filtering steps refer to the processing of camera images to improve the current position estimates, and prediction steps continue the position estimates until the next camera image, ie, next filter step. The prediction phase following the tracking phase consists only of prediction steps since no filter step can be performed due to missing camera data.

The tracking phase can be performed in different ways: Either non-recursively, whereby the current object positions or object positions are determined from each image (no movement models have to be used.) All object positions acquired over time can be collected to form trajectories for the individual objects Recursive processing is also possible, so that only the current position estimation of an object has to be provided, using the motion models (prediction steps) to predict the movement of the object between camera measurements and thus correlating different filter steps the prediction of the results of the previous filtering step as prior knowledge In the case, a weighting occurs between the predicted positions and the positions determined from the current camera image. It is also possible to work recursively with an adaptation of the movement models: Here, a simultaneous estimation of object positions or positions and model parameters takes place. By observing image sequences, for example, acceleration values can be determined as model parameters. The movement models are thus identified during the tracking phase. It can be a fixed model for all objects or individual movement models.

The reference numeral 1 'designates the extrapolation of the movement path 1 of an object determined in the tracking phase over the detection period of this object by the camera 3, ie the predicted trajectory of the object after leaving the detection range of the camera 3', ie in particular also at the time of the flyby the blow-out unit 4 (or by the detection area 4 'thereof).

The prediction phase can directly use the model information previously determined in the tracking phase and consists of pure prediction steps since camera data are no longer available and therefore no filter steps can be performed any more. The prediction phase can be subdivided further, for example into a phase in which the objects are still on the conveyor belt and a flight phase after leaving the conveyor belt. For the prediction of the movements, two different movement models can be used in both phases (for example a two-dimensional movement model on the conveyor belt and a three-dimensional movement model in the subsequent flight phase).

One way to render the camera image data for object tracking is to transform the data into a set of object locations by image preprocessing and segmentation techniques. Applicable image preprocessing methods and segmentation methods include, for example, inhomogeneous point operations for the removal of illumination inhomogeneities and area-oriented segmentation methods as described in the literature ( B. Jähne, digital image processing and image acquisition, 7th, newly edited edition 2012, Springer, 2012 ; or J. Beyerer, FP Leon, and C. Frese "Automatic Visual Inspection: Fundamentals, Methods and Practice of Image Acquisition and Image Evaluation ", 2013th ed. Springer, 2012 ) are described.

The assignment of measurements to prior estimates can be made, for example, explicitly by a nearest neighbor search or implicitly by association-free methods, adapted to the available computing capacities in the computer system 6. Corresponding methods are for example in RPS Mahler's "Statistical Multisource Multitarget Information Fusion", Boston, Mass .: Artech House, 2007 , described.

For the simultaneous estimation of object positions and model parameters, for example, Kalman filter methods or other methods for (non-linear) filtering and state estimation can be used, as described, for example, in US Pat F. Sawo, V. Klumpp, UD Hanebeck, "Simultaneous State and Parameter Estimation of Distributed Physical Systems based on Sliced Gaussian Mixture Filters", Proceedings of the 11th International Conference on Information Fusion (2008 Fusion), 2008 , are described.

1. 1. Zunächst werden diese Parameter sowohl in der Tracking- als auch in der Prädiktionsphase zur Berechnung des/der Prädiktionsschritte(s) herangezogen, um eine präzise Vorhersage von Ausblaszeitpunkt und -position zu ermöglichen (beispielsweise kann während der Trackingphase die vom Modell vorhergesagte Position eines Objekts mit der tatsächlich in dieser Phase gemessenen Objektposition verglichen werden und die Parameter des Modells können ggfs. angepasst werden).
2. 2. Darüber hinaus erweitern die Modellparameter den Merkmalsraum, auf dessen Grundlage die Klassifikation und die anschließende Ansteuerung der Ausblaseinheit erfolgen können. Insbesondere können Schüttgüter hierdurch zusätzlich zu den optisch erkennbaren Merkmalen anhand von Unterschieden im Bewegungsverhalten klassifiziert und entsprechend sortiert werden.

The determination of motion model parameters has two functions:

1. 1. First, these parameters are used in both the tracking and prediction phases to calculate the prediction step (s) to allow accurate prediction of blowout timing and position (for example, the position predicted by the model during the tracking phase) Object can be compared with the actually measured in this phase object position and the parameters of the model can be adjusted if necessary).
2. 2. In addition, the model parameters extend the feature space on the basis of which the classification and the subsequent control of the blow-out unit can take place. In particular, bulk materials can be classified in addition to the visually recognizable features based on differences in the movement behavior and sorted accordingly.

Alternative to in FIG. 1 For example, the structure shown in FIG FIG. 5 sketched construction to be used in the FIG. 1 shown is very similar, so that only the differences are described here. In FIG. 5 Instead of a single area camera 3, several individual line scan cameras arranged along the conveyor belt 2 and above it are used (line orientation perpendicular to the transport direction x and the direction of deflection z of the cameras 3a to 3c onto the plane of the conveyor belt xy, ie in the y direction). The z-direction here corresponds to the direction of the camera 3 ( FIG. 1 ) or the plurality of cameras 3a to 3c ( FIG. 5 ). As FIG. 5 Thus, a plurality of line scan cameras (or even multiple area cameras with one or more regions-of-interest, ROIs) spatially distributed along the conveyor belt 2 relative to one another, including the illuminations 5 assigned to each of the cameras, can thus also be used. The line scan cameras or the surface cameras can be mounted both above the conveyor belt 2 and above the trajectory of the bulk material in front of a problem-adapted background 7 (this applies in the example shown for the last camera 3c seen in the transport direction x of the belt 2). The resulting image acquisition is in FIG. 6 sketched, as opposed to FIG. 5 (Which shows only three line scan cameras 3 a to 3 c) here for a total of six different line scan cameras arranged one behind the other along the transport direction x (whose detection ranges are designated by 3 a 'to 3 f').

By using several line scan cameras 3a to 3c ( FIG. 5 ) or 3a to 3f ( FIG. 6 ) and multi-object tracking methods, the position of one and the same object can be determined at several times when traversing the line-scan camera fields 3a 'to 3f', whereby a movement path 1 can be obtained as described above.

Compared to the prior art, the present invention has a number of significant advantages.

By determining the movement path 1 of each object O1, O2, ... a significantly improved prediction or estimation (calculation) of the Ausblaszeitpunkt t _b and the blow-off x _b (t _b ) is possible, even if the constant linear motion assumption of the bulk material by the speed v _{band is} not met. This allows the mechanical effort for the Material settling of uncooperative bulk materials are significantly reduced.

For extremely uncooperative materials, such as spherical bulk material, it is in many cases even possible at all to carry out an optical sorting of the type described by the present invention.

Against the background that end customers, especially in the food industry, can sort a large number of different bulk material products M on one and the same sorting system, a broad product spectrum can be processed without having to change conveyor belts (for example use of conveyor belts with differently textured surfaces) or other mechanical changes an adaptation to uncooperative bulk material must be made.

In addition, the method for multi-object tracking enables improved optical characterization and feature extraction from the image data of the individual objects O of the observed bulk material flow M. Since the non-cooperative objects generally present themselves in different three-dimensional layers of the camera due to their additional proper motion, image features of different object views can become one cumulative object feature over the individual observation times. For example, the three-dimensional shape of an object can also be estimated and used as a feature for the sorting. The extrapolation of the three-dimensional shape of an object from the recorded image data can be described as described in the literature (see, for example, US Pat SJD Prince "Computer vision models, learning, and inference", New York, Cambridge University Press, 2012 ) z. B. based on the visual envelope of each object in different poses (shape-from-silhouettes method) done.

This achieves an improved discrimination of objects with orientation-dependent appearance. In some cases, this can be dispensed with another camera for a two-sided examination. The extended object features can also be used for improved motion modeling in the context of predictive tracking, for example, the three-dimensional shape for the prediction of the trajectory is taken into account.

Moreover, the identified model that characterizes the motion path 1 of a particular object may itself be used as a feature for a classification or sorting decision. The movement path 1 determined on the basis of the individual camera shots as well as the future movement path 1 'estimated on the basis of the movement path 1, are influenced by the geometric properties and the weight of the object and accordingly offer an inference possibility on the affiliation to a bulk fraction.

Another technical advantage for the bulk material sorting is provided by the evaluation of the additional uncertainty descriptions for the estimated discharge time and the discharge position. This allows an adapted control of the pneumatic blow-out unit for each object to be ejected. If the estimated sizes are subject to a great deal of uncertainty, a larger blow-out window can be selected to ensure the discharge of a bad object. Conversely, the size of the blow-off window, and thus the number of nozzles driven, can be reduced in estimates with little uncertainty. As a result, the consumption of compressed air can be reduced during the sorting process, whereby costs and energy can be saved.

The multiple position determination of objects of the bulk material flow at different times as well as the evaluation of a sequence of images instead of a snapshot image (this may also involve a multiple measurement, calculation and cumulation of object features at different times as well as a use of identified motion models as a feature for an object classification) generally achieved a significantly improved separation in the automatic sorting of any bulk materials. In addition, in comparison to the prior art for the sorting of uncooperative materials, the mechanical outlay for material calming can be considerably reduced.

Moreover, the present invention can be used for the sorting of complex shaped bulk materials, which must be checked from several different views, using only a single area camera in a fixed position.

By using an identified movement model as a distinguishing feature, bulk goods with the same appearance, but object-specific movement behavior (for example, by different masses or surface structures) can additionally be classified and automatically sorted.

Claims (15)

1. Conveying system for transporting a material flow (M) comprising a large number of individual objects (O1, O2, ...),
   characterised in that
   with the conveying system, by means of optical detection of individual objects (O1, O2, ...) in the material flow (M), for these objects (O1, O2, ...), respectively, the location position (x(t),y(t)) thereof at several different, fixed times (t_-4, t_-3, ...) can be determined and
   by means of the location positions (x(t),y(t)) determined at the different, fixed times (t_-4, t_-3, ...) for these objects (O1, O2, ...), the location (x_b(t_b),y_b(t_b)) thereof at at least one defined time (t_b) after the respective latest of the different, fixed times (t_-4, t_-3, ...) can respectively be calculated.
2. Conveying system according to the preceding claim,
   characterised in that
   movement paths (1) composed of a plurality of location positions (x(t),y(t)) of the respective object at different times (t_-4, t_-3, ...) can be determined for the individual objects (O1, O2, ...),
   preferably the movement paths (1) of different objects (O1, O2, ...) being able to be determined via recursive or non-recursive estimating methods and/or being able to be differentiated from each other.
3. Conveying system according to the preceding claim,
   characterised in that
   a movement model can be determined respectively for the objects (O1, O2, ...) by means of the respective movement paths (1) thereof, in particular can be selected from a prescribed quantity of movement models, and/or parameters for such a movement model can be determined.
4. Conveying system according to one of the preceding claims,
   characterised in that
   the individual objects (O1, O2, ...) can be classified on the basis of the optical detection.
5. Conveying system according to the preceding claim,
   characterised in that
   the classification of an object (O1, O2, ...) can be implemented by taking into account the location positions (x(t),y(t)) determined for this object at the different, fixed times (t_-4, t_-3, ...), the movement path determined for this object and/or the movement model determined for this object.
6. Conveying system according to one of the preceding claims,
   characterised in that
   two-dimensional location positions (x(t),y(t)), in particular two-dimensional location positions relative to the conveying system, can be determined for the objects (O1, O2, ...), or
   in that three-dimensional location positions in space can be determined for the objects (O1, O2, ...).
7. Conveying system according to one of the preceding claims, characterised in that
   with the conveying system, by means of optical detection of the individual objects (O1, O2, ...) in the material flow (M), for these objects (O1, O2, ...) respectively in addition to the location position (x(t),y(t)) thereof, also the orientation thereof at several different times (t_-4, t_-3, ...) can be determined and in that, by means of the location positions (x(t),y(t)) determined at the different times (t_-4, t_-3, ...) and orientations for these objects (O1, O2, ...), respectively the location (x_b(t_b),y_b(t_b)) thereof at the at least one defined time (t_b) after the respectively latest of the different times (t_-4, t_-3, ...) can be calculated.
8. Conveying system according to the preceding claim,
   characterised in that
   by means of the location positions (x(t),y(t)) determined at the different times (t_-4, t_-3, ...) and orientations for these objects (O1, O2, ...), respectively in addition to the location (x_b(t_b),y_b(t_b)) thereof also the orientation thereof at the at least one defined time (t_b) after the respectively latest of the different times (t_-4, t_-3, ...) can be calculated.
9. Conveying system according to one of the preceding claims,
   characterised in that
   the optical detection is effected by means of one or more optical detection unit(s), which comprises/comprise or is/are preferably one or more surface sensor(s) (3) and/or a plurality of line sensors at a spacing from each other,
   and/or
   in that, during the optical detection, a sequence of two-dimensional images can be recorded, from which the location positions of the objects at the different times can be determined.
10. Conveying system according to one of the preceding claims,
    characterised in that
    within the scope of the optical detection of one or more of the objects (O1, O2, ...) at several different times (t_-4, t_-3, ...), images, in particular camera images, of this/these object/s can be produced, in that respectively the shape(s) of this/these object/s in the produced images can be determined and in that respectively a three-dimensional image of this/these objects/s can be calculated from the determined shapes.
11. Conveying system according to the preceding claim,
    characterised in that
    calculation of the location(s) of the object/s at the defined time(s) is effected taking into account the calculated three-dimensional image/s.
12. Conveying system according to one of the two preceding claims with reference back to claim 4,
    characterised in that
    the classification of the object/s is effected using the calculated three-dimensional image/s.
13. Plant for bulk material sorting comprising a conveying system according to one of the claims 1 to 12,
    characterised by
    a sorting unit (4) with which the objects (O1, O2, ...) can be sorted on the basis of the calculated locations (x_b(t_b),y_b(t_b)) at the defined time(s) (t_b).
14. Plant according to the preceding claim with reference back to claim 4,
    characterised in that
    the objects can be sorted on the basis of the classification thereof, preferably the classification being effected into good objects (GO1, GO2, ...) and into bad objects (SO1, SO2) and preferably the sorting unit (4) having an ejection unit, in particular a blow-out unit, which is configured to remove bad objects from the material flow (M) using the calculated locations (x_b(t_b),y_b(t_b)) at the defined time(s) (t_b).
15. Method for transporting a material flow (M) comprising a large number of individual objects (O1, O2, ...),
    characterised in that
    in the method, by means of optical detection of individual objects (O1, O2, ...) in the material flow (M), the location position (x(t),y(t)) of these objects (O1, O2, ...) is respectively determined at several different, fixed times (t_-4, t_-3, ...) and,
    by means of the location positions (x(t),y(t)) determined at the different, fixed times (t_-4, t_-3, ...) for these objects (O1, O2, ...), the location (x_b(t_b),y_b(t_b)) thereof at at least one defined time (t_b) after the respective latest of the different, fixed times (t_-4, t_-3, ...) is respectively calculated,
    the method being implemented preferably using a conveying system or a plant according to one of the claims 1 to 14.

Images