DK2565851T3

DK2565851T3 - An apparatus for receiving returnable packaging and method for classification of returnable packaging by means of the light panels

Info

Publication number: DK2565851T3
Application number: DK12182005.4T
Authority: DK
Inventors: Roy Gergs; Ralph Maschotta
Original assignee: Wincor Nixdorf Int Gmbh
Priority date: 2011-09-01
Filing date: 2012-08-28
Publication date: 2015-09-21
Also published as: EP2565851A3; EP2565851A2; ES2546990T3; EP2565851B1; DE102011053179A1

Description

Device for taking back empties and method for classifying empties with the help of light fields

Description

The invention relates to an apparatus for taking back empties in which the empties are placed in a capturing region of a placement compartment. The empties that have been placed in are classified with the aid of optical methods. The invention furthermore relates to a method for classifying empties in which the empties placed in the placement compartment are classified with the aid of optical methods.

The apparatus is in particular a reverse vending machine used in retail stores so that customers can return their empties and receive back the deposit paid for the empties. The empties in particular are containers consisting of a beverage crate with a plurality of compartments and potentially of bottles held in the compartments. The bottles held in the compartments are in particular glass bottles, i.e. reusable bottles. A stereo camera is provided in known reverse vending machines, comprising two cameras, each of which is used to capture one image with a representation of the empties that have been entered. A three-dimensional measurement of the empties is taken with the aid of a control unit, wherein, in particular, the height, width and length of the beverage crate, the position and/or size of the compartments of the beverage crate, and the dimensions of the bottles arranged in the compartments are ascertained. The empties are classified in dependence on the ascertained dimensions. It is ascertained here in particular, whether the empties are empties that are subject to deposit, and how large the deposit for the empties concerned is. The customer is accordingly credited with this deposit, and it is issued to him on a credit note with the value of the credit after all the empties have been entered. A problem with the use of stereo cameras for determining the dimensions and thus for classifying the empties is that often only a very imprecise ascertainment of the dimensions of the crates and bottles is possible with the aid of the stereo camera. In particular, edges that are located outside the focusing plane of the stereo camera can only be represented in a relatively blurred form in the images captured by the stereo camera, so that it is difficult to determine the dimensions of these blurred body edges. It is additionally problematic that only two images of the empties are ascertained, so that the measurement of edges that run parallel to the epipolar lines is problematic. It is in particular important to detect the bottle mouth in order to recognize the bottles and to ascertain their dimensions. This is done in particular through the reflection of light at this bottle mouth, which however, is difficult to capture on glass bottles, since no Lambertian reflection takes place. It is furthermore disadvantageous - done with a stereo camera or a mono camera, and essentially the same problems occur as previously described.

The object of the invention is to indicate an apparatus for taking back empties and a method of classifying empties, with the aid of which empties that have been inserted can be reliably classified in a simple manner.

This object is achieved by an apparatus with the features of Claim 1 and by a method with the features of the independent method claim. Advantageous developments of the invention are given in the dependent claims.

According to the invention the apparatus comprises an image capturing unit with the aid of which at least one light field of the capturing region, and with that any empties that may have been inserted, is recorded. The control unit classifies the empties with the aid of this light field. A light field refers in particular to a function that describes the quantity of light that falls in all directions at any point of the three-dimensional space. Thus preferably in the light field, not only the light intensity at each individual image point, but also the direction from which the light falls onto the corresponding point of the image capturing unit, is detected. DE 10 2005 007492 describes an arrangement for three-dimensional measurement of empties in reverse vending machines, wherein a stereoscopic approach is used. US 5,898,169 describes an apparatus for recognizing the contour of a liquid container, wherein a combination of a Fresnel lens and a mirror is used. EP 0 174 549 describes an apparatus for recognizing and logging empties in which the is illuminated by means of a light source and passed in front of a camera. DE 10359781 describes an apparatus for inspecting empties containers, wherein a line scanning camera is used. A light field refers in particular to a function that describes the quantity of light that falls from all directions onto any point of the three-dimensional space. Thus preferably in the light field, not only is the light intensity at each individual image point detected, but also the direction from which the light falls onto the corresponding point of the image capturing unit.

The use of a light field for classifying the empties has the advantage that the three-dimensional measurement of empties containers is possible with the aid of only one capturing unit and only one light field, since this one light field contains more information that is relevant to the classification of crates of bottles than the images of a stereo camera. The light field is, in particular, seven-dimensional, i.e. the light field comprises the three-dimensional geometry, the two-dimensional reflections, the wavelength and the time, so that more information is also available for classifying the empties.

It is in principle possible to perform what is known as refocusing in a light field, i.e. the focusing on different image planes of the light field can be changed subsequently without having to take further pictures of the empties. All the edges needed for the measurement of the empties can thus be made sharp, so that an accurate measurement of these edges is possible. In addition, the light field makes it possible for a large number of different points of view within a region on the empties determined by the recording of the light field to be captured, so that the problem of recognizing and measuring edges that run parallel to epipolar lines is avoided or does not occur.

The apparatus comprises in particular an illumination unit for illuminating the capturing region, wherein the illumination unit and the image capturing unit are preferably arranged such that dark-field illumination of the empties results. With such a dark-field illumination, the empties are illuminated in such a way that only light reflected indirectly by the empties enters the objective lens of the image capturing unit. As a result of this dark-field illumination, the bottle mouths of the empties are illuminated at a shallow angle, and thus the reflection of the mouth in the light field can be classified in a simple manner.

The image capturing unit in particular comprises a plenoptic camera, so that only one camera is necessary for recording the light field. An elaborate calibration of the individual cameras, as is needed with a stereo camera, is therefore not necessary. The plenoptic camera also makes it possible to record the light field of the capturing region economically and in a short time. A plenoptic camera refers in particular to a camera that records a 4-D light field. In contrast to this, a stereo camera only records a 2-D image. In a 4-D light field, not only are the position and intensity of a beam of light at the image sensor known, but also the direction from which the beam of light arrived. The measurement of the light field is made possible by a grid consisting of a plurality of microlenses in front of the image sensor. The special capability of a plenoptic camera is that the maximum depth of field is very high, there is no need to wait for focusing, and the focusing plane of a recorded image can be adjusted subsequently. Depth information can be ascertained from the image data, so that a plenoptic camera is also suitable as a 3-D camera.

The plenoptic camera in particular comprises a lens grid comprising a large number of lenses and arranged in front of the image sensor. Each image point is once again refracted by this lens grid and widened into a cone which meets the sensor surface as a circle. The place at which the beam of light meets the sensor surface indicates the direction from which the beam of light originally came. A beam of light arriving perpendicularly, for example, lands in the centre of the circle, while one which arrives obliquely lands further towards the edge. Thus with the aid of a control unit and/or software, the sharpness can be determined again subsequently, in particular calculated, and the focus can be changed as it can with a "proper" objective lens. A database in particular is stored in the control unit, in which characteristic features are assigned to various classes of empties, wherein the control unit ascertains at least one characteristic feature in the light field, and the empties are classified in dependence on this ascertained characteristic feature. The features in particular are dimensions of the empties, wherein at least one image processing algorithm is stored in the control unit and is executed by the control unit to determine from the light field at least one dimension of the empties.

The empties in particular are empties containers consisting of a beverage crate and potentially of bottles held in it. The dimensions of the beverage crate, the number and/or position of the bottles in the crate, the height of the bottles and/or at least the radius of the bottles are determined as characteristic features. With the help of the light field, the control unit carries out, in particular, a three-dimensional measurement of the empties. In this way an accurate, reliable classification of the empties is achieved, and thereby an incorrect classification, entailing the payment of an incorrect deposit amount, is avoided.

The image capturing unit records in particular only one light field for every empty item entered, so that the classification of the empties is possible with little effort. As a result of the large amount of information contained in just one light field, it is not necessary to capture a plurality of light fields.

The control unit classifies the empties in particular in dependence on the information contained in the light field about the three-dimensional geometry of the empties, the information contained in the light field relating to the two-dimensional reflections that occur, the information contained in the light field relating to the wavelengths, i.e. the spectral composition of the light, and/or the information contained in the light field about the time, i.e. about the change in the incoming light over time. A precise classification of the empties is possible in this way.

Classification refers in particular to the assignment, by the control unit, of the empties that have been entered in the various empties classes. The empties classes can be designed such that each class of empties corresponds to a predetermined deposit amount. Alternatively, the empties classes can be predefined such that a unique class of empties is predefined for each kind of beverage crate used and/or for each type of bottle used. The empties entered can furthermore be classified in the sense of whether an empty item is subject to deposit, is one that is not subject to deposit, or is a foreign object. The empties that are subject to deposit are taken in by the apparatus, whereas the empties that are not subject to deposit and/or the foreign bodies are given out again to the person operating the apparatus.

The apparatus comprises in particular a receipt printer by means of which, after the empties have been entered and been classified, a receipt can be given to the operating person showing the amount of deposit that is due. A further aspect of the invention relates to a method for classifying empties in which the empties are placed in a placement compartment and at least one light field of the empties is ascertained. The empties are classified in dependence on the light field. In this case again, the light field is preferably recorded by a plenop-tic camera.

An accurate classification of the empties can be made with little effort through the classification of the empties using the information contained in the light field. In particular, even edges that run parallel to epipolar lines, and edges in different focus planes, can be measured accurately.

During classification of the objects, in particular at least two different focus planes are ascertained in the light field by the control unit, and at least one measurement of at least one edge of the empties is ascertained in each of these two focus planes.

The classified empties are in particular illuminated with the help of dark-field illumination, so that the edges of bottles, in particular their mouths, can be detected reliably and thus also the dimension of the bottle can be ascertained simply.

The method specified by the independent method claim can be developed further with the aid of the features described in the claims that refer back to the independent apparatus claim, or of their corresponding method features.

Further features and advantages of the invention emerge from the following description, which explains the invention in more detail with reference to exemplary embodiments in association with the annexed figures.

Here:

Figure 1 shows a schematic illustration in plan view of an apparatus for taking back empties according to a first embodiment;

Figure 2 shows a schematic illustration in a side view of an apparatus for taking back empties according to a second embodiment;

Figure 3 shows a schematic illustration in a side view of an apparatus for taking back empties according to a third embodiment;

Figure 4 shows a schematic illustration of a plenoptic camera;

Figure 5 shows a schematic illustration of a light field of a beverage crate when focused on a first plane; and Figure 6 shows a schematic illustration of the light field according to Figure 5 when focusing on the second plane.

Figures 1 to 3 each show a schematic illustration of an apparatus 10, 12, 14 for taking back empties 16, wherein Figure 1 shows a plan view of a first embodiment, Figure 2 shows a side view of a second embodiment, and Figure 3 shows a side view of a third embodiment. The embodiments only differ in the arrangement of a plenoptic camera 18 for recording light fields, for which reason the general structure is described below for all three embodiments together.

The apparatus 10, 12, 14, which is also referred to as a reverse vending machine, comprises a placement compartment 20, in which the empties 16 can be placed by a person who operates the apparatus 10, 12, 14. The empties 16 in particular are empties containers consisting of a beverage crate 22 and a plurality of bottles 24 held in the compartments of this beverage crate 22. In the exemplary embodiment illustrated in Figures 1 to 3, six bottles 24 are arranged in the beverage crate 22. Alternatively beverage crates 22, i.e. beverage crates 22 that do not hold any bottles 24, can be entered as empties 16. It is equally possible for beverage crates 22 with more than six bottles 24, or less than six bottles 24, to be placed in as empties 16. The bottles 24 are in particular glass bottles, i.e. reusable bottles. They can alternatively also be PET disposable bottles or plastic reusable bottles 24.

The apparatus 10, 12, 14 furthermore comprises an illumination unit 26, wherein the illumination unit 26 and the plenoptic camera 18 are arranged such that dark-field illumination of the empties 16 is effected. Dark-field illumination refers in particular to the fact that there is no direct illumination, but that only scattered light from the empties 16 enters into the objective lens of the plenoptic camera 18, which has the advantage that reflections 28 at the bottle mouths 30 of the bottles 24 can be better detected.

With the help of the plenoptic camera 18, a light field of a capturing region of the plenoptic camera 18, in which the empties 16 are arranged is recorded, and thus also the empties 16 themselves. The general structure of the plenoptic camera 18 is described in more detail below in connection with Figure 4. Alternatively, instead of a plenoptic camera 18, it is also possible to use another image capturing unit that is capable of recording light fields.

The apparatus 10, 12, 14 furthermore has a control unit 40, which executes at least one image processing algorithm stored in it, in order to ascertain dimensions of the empties 16 from the recorded light field of the empties 16, and to classify the empties 16 in dependence on these determined dimensions. In particular, the edges of the beverage crate 22, the number and/or position of the compartments of the beverage crate 22, and/or the dimensions of the bottles 24 are ascertained with the aid of the light field or from the light field.

Through the comparison of the three-dimensional dimensions determined for the empties 16 with dimensions stored in a database, the control unit 40 determines whether the object 16 that has been entered is indeed empties 16, and whether these empties 16 are subject to a deposit. In particular, the control unit 40 determines what kind of empties 16 are involved, and how great the deposit is for these empties 1 6. A light field refers to a function that describes the quantity of light that falls in all directions at any point of the three-dimensional space. Thus in a four-dimensional light field, not only is the grey level of each image point ascertained, but the direction of the incoming light beam is also ascertained with the aid of the plenoptic camera 18. This first of all permits refocusing, as is described in more detail in connection with Figures 5 and 6, and also means that the empties 16 in a region predetermined by the plenoptic camera 18 can be observed from different angles of view with the aid of only one recorded light field. The consequence of this is that an accurate measurement of the empties 16 is possible. In particular, edges that run parallel to the epipolar lines can be measured accurately, since appropriate angles of view can be adopted. It is furthermore possible to represent every edge that is to be observed sharply through an appropriate refocusing, so that this edge can be detected reliably and accurately, and its dimensions can be ascertained.

The plenoptic camera 18 also has the advantage that only a single camera is necessary, with the aid of which only precisely one light field has to be recorded for each set of empties 16. A simple and economical construction is thus achieved, and the expense of a calibration, as is necessary in the case of a stereo camera, is avoided.

In the first and the second exemplary embodiments according to Figures 1 and 2, the plenoptic camera 18 records the light field of the empties 16 directly, whereas in the third exemplary embodiment according to Figure 3, a mirror 50 is provided that reflects the empties 1 6 and towards which the plenoptic camera 18 is aligned. This achieves a particularly compact construction.

Figure 4 shows a schematic illustration of a plenoptic camera 18 and of a bottle 24. The plenoptic camera 18 comprises a main lens 42, with the help of which a virtual image 44 of the bottle 24 is created. A lens array 48 of microlenses, one of which is indicated by way of example by reference sign 50, is arranged between the image sensor 46 of the plenoptic camera 18 and its virtual image 44. These microlenses 50 "observe" the virtual image 44, and represent it on the image acquisition sensor 46. Each image point of the virtual image 44 is again refracted by the microlenses 50 of the lens array 48, and expanded into a cone that meets the image acquisition sensor 46 in the form of a circle. The direction of the incoming beam of light can be determined through the position in which the beam of light meets the image acquisition sensor 46 within the corresponding cone of the respective microlens 50.

In an alternative plenoptic camera 18, a 7-D light field that contains even more information about the empties 16 that have been entered can be ascertained instead of a 4-D light field. The seven dimensions in particular are the three-dimensional geometry, the two-dimensional reflections, the wavelength and the time.

Figures 5 and 6 illustrate a light field of a beverage crate 22, wherein the same light field is illustrated in both figures. In Figure 5 the focusing plane, which can also be called the sharp plane, is adjusted to be in the region of the front face of the beverage crate 22, while on the other hand in Figure 6 it is in the region of the rear face of the beverage crate 22, so that in Figure 5 the front face is sharp, and in Figure 6 the rear face is sharp, and the corresponding edges can be reliably and precisely ascertained and measured by executing the appropriate image pro cessing algorithms. Through the use of a light field it is possible to make various focus planes sharp without the need to record multiple images for this purpose. A corresponding change to the focus plane is also referred to as refocusing. In Figures 5 and 6, the edges that are sharply represented are each illustrated by a single line, while the blurred edges are illustrated by double lines.

List of reference signs 10, 12, 14 Apparatus 1 6 Empties 18 Camera 20 Placement compartment 22 Beverage crate 24 Bottle 26 Illumination unit 28 Reflection 30 Mouth 40 Control unit 42 Main lens 44 Virtual image 46 Image sensor 48 Lens array 50 Microlens

Claims

An apparatus for receiving return packaging, with an insertion compartment (20) for introducing the return packaging (16) into a recording area, characterized by a car-led recording unit (18) for recording at least one light field in the recording area, which represents a function which describes the amount of light falling at each point of the three-dimensional space in all directions, and by a control unit (40) which uses the light field to classify the return packaging (16), wherein the return packaging is measured three-dimensional for classification purposes, wherein all the edges of the return packaging, which is necessary for measuring the return packaging, when re-focusing is placed sharply within the light field.

Apparatus (10, 12, 14) according to claim 1, characterized in that a lighting unit (26) is provided for illuminating the recording area.

Apparatus (10, 12, 14) according to claim 2, characterized in that the illumination unit (26) and the image recording unit (18) are arranged such that a dark field illumination of the return packaging (16) is provided.

Apparatus (10, 12, 14) according to one of the preceding claims, characterized in that the image recording unit (18) comprises a plenoptic camera.

Apparatus (10, 12, 14) according to one of the preceding claims, characterized in that a database is stored in the controller (40) and in which characteristic features are associated with different classes of return packaging, the controller (40) in the light field determines at least one characteristic feature and that the controller (40) classifies the return packaging (16) depending on the characteristic feature determined.

Apparatus (10, 12, 14) according to one of the preceding claims, characterized in that at least one image processing algorithm is stored in the control unit (40), the controller (40) executes this image processing algorithm to determine at least one dimension of the return packaging (16). ) from the light field and that the controller (40) classifies the return packaging (16) depending on this dimension.

Apparatus (10, 12, 14) according to claim 6, characterized in that the control unit (40) performs a three-dimensional measurement of the return packaging (16) by means of the light field.

Apparatus (10, 12, 14) according to claim 7, characterized in that the control unit (40) determines the height, width and / or depth of a beverage box (22) introduced as return packaging (16), the positions and / or the dimensions of the space of the beverage boxes (22), and / or the height and / or at least one diameter of at least one bottle (24) accommodated in one of the compartments.

Apparatus (10, 12, 14) according to claim 7 or 8, characterized in that the control unit (40) focuses the image of the return packaging (16) in the light field of a first image plane and at least a second image plane for measuring the return packaging (16).

Apparatus (10, 12, 14) according to one of the preceding claims, characterized in that exactly a light field in the recording area is recorded by the image recording unit (18).

Apparatus (10, 12, 14) according to one of the preceding claims, characterized in that the control unit (40) classifies the return packaging (16) according to the information contained in the light field regarding the three-dimensional geometry of the return packaging (16) regarding the two-dimensional reflections on the return packaging (16), regarding the wavelength and / or the time.

A method for classifying return packaging, wherein - the return packaging (16) is placed in an insertion space (20) in a registration area, - at least one light field (16) of the return packaging (16) is registered with an image recording unit (18) representing the light field a function describing the amount of light falling at each point of the three-dimensional space in all directions, and - the return packaging (16) is classified according to the light field with a control unit (40), - the return packaging is measured three-dimensionally for classification purposes in which all edges of the return packaging needed to measure the return packaging is sharpened by refocusing within the light field.

A method according to claim 12, characterized in that the return packaging (16) is illuminated using a dark field illumination.