DE102008029952A1 - Method and device for detecting transport containers - Google Patents

Abstract

Proposed is a method for detecting transport containers, in particular beverage crates and / or bottles, each of which is provided with at least one graphical identifier. It is of the graphic designation, the z. B. may be a label or logo, made a digital image capture and created an image data matrix. From this an image data list (PICL) is created. Then, by means of a feature list-based cross-correlation, the image data list (PICL) is linked to a pattern data list (TMPL) representing an optical recognition feature feature list to check whether the visual recognition feature is present in the graphic label. This method is particularly well suited for the fast and secure detection of transport containers in reverse vending machines or automatic machines.

Description

The The invention relates to a method for detecting transport containers according to the generic term of claim 1 and a method performing the device.

Especially The invention relates to a method for detecting transport containers in the form of beverage boxes and other empties and a device in the form of an empties return system.

at the optical detection of transport containers, as in particular in empties return systems carried out will come, usually Cameras and image processing systems are used, the graphic Markings, such. B. lettering, pixels or so-called. Detect labels that are provided with the transport containers should. To do this, the cameras take photographs, which then be evaluated in the image processing by means of pattern recognition, to the respective marking, in the following short also called logo, clearly recognizable. Especially for the reliable classification of beverage boxes that often differ only by the particular logo, it is necessary that the respective logo is recognized safely and clearly.

Based on 1 the initial situation is illustrated:
The 1 schematically shows a beverage container designed as BC transport container, which is introduced as empties in a (not shown) empties return system and is to be recognized there. On the beverage box BC graphic markings LBL (short: logos) in the form of lettering and pictograms are appropriate, which indicate a reference to the origin or the owner of the beverage crate (here a brewery) and should be automatically detected safely.

Usually the recognition of such logos is based on the use of distance measurements or neural networks that evaluate previously calculated features. however bump these methods to their limits when the logos or markings different owners are very similar. In particular, here it depends, clearly distinguishable recognition features to find. Could do this Cross-correlation methods are used to capture in the image to search for unique recognition features. However, conventional ones are Cross-correlation method very compute-intensive and can not be carried out promptly or even in real time without considerable effort. Thus, the conventional ones Not for the use in automatic systems, such. B. Empties return systems suitable.

Desirable would it be, to have a method of detecting transport containers that quickly accomplished can be and therefore also for time-critical applications suitable is. In particular, a process to be implemented cost-effectively and a corresponding device was sought, with which a fast and reliable Search for unique recognition features is enabled

Therefore It is an object of the present invention to solve the above problems Of the prior art overcome advantageous. In particular, should a fast working and technically easy to implement solution for optical detection of transport containers are proposed.

Is solved the object by a method having the features of the claim 1 and also by a device with the characteristics of the sibling Claim.

Therefore it is proposed that at least the part of the transport container, which is provided with the graphic identification, a digital one Image capture is done by means of digital image capture or an image processing thereof creates an image data matrix is that creates an image data list from the image data matrix and that by means of a feature list-based cross-correlation the image data list is linked to a pattern data list, the one feature list for represents an optical recognition feature to check whether the visual identification feature is present in the graphic designation is.

It is therefore checked by a feature list-based cross-correlation, whether the respective transport container carries a graphic marking, having a recognition feature that of a template (Template) corresponds. The template or its extracted and generated pattern data list, for example, from a particular succinct and certainly identifiable sub-range of the label LBL won be such. B. the coat of arms of a brewery logo or the like. The pattern data list corresponds to one for the template (template) typical feature list that is based on a characteristic list Cross-correlation can be used. The generated image data list again corresponds to a very strongly data-reduced representation the digital image capture of the respective logo and is in the frame the feature list based cross correlation directly with the pattern data list connected.

As a result, a very fast and reliable detection and classification of transport containers, especially of beverage crates, is achieved. The proposed solution makes it possible to parallelize the data link very much and thus divide it into several cores. The invention can also be applied to individual processors or programmable circuits, such. B. on DSPs or FPGAs, are well implemented and is therefore very suitable for industrial use. In particular, logos, but also other and quite large images can be detected quickly and safely.

advantageously, the image data list is created from the image data matrix by: only for such Coordinate pairs of the image data matrix an entry in the image data list takes place, which indicate a pixel that does not match a data value Zero.

It can advantageously by means of linking the image data list with the pattern data list, a result data matrix are created, the is equal to the result of a conventional cross - correlation of Image data matrix with a pattern data matrix from which the pattern data list is creatable. It can the image data list and the pattern data list in rows or columns Have entries, each pair of coordinates and at least an associated one Specify data value. In this context, it is beneficial if the image data list in particular line by line with the pattern data list connected is determined by taking each of the coordinate values of a pair of coordinates the image data list in each case the coordinate values of a coordinate pair subtracted from the pattern data list by one coordinate pair at a time in one or the result data matrix to determine a result data value there to be entered by summing intermediate results is, each consisting of a shortcut, in particular multiplication or absolute difference, of the respective data value the image data list with the corresponding data value from the pattern data list result.

In addition, the Interim results at the same time over Parallelized calculation methods can be determined and then the Intermediate results for determining the result data values merged, in particular be added.

The Image processing for creating the image data matrix preferably comprises a feature extraction, in particular an edge detection, of the image data the digital image capture.

• – von der Mustervorlage bzw. der grafischen Kennzeichnung selbst wird eine Musteraufnahme gemacht;
• – die Musteraufnahme wird einer Mittelung bzw. Filterung unterzogen;
• – aus der insbesondere gemittelten Musteraufnahme wird mittels einer Kantendetektion ein Merkmalsbild erstellt;
• – aus dem Merkmalsbild wird das Erkennungsmerkmal extrahiert und eine entsprechende Musterbilddatenmatrix erstellt,
• – aus der Musterbilddatenmatrix wird die Musterbilddatenliste erstellt, indem jeweils nur für solche Koordinaten-Paare der Musterbilddatenmatrix ein Eintrag in die Musterbilddatenliste erfolgt, die einen Bildpunkt angeben, der einen Datenwert ungleich Null aufweist.

The pattern data list can also be generated by means of an extraction of image data from a template for the graphic designation or from the respectively present graphic designation itself. In this context, the generation of the pattern data list preferably comprises at least one of the following steps:

• - a sample is taken of the template or graphic designation itself;
• The pattern recording is subjected to averaging or filtering;
• - From the particular averaged pattern recording a feature image is created by means of edge detection;
• The identification feature is extracted from the feature image and a corresponding pattern image data matrix is created,
• - From the pattern image data matrix, the pattern image data list is created by an entry into the pattern image data list is only for such coordinate pairs of the pattern image data matrix, which specify a pixel having a data value other than zero.

The Invention will now be described in detail with reference to exemplary embodiments, with reference to the following accompanying schematic drawings is taken:

1 shows a recognizable drink box with label or logo;

2a shows a flowchart for the inventive method, are recognized with the beverage boxes and the like on the basis of logos, in particular sub-steps for generating an image data list are shown;

2 B indicates the procedure 1 in particular sub-steps for a previously performed generation of a pattern data list;

3 refers to 2 B and shows a stepwise extraction of a pattern template for the generation of the pattern data list carried out in the method according to the invention;

4 illustrates the computational effort of a conventional cross-correlation of an image data matrix with a pattern image data matrix;

5 shows a conversion of an image and pattern image data matrix into a pattern image data list performed in the method according to the invention;

6 illustrated in comparison to 4 the result obtained by feature list-based cross-correlation.

7a ) -N) illustrate the principle of feature list-based cross-correlation.

Starting from the in 1 illustrated initial situation in which, for example, the logo or label LBL a beverage box BC he to be known, the show 2a and 2 B the steps of a method according to the invention 100 in that advantageously solves this problem as follows:
In a later step described in detail 110 For example, a template image data list TMPL is generated to provide for the feature list based cross-correlation proposed herein (see, in particular, step 130 ) to be used.

In one step 120 , the sub-steps 121 to 123 includes, an image of the image to be recognized beverage box or its logo is first made and therefrom an image data list PICL generated. This is done first in a sub-step 121 the actual digital image capture PIC made of the logo. In a partial step 122 From this, an image data matrix PICD is created whose coordinates x and y indicate the respective pixel with an associated data value Wb. For example, a black-and-white image acquisition is made with an 8-bit resolution, so that the respective data value Wb corresponds to a gray scale value between 0 and 255. In a further sub-step 123 Now, a greatly reduced image data list PICL is generated from the image data matrix PICD without loss of information. This will be even closer to the 5 described.

With the help of the generated image data list PICL and the provided pattern image data list TMPL is then in step 130 a feature list-based cross-correlation is performed, which has the same quality as a conventional cross-correlation in the result, but due to the created data lists PICL and TMPL and their inventive combination allows parallelized data processing and thus can be performed much faster. Thus, a particularly fast and effective detection of the respective beverage box can be achieved by its logo or similar graphic markings.

Before the logo recognition using feature list-based cross-correlation is discussed in more detail, we will start by looking at the 2 B the creation or generation of the pattern image data list TMPL are described in more detail:
This can in the context of installation of the device according to the invention, which is preferably an empties return system, be carried out once or only from time to time and takes place for. B. in the process step 110 , the sub-steps 111 to 114 includes.

First, in the sub-step 111 from a template that represents an ideal image capture of the label, or even made from a label already located on a beverage box a digital pattern image recording PIC * made. Alternatively, a corresponding sample image file can be used as a template.

Then in the sub-step 112 From this a feature image FPIC * is created, ie a picture that reproduces as concisely as possible the graphic structure typical of the label. The feature image FPIC * is z. B. generated by edge detection from the pattern image PIC *. The feature image FPIC * then shows in particular the outlines of logo elements and lettering that characterize the label.

In a next step 113 Then an area is extracted from the feature image FPIC *, as here z. B. a coat of arms. It may also be another or similar characteristic picture element. This is used as an optical identification feature or template TMP for later recognition (step 130 ) used. The template thus serves as a sample image. The 3 also illustrates the generation of such a pattern image TMP.

In order to be able to carry out the subsequent recognition of beverage crates on the basis of a feature list-based cross-correlation, the pattern image TMP or its corresponding pattern image data matrix TMPD is brought into a list form according to the invention and made available as a pattern data list TMPL. This process will be described in more detail on the basis of 5 described.

First, based on the 4 be shown how a conventional cross-correlation would have to be performed and what computational effort would be involved:
In the 4 is exemplified in a simplified form the image data matrix PICD represented, which represents the digital image acquisition of a logo or label LBL. Furthermore, the pattern image data matrix TMPD is shown, which as a reference represents the template TMP. This should be found again in the image to identify the beverage box. Usually, a result image data matrix RD is calculated for this purpose by means of cross-correlation of the two matrices PICD with TMPD, which then indicates how strongly and at which pixel position the image acquisition PIC correlates with the pattern image TMP. In the example shown here in simplified form, an M × M matrix PICD (M = 5) is linked to the N × N (N = 3) matrix TMPD, so that one (M + N-1) × (M + N - 1) gives a large matrix RD. In the example shown here, it can be seen from the result image matrix RD that a significant singularity in the form of a very high result value of 225 occurs at one point (coordinates X = 4 and Y = 3). This indicates for this point a very strong correlation of the image pickup PIC with the pattern image TMP, from which it follows that in the image pickup PIC the pattern image TMP is present in the middle, upper area of the pixels. This will make the corresponding drinks box clearly identified.

However, the in 4 For example, conventional cross-correlation illustrated an expensive calculation of M × M × N × N, that is, 5 × 5 × 3 × 3 = 25 × 9 = 225 image data values. The resulting result data matrix RD then has M + N - 1 columns or rows, so it has 7 × 7 entries.

According to the present invention, a much faster link in the form of a feature list-based cross-correlation is now proposed here, which in step 130 (S. 2a / b) and subsequently using the 5 and 6 such as 7a ) to n) is described in detail:
First, the image data matrix PICD is brought into a list form and converted into an image data list PICL. Each line of the image data list PICL contains the following information: The X and Y coordinates for a pixel and the data value Wb indicated there. Only those coordinate pairs which have a data value Wb other than zero are entered in the list PICL. This results in a much smaller amount of data without any loss of information.

The same thing was previously performed with the pattern image data matrix TMPD, the thus in the form of a short pattern data image list TMPL, where there also only data values Wh not equal to zero are listed.

As the 5 illustrates, a combination of the two lists PICL is performed with TMPL, which is referred to here as a feature list-based cross-correlation and also, but with significantly lower computational effort, leads to the same result, namely to the result data matrix RD.

The 7a ) to k) clarify the principle or the concrete procedure of the feature list-based cross-correlation proposed here:
The combination of the two lists PICL and TMPL is performed line by line, starting from the first line of the list PICL, the local coordinates X and Y and the data Wb are linked to the corresponding information from the first line of the list TMPL; then follows the second and third line of the list TMPL. Then it continues with the second line of the list PICL, whose information is then linked in each case with the information from the first, second and third line of the list TMPL. This continues until all the rows of the list PICL have been linked to those of the list TMPL. In the example shown, the list PICL has 5 lines and the list TMPL has 3 lines, so that in total only 5 × 3 = 15 links have to be calculated. It should be emphasized that these arithmetic operations, which are described in more detail below, can be distributed in parallel to several arbitrary arithmetic cores and only at the end of the individual intermediate results must be merged via a summation element or accumulator, which in the realization of the proposed method is a very fast and powerful image recognition means.

The 7a ) -C) show the beginning of the procedure, in which first the information from the respective first line of both lists PICL and TMPL are linked. First, a new coordinate pair Y * / X * is determined from the coordinate values of the specified coordinate pairs Y / X or Y '/ X', which indicates in the result data matrix RD the position at which a first intermediate result W11 (see also FIG Intermediate result matrix g ') is to be expected as a share or contribution for the correlation result due to the combination of the corresponding data values Wb with Wh.

The new coordinate values Y * and X * result from subtraction the respective coordinate values from the lists, namely how follows: Y * = Y - Y 'and X * = X - X'.

In the example shown 7a ) -C) the first new pair of coordinates is calculated to be: Y * = 0 - 0 = 0 and X * = 2 - 1 = 1. Thus, the first position in the result data matrix is known. The intermediate result W11 to be entered there (proportion of the correlation result W *) results, for example, from B. by multiplying the two data values Wb and Wh from the lists PICL and TMPL (according to the formula: W11 = Wb × Wh) and is in the example shown 10 × 10 = 100. So this value is the first intermediate result W11 and is at the Y * / X * position 0/1 entered in the result data matrix RD. Instead of multiplication and operations with any distance measures, such. As the absolute difference, or it can also be performed a binary link.

Would be there already entered a data value W * not equal to zero (which at the beginning the method is not the case), the value W11 would be used as a proportion or Contribution to the overall result can be added.

ist die Formel für die Berechnung des Ergebnisbildes bzw. der Ergebnisdatenmatrix. Diese setzt sich aus der Summation der einzelnen Zwischenergebnismatrizen g' zusammen und ergibt zum Schluss die gewünschte Ergebnisdatenmatrix RD (s. auch 7n). Insgesamt werden in Schritten, deren Anzahl durch die Listenlängen Sb bzw. Sh bestimmt ist, jeweils zeilenweise Zwischenergebnisse Cij berechnet und an der betreffenden Position in der Matrix g' zu dem vorherigen Wert hinzugefügt. Px,y,i,j = δ(x – (bx[i] – hx[j]))·δ(y – (by[i] – hy[j]))entspricht einem Ausblendfaktor und dient der Positionsbestimmung im jeweiligen Zwischenschritt. Somit wird nur dann eine Rechenoperation durchgeführt, wenn an den betreffenden X- und Y-Positionen Datenwerte W ungleich Null vorkommen. ci,j = bv[i]·hv[i]gibt das Abstandsmaß an und entspricht somit dem jeweiligen Wert W, der hinzuzufügen ist, hier dem Wert W11 für die Position X = 1 und Y = 1.The particular in 7c The formulas shown here indicate the following:

is the formula for calculating the result image or result data matrix. This is composed of the summation of the individual intermediate result matrices g 'and finally results in the desired result data matrix RD (see also FIG 7n ). Overall, in steps whose number is determined by the list lengths Sb and Sh respectively, row-by-row intermediate results Cij are calculated and added to the previous value at the respective position in the matrix g '. P x, y, i, j = δ (x - (b x [i] - h x [j])) · δ (y - (b y [i] - h y [J])) corresponds to a blanking factor and serves to determine the position in the respective intermediate step. Thus, an arithmetic operation is performed only when data values W other than zero occur at the respective X and Y positions. c i, j = b v [I] · h v [I] indicates the distance measure and thus corresponds to the respective value W to be added, here the value W11 for the position X = 1 and Y = 1.

It then goes on gradually, ie that here now, as in 7d ), linking the data from the first row of PICL with those from the second row of TMPL.

from that results in that in the result data matrix RD at the Y * / X * position -1/0 the Interim result W12 = 20 entered or added got to.

The 7e ) to 7i ) illustrate the subsequent steps in which, line by line, the contributions W13, W21, W22 ... are calculated as intermediate results to be added. It becomes clear from the example shown here that only data linking or arithmetic operations are carried out which actually deliver an intermediate result not equal to zero as a contribution to the result data matrix RD. This makes the feature list based cross-correlation very efficient. In addition, the individual intermediate results can be calculated in parallel arithmetic operations and need only be merged at the end by a simple summation. This allows a realization by a parallel and thus fast computer structure. This can, for. B. according to the number of lines of the pattern data image list TMPL a computer structure with N cores, so here with 3 cores be. Since N is determined according to the respectively used template TMP or its list TMPL and remains constant, this division is particularly easy to implement. It may also be a structure corresponding to the number of lines M of the image data list PICL, but it should be noted that M is not constant. Also, further divisions are conceivable.

The 7k ) illustrated in comparison to the previous ones 7a ) -I) a further intermediate result W32 = 4 calculated for the Y * / X * position 0/1 is added to the previous data value W * = W11 = 100 and thus yields the new data value W * = 104.

The 7l ) to n) illustrate the continuation of the method up to the linking of the last line of PICL with the last line of TMPL, so that the result is the same result data matrix RD, which has a conventional cross-correlation (s. 4 ) could be calculated, but at a much greater expense. Moreover, conventional cross-correlation could not be realized by parallelized arithmetic operations or structures.

By is the feature list based cross correlation described herein it is possible also larger pictures especially fast to process. Because the feature lists are not essentially dependent on the data volume of the respective image recordings, but rather, the data content or whether and how many feature-typical image components available. By the feature extraction proposed here can correspondingly short lists are created without the correlation result to influence negatively. The feature list-based cross-correlation itself is especially in high-contrast and low-noise Images much faster than a conventional cross-correlation. By the application of the feature list based cross-correlation itself there is no loss of image information. The procedure presented here indicates in particular by the parallelizability of computational structures a high computing speed. This is essentially not more (as with conventional Method) depends on the image size, but from the image content, in particular from the number of images present in the image Feature points.

The Application of the method is particularly good for fast and safe Detection of transport containers suitable, as z. B. in empties return systems or machines is required. The use is not, however, on such systems, in particular, not limited to the recognition of beverage crates and / or bottles, but includes all similar ones Systems or machines and more going beyond all comparable applications.

BC

Transport container, here in the form of a beverage box

LBL

graphic Label (label or logo), here as lettering and picture element

PIC

digital Image capture of the logo

PICD

Image data matrix

PICL

Image Data List

TMP

optical Distinguishing feature (sample image or template)

TMPD

Pattern picture data matrix

TMPL

Pattern image data list

RD

Result data matrix (from summation of intermediate result matrices g ')

PIC *

Sample image capture and

FPIC *

feature image for obtaining the recognition feature (template)

100

Method with step sequences 110 - 130

x, y

coordinates the image data matrix PICD

x ', y'

coordinates the pattern image data matrix TMPD

x * y *

coordinates a result image data matrix RD

wb

data values the image data matrix PICD

Wh

data values the pattern image data matrix TMPD

W11, W12, W13 ...

interim results (Shares in the overall result)

W *

data values the result image data matrix RD

Claims (11)

Procedure ( 100 ) for detecting transport containers (BC), in particular beverage crates and / or bottles, each of which is provided with at least one graphic identification (LBL), the method ( 100 ) comprises the following steps: - a digital image acquisition (PIC) is made at least of the part of the transport container (BC) provided with the graphic identification (LBL) (substep 121 ); - By means of the digital image acquisition (PIC) or image processing thereof, an image data matrix (PICD) is created (sub-step 122 ); - From the image data matrix (PICD) an image data list (PICL) is created (sub-step 123 ); By means of a feature list-based cross-correlation, the image data list (PICL) is linked to a pattern data list (TMPL), which represents an optical recognition feature (TMP) feature list, to check whether in the graphic designation (LBL) the visual recognition feature (TMP ) is present (step 130 ). Procedure ( 100 ) according to claim 1, wherein the image data list (PICD) is generated from the image data matrix (PICD) ( 123 ) (sub-step 123 ), in that an entry in the image data list (PICD) is only entered for those coordinate pairs (x, y) of the image data matrix (PICD). 123 ) indicating a pixel having a non-zero data value (Wb). Procedure ( 100 ) according to claim 1 or 2, wherein by means of the combination of the image data list (PICL) with the pattern data list (TMPL) a result data matrix (RD) is created (step 130 ), which is equal to the result of a conventional cross-correlation of the image data matrix (PICD) with a pattern data matrix (TMPD) from which the pattern data list (TMPL) can be created. Procedure ( 100 ) according to one of the preceding claims, wherein the image data list (PICL) and the pattern data list (TMPL) have entries in rows or columns, each containing coordinate pairs (x, y; x ', y') and at least one associated data value (Wb, Wh). Procedure ( 100 ) according to claim 4, wherein the image data list (PICL) is linked, in particular, line by line with the pattern data list (TMPL), by in each case from the coordinate values of a coordinate pair (x, y) of the image data list (PICL) the coordinate values of a coordinate pair (x ', y') of the pattern data list (TMPL) are subtracted in order to respectively determine a coordinate pair (x *, y *) in a result data matrix (RD) in order to enter there a result data value (W *) Summation of intermediate results (W11, W12, W13 ... W21, W22 ...) is determined, each of a link, in particular multiplication or absolute difference, the respective data value (Wb) from the image data list (PICL) with the corresponding Data value (Wh) from the pattern data list (TMPL) result (step 130 ). Procedure ( 100 ) according to claim 5, wherein the intermediate results (W11, W12, W13 ... W21, W22 ...) are determined at the same time via parallel computing methods (substep 131 ) and then the intermediate results (W11, W12, W13... W21, W22...) for determining the result data values (W *) are combined, in particular added, (substep 132 ). Method according to one of the preceding claims, wherein the image processing for producing the image data matrix (PICD) comprises a feature extraction, in particular an edge detection, of the image data of the digital image acquisition (PIC) (sub-step 122 ). Method according to one of the preceding claims, wherein the pattern data list (TMPL) is generated by means of an extraction of image data from a template for the graphic designation (LBL) or from the respectively present graphic designation (LBL) itself (step 110 ). The method according to claim 8, wherein the generation of the pattern data list (TMPL) comprises at least one of the following steps: - a pattern recording (PIC *) is made of the pattern or graphic designation (LBL) itself (sub-step 111 ); - The pattern recording (PIC *) is subjected to averaging or filtering (sub-step 112 ); - From the particular averaged pattern recording (PIC *) a feature image (FPIC *) is created by means of edge detection (sub-step 113 ); - The feature (TMP) is extracted from the feature image (FPIC *) (sub-step 114 ) and a corresponding pattern image data matrix (TMPD) is created, from the pattern image data matrix (TMPD) the pattern image data list (TMPD) is created (substep 114 ) In each case only for such coordinate pairs (x ', y') of the pattern image data matrix (TMPD) is an entry in the pattern image data list (TMPD) indicating a pixel having a data value (Wh) not equal to zero. Device for detecting a transport container (BC), in particular empties, containing at least one graphic Labeling (LBL) is provided, wherein the device at least one Camera has with at least the part of the transport container (BC), which is provided with the graphic identifier (LBL), a digital image acquisition (PIC) is made; and computing resources having by means of digital image acquisition (PIC) or a Image processing of which creates an image data matrix (PICD), the create an image data list (PICL) from the image data matrix (PICD), and the by means of a feature list-based cross-correlation the image data list (PICL) is linked to a pattern data list (TMPL), the a feature list for represents an optical identification feature (TMP) to check whether in the graphic designation (LBL) the optical identification feature (TMP) is present. Device according to claim 10, characterized in that that the device is an empties return machine, the transport containers in the form of beverage boxes and / or bottles recognizes.