WO2019002114A1

WO2019002114A1 - Device and method for all-round inspection of containers on a conveyor belt

Info

Publication number: WO2019002114A1
Application number: PCT/EP2018/066707
Authority: WO
Inventors: Anton Niedermeier; Herbert Kolb; Stefan Piana
Original assignee: Krones Ag
Priority date: 2017-06-26
Filing date: 2018-06-22
Publication date: 2019-01-03
Also published as: DE102017114081B4; DE102017114081A1

Abstract

The invention relates to a device (1) for the optical inspection of objects (10), said device having a transport device (40) which transports the objects (10) along a transport path, having a first image recording device (20), which is suitable and intended for recording at least one first spatially resolved image (B1) of a first surface region (01) of the object (10) under a first observation angle (W1), having at least one second image recording device (20), which is suitable and intended for recording at least one second spatially resolved image (B2) of a second surface region (02) of the object (10) under a second observation angle (W2) which differs from the first, wherein the first surface region (01) and the second surface region (02) of the object (10) coincide at least in an overlapping region (U12). An evaluation unit is provided in accordance with the invention and is suited and intended for identifying an area of the second image (B2) corresponding to at least one area of the first image (B1), in which in each case the overlapping region (U12) is represented at least in part and preferably completely, and the evaluation unit is suited and intended for generating a composite image (E) based on this identification from at least the first spatially resolved image (B1) and the second spatially resolved image (B2), wherein the evaluation unit takes into consideration in its evaluation a surface curvature of at least a section of the object (20) and/or compression or distortion of at least one represented region of the image (B1, B2) caused by the process of recording an image in an image recording device (20).

Description

Device and method for all-round inspection of containers on the conveyor belt

description

The present invention relates to an apparatus and a method for optical inspection, preferably for all-round inspection of objects, in particular the outer surface of objects or containers and preferably for the purpose of checking or checking the positioning of labels on containers. From the prior art devices and methods are known in which a 360 ° -tags control rotationally symmetrical container is performed with calibrated cameras, which share a common coordinate system. Based on the contour of the container, the position of the container within the common coordinate system is determined by each camera and a panoramic image is created on the basis of this position. For this purpose, a previously created SD model of the container is projected onto the position in the coordinate system and then a 3D-2D transformation is performed individually by each camera. These frames are now combined to a panorama picture. On this panorama picture, which contains the complete view of the container, checks are now made regarding a correct arrangement of the label.

Assembling the frames works very well if the original (container) exactly matches the 3D model. However, the devices and methods that are currently known from the prior art have the disadvantage that, if deviations occur, for example diameter tolerances, an incorrectly detected contour, an inaccurate position calculation, poor calibration, etc., to the Interfaces of the frames information is displayed twice or not at all. If at the If a barcode is present in a vertical display of the barcode modules, one or more modules will not be displayed twice or in the image. A downstream barcode analysis can not read the barcode. Thus, there can not be 100% control of the products in production. This is important, for example, if the barcode should be judged whether it is also readable, such as a barcode reader at the cashier in the supermarket. A system with these defects is then rejected by the customers. This disadvantage also applies to other codes, such as data matrix or alphanumeric coding, for example the imprint of the expiration date (MHD).

The object of the present invention is to overcome the drawbacks known from the state of the art and to provide an apparatus and a method which is able to tolerate generally occurring tolerances, such as in the manufacture of the object or the exact position of the object Object to compensate, while ensuring a short measuring time or a high flow rate of the objects in order to achieve a secure quality control of the outer surface of an object.

The object is achieved by the subjects of the independent claims. Advantageous embodiments and further developments of the invention are the subject of the dependent claims.

A device according to the invention for optical inspection, preferably for optical all-around inspection of objects, has a transport device which transports the objects along a transport path, and a first image recording device which is suitable and intended for this purpose at least a first spatially resolved image of a first Receiving at least a second spatially resolved image from a second surface area of the object at a second, different viewing angle from the first, wherein the first surface area; and at least one second image pickup device the second surface area of the object coincide at least in an overlap area. Preferably, the device performs a 360 ° (label) control of the objects. According to the invention, an evaluation device is provided which is suitable and intended to identify a region of the second image corresponding to at least one region of the first image, in which the overlapping region is imaged at least regionally and preferably completely, and the evaluation device is suitable and is determined based on this identification at least from the first spatially resolved image and from the second spatially resolved image, preferably from at least one of the image pickup devices taken images, which particularly preferably at least one overlap region with at least one image pickup device to produce a composite image, wherein the evaluation device preferably favors a surface curvature of at least one section of the object and / or a compression or distortion of at least one caused by the image recording of an image recording device formed area of the image in their evaluation. In other words, preferably, the first spatially resolved image and the second spatially resolved image (or the spatially resolved images taken by the image pickup devices, and more specifically, the images used to produce (more precisely) a composite image) are not picked up by the same shot point. That is to say, preferably the first image recording device and the second image recording device and / or the and particularly preferably all image recording devices (at least at the time of their respective image recording) are spatially arranged at different points relative to the inspected object.

A region of the second image corresponding to a region of the first image is preferably understood to mean that in this region the same surface region of the object inspected or to be inspected has been recorded or imaged by the first image recording device and by the second image recording device. In the first spatially resolved image recorded by the first image recording device and in the second spatially resolved image recorded by the second image recording device, at least one identical section of the object is therefore to be seen or recorded or imaged. The overlapping area (with respect to the first and the second image pickup device or the first and the second image) is preferably just the (surface) area of the object that is from both the first image pickup device in the first image and the second image pickup device shown in the second picture becomes. In the case of a plurality of image recording devices, the overlapping region with respect to two image recording devices is just the (surface) region of the object which is imaged both by the one image recording device in the image taken by it and by the other image recording device in the image taken by it. Due to the different viewing angles, this overlapping area is imaged in different ways; in particular, this may result in a different compression or distortion of the respective overlapping area in the respective recorded image as a result of a 3D 2D transformation respectively generated by the image recording device.

The generation of a composite image is preferably understood as the creation of a large image from different (preferably) smaller individual images, preferably from images recorded by different image recording devices. The composite image can be seen as a kind of panoramic image of a larger recording area of the outer surface of the object and can be composed by comparable methods or algorithms as in a stitching process. However, in contrast to a panoramic image, the recording of the smaller individual images does not take place from a single recording position and rotation of the image recording device, but by the smaller individual images (or the spatially resolved images) are recorded (preferably from different image recording devices) at different recording positions.

A complete elimination of the deviations occurring during assembly of the images by methods of the above-described prior art is practically impossible or associated with extremely great effort. The invention now exploits the image information of the overlapping areas at the interfaces to produce a correctly assembled image.

Due to the proposed method or the proposed device, it is preferably possible on the basis of an image which is also perfectly assembled at the interfaces, for example to carry out bar code analyzes, if the code is distributed over several partial images. Complete control of the objects or containers is thereby made possible. Likewise, other codings or imprints can be verified or read much more reliably. Furthermore, the effort for the correct contour search, etc. is limited. In addition, that is System fault-tolerant against incorrectly set parameters or very large diameter tolerances in the containers.

In other words, in the device or method proposed according to the invention, it is preferable to search for identical points by different algorithms using an algorithm in two images, and then to superimpose the images exactly at the same points. This approach minimizes the error at the interfaces largely or preferably optimally (as soon as the image content provides enough information for this). Furthermore, the proposed method or device has the advantage that no alignment of the objects or containers to be inspected, for example a specific rotational position of the (stationary) object on the conveyor belt or the transport device is required. This approach can be used in general for all applications in which images must be assembled. Preferably, the objects are containers, preferably labeled and / or printed containers. In general, the at least one region of the image to which the evaluation device searches a corresponding region of the second image can be a single pixel. Preferably, however, the area is a plurality of pixels or pixels.

Preferably, the transport device takes at least sections and particularly preferably in the section of the device in which the optical inspection is carried out or in which the at least first and the at least second image pickup device and in particular in all involved to produce a composite image image pickup devices are arranged a one-lane transport of the objects or of the object to be inspected. Preferably, the transport device has a lateral fixing or guiding of the objects, whereby the objects can be held on the transport path. Preferably, therefore, the degree of freedom of movement in at least one spatial dimension, preferably at least one direction perpendicular to the transport path, is limited. This allows the most accurate positioning possible and minimal lateral deviation from a given transport path can be achieved. This offers the advantage that the evaluation device can be set up in such a way that only minor fluctuations of the lateral positioning of the objects to be inspected relative to the transport path have to be taken into account, which results in a shorter evaluation time and This can be associated with a higher throughput rate. The transport device preferably transports the objects or containers to be inspected upright. Preferably, the transport device transports the objects to be inspected such that the area to be inspected is (at least in sections) located on a peripheral wall of the object, that is, in particular is not located on a bottom area of the object. Therefore, the bottom area of the object preferably lies on the transport device at least in sections during transport. The transport device may have at least one conveyor belt. Preferably, the transport device transports the objects separated and / or spaced apart or with defined gaps (minimum distances) between the objects. Preferably, therefore, the transport device leads the objects of the optical inspection isolated and / or spaced from each other and again from. Preferably, the transport device is suitable and intended to transport a plurality of objects, in particular one after the other, preferably to transport at least 5,000, preferably at least 10,000, preferably at least 20,000, preferably at least 40,000 and particularly preferably at least 60,000 objects (or containers) per hour.

In an advantageous embodiment, the device has a separating device, which is arranged upstream with respect to the image recording devices in the transport direction and the objects directly transported by the transporting device separated and / or spaced and / or a (minimum) distance between at least influences and / or adjusts two immediately consecutive objects. The separating device can be arranged directly in front of the optical inspection unit, which has the first and second image recording device. This has the advantage that no monitoring or control of the backwater or the required object distance or container spacing in the inlet or outlet must be present. This is preferably taken into account in the scheduling of the transport technology or the transport device or the singulation device. Preferably, the influencing or the adjustment of the (minimum) distance between at least two, preferably between two directly consecutive or subsequently transported objects takes place in such a way that this (minimum) distance is at least 40%, preferably at least 50% is at least 60% and more preferably at least 70% of the object diameter or the container diameter. Preferably, the influencing or the adjustment of the (minimum) distance between at least two, preferably between two immediately consecutive or subsequently transported objects takes place in such a way, that this distance is at most threefold, preferably at most twice, preferably at most 1.5, and preferably at most the simple object diameter or container diameter. As a result of the two-dimensional imaging of the three-dimensional object by the image recording device, a distortion or compression of the pixels dependent on the spatial arrangement of the respectively imaged surface region relative to the image recording device preferably takes place in comparison to the imaged regions of the object, which therefore also depends in particular on the surface curvature of the object depends. The surface curvature of at least one section of the object can be given to the evaluation device as a known quantity (regionally), but it can also be determined by the evaluation device itself (for example, from the object diameter). However, it is also conceivable for the evaluation device to specify a specific area for the surface curvature (at least for a given area or for a plurality of predefined areas of the object) which preferably corresponds to the tolerance range for the production or production of the object or such a tolerance range also included. This offers the advantage that the evaluation device thus has information for the surface curvature (s) available for its evaluation, within which the surface curvature lies and a fine adjustment or fine determination of this can take place on the basis of the recorded images of the object. As a result, the evaluation device may for example pre-select a preselection of a region of the first image or predetermine it, which at least in sections and preferably completely contains an image of the overlap region. In this way, an identification of a region of the first image and of a region of the second image corresponding thereto, in which the overlapping region is imaged at least in regions and preferably completely, can take place more quickly. It is also conceivable that an area of an image is selected as the overlapping area or as a corresponding area, which has been found by the evaluation device in a preceding object inspected by the device as a corresponding area.

In a further advantageous embodiment, the evaluation device takes a, in particular linear, transformation of the recorded pixels of at least one region of an image captured by an image recording device, particularly preferably of at least one region of the first image and / or of the second image which takes into account a surface curvature and / or a distortion caused by a 3D-2D projection. The evaluation device preferably first makes equalization of the images recorded by the image recording device or by the image recording devices. In this step, advantageously, no information is lost because the original image is preferably processed linearly. This means that every existing original image pixel is present in the equalized image. Thus, it is preferred to make only negligible errors in case of inaccuracies. In the case of the equalization carried out by the evaluation device, a (surface) curvature of the object is preferably eliminated or a distortion caused by the 3D-2D projection is transformed back.

In a further advantageous embodiment, the evaluation device makes an identification of mutually corresponding areas on the pixel level and / or identifies mutually corresponding pixels of at least two images. Preference is therefore given to pixel-precise stitching.

In a further advantageous embodiment, the evaluation device takes into account only a predetermined area of a recorded image for identifying mutually corresponding areas of two recorded images and preferably each alone a predetermined range, which is preferably less than 20%, preferably less than 15%, preferably less than 10 %, preferably less than 5%, and more preferably less than 1% of the total area of the recorded image occupies.

Preferably, the first and / or the second image or preferably the images taken by the image recording devices are in each case only tested in a specific area which occupies less than 20%, preferably less than 1% of the recorded area. In the identification of mutually corresponding regions, the evaluation device preferably takes into account only the interfaces of two images, which can preferably be given in each case by at least one (lateral) image edge region and particularly preferably by a captured or imaged (lateral) edge region of the object.

In a further advantageous embodiment, the evaluation device uses at least to identify mutually corresponding regions of two recorded images an evaluation method that is selected from a group that includes the analysis of contrasts, the analysis of offset and or skew through lines, edge filters, Laplace filters, correlation functions, and combinations thereof. For this purpose, the evaluation device preferably selects a group of pixels from the first image (preferably from an edge region). The evaluation device preferably carries out a search for a uniqueness or a high difference, for example in the intensity of these pixels. In the image evaluation by the evaluation device, in particular filters such as edge filters and / or Laplace filters can be used. Preferably, the identification of mutually corresponding regions of two recorded images is performed on the basis of the respective images (which have been rectified by the evaluation device). Preferably, the evaluation device uses a rectangular, preferably square area of the first image, preferably after its equalization, as so-called reference image in order to identify mutually corresponding regions. The evaluation device preferably selects a region of the second image which is in particular the same size (particularly preferably already equalized by the evaluation device) as a so-called search image. This can happen at random (in a certain image area of the second image, for example an edge area), but this exit area for the search image is preferably already specified (by the operator and / or the surface curvature or the set object type to be inspected). Preferably, the selected area of the second image, the search image, is shifted in at least one (predetermined, preferably rectilinear) direction and, based on such translation, a correlation function of the area of the first (equalized) image and the area of the second (rectified) image determined. The direction of translation preferably corresponds to a (depicted) rotational direction of the object about a (vertical) longitudinal axis of the object (preferably around a longitudinal axis of the container), preferably about a (vertical) longitudinal axis of the object perpendicular to the transport direction. The gray values of the reference image and of the respective search image are preferably compared (pixel by pixel). A correlation function based on the gray values of the reference image is preferably set up or calculated in comparison to the gray values of the respective (varied or translated) search image.

As a correlation function, a function can be used, in each of which the absolute differences of the gray values (or the intensities or color values) (pixel by pixel) are used. The sum of the absolute differences (SAD, "Sum of Absolute Differences") can be summed, but alternatively and / or additionally, the evaluation can also take place via a correlation function which takes into account the sum of the squared absolute differences (so-called "absolute differences"). SSD, Engl, for "Summed Squared Differences"). As correlation functions, however, (alternatively or additionally) so-called optical flow

Methods, general Hough transformations, distance transformation functions such as true-distance and / or fast-marching and / or object-detection methods can be used around with hair-like features.

The evaluation device preferably determines, via the evaluation of the comparison of the reference image with the at least one search image and preferably the translated or (preferably in the row direction or in the rectilinear direction) shifted or varied search images, preferably via the evaluation of a correlation function corresponding to the reference image Area.

The evaluation device preferably generates a composite image based on the region of the second image corresponding to the (at least) one region of the first image (the reference image), in which the first image (preferably after its equalization) and the second image (preferably after its equalization) , in particular after calculation of the distance function (for example via one of the above-mentioned correlation functions), are combined, preferably in that the mutually corresponding regions are superimposed. In other words, preferably the evaluation device in the first image and in the second image, which represent different views of the object, makes a search for identical points, and then places the two (equalized) images exactly at the same points on top of each other. This procedure optimally minimizes the error at the interfaces as soon as the image content provides sufficient information for this.

Thus, the generation of a composite image is preferably no longer exclusively based on the evaluation of (exact) position determination of the object to be inspected and comparison with a 3D model of the object to be inspected, but image information of at least one surface area (a group of several pixels) preferably used to create a composite image. In principle, such a production of a composite image could (even) be independent of a predetermined or determined from the recorded images whores the object is scanned. This procedure offers the advantage that information losses or faulty representations of an overlap area due to manufacturing tolerances of, for example, a container diameter or the container shape are at least reduced if not avoided.

In a further advantageous embodiment, the device has a (preferably downstream), preferably in the transport direction of the objects downstream of at least the first and second image pickup device, arranged, which is suitable and intended to capture an image content of the composite image and its correct To check representation. The analysis device preferably carries out a check or check of a barcode, a data matrix and / or an alphanumeric coding, for example (of an imprint), of a best before date. Preferably, more than two image recording devices are provided. The receiving region of the image recording devices preferably covers at least one (outer) surface region to be inspected, preferably a complete peripheral region of the object to be inspected. Preferably, a composite all-around image ("panorama image") of at least one complete peripheral region of the object is generated (comprising at least two, more preferably more than two) images , in particular at the respective time of image acquisition and particularly preferably at a point on the transport path, all have the same spacing.

In a further advantageous embodiment, at least four and preferably exactly four image recording devices are provided, which are preferably arranged at the time of the image acquisition substantially equidistantly to the object to be inspected, particularly preferably in a plane, and have different viewing angles. Preferably, the alignment of at least one and preferably each image pickup device with the (straight) transport direction (at least in sections) includes a 45 ° angle. Preferably, at least two, preferably at least three and particularly preferably at least four image recording devices are arranged at the same (vertical) height (vertical extension) of the object. But it is also possible, for example, to take six pictures receiving devices, preferably in a 60 ° grid, particularly preferably in a plane, (particularly preferably in two levels) may be arranged. Preferably, however, the six image recording devices can also be arranged at 45 ° -45 ° -90 ° -45 ° -45 ° -90 ° (in each case with respect to the transport path of the object to be inspected), preferably in one plane. Thus, the viewing angles of four image pickup devices each include a 45 ° angle with the transport path, while the viewing directions of two (preferably six) image pickup devices are perpendicular to the transport path and are arranged opposite each other (with respect to the transport path). Preferably, in each case three image recording devices are arranged on one side of the transport path, and particularly preferably two of these image recording devices arranged on one side of the transport path enclose a 45 ° angle with respect to the location on the transport path at which the object is inspected. Thus, it is also conceivable that the arranged within a plane image pickup devices are not distributed angularly equidistant, but that preferably around the transport path or in the vicinity of the transport path a greater distance is maintained to provide advantageous for the transported objects sufficient space. This can produce a higher resolution of the composite image. The limitation on the use of only four image recording devices has the advantage that only four images must be superimposed or combined, which advantageously results in a faster computing time.

Preferably, the orientation of an image pickup device is rotated by 90 ° with respect to an adjacent image pickup device. The individual image recording devices are thus preferably always arranged rotated by 90 ° relative to its predecessor. Therefore, the partial views of an object thus obtained or receivable are rotated by 90 ° in relation to each other. As a result, an overlap area is created in which corresponding points are searched.

The image recording devices are preferably cameras. It may be a color camera, but color detection is not required, so that black-and-white cameras may be used as image capture devices.

Preferably, a calibration device is provided which in each case has two different and preferably all of those involved in producing (exactly) a composite image. calibrated henen image recording devices. This calibration can be done prior to image acquisition and may include (same) adjustment or adjustment of exposure time, aperture, ISO settings and / or white balance or combinations thereof. From a calibration process or an evaluation, for example with a pattern of an object to be inspected, an overlapping area or a preselection of a reference area or a probable location of a corresponding area within the recorded images can also be forwarded to the evaluation device.

Just as many image recording devices (equidistantly) are provided (particularly preferably depending on the object to be inspected, for example of its geometric data) that the images taken by preferably two adjacent image recording devices each have an overlap fraction (ratio of the area of the overlap region to the total area of a recorded ( preferably equalized image) of at least 5%, preferably at least 10%, preferably at least 20% and more preferably at least 25% of the total area of a recorded image and a maximum degree of overlap of 40%, preferably 35% and particularly preferably 30%.

Preferably, at least one and preferably each image recording device has an objective, such as a 12 mm / fixed focal length video lens or an inspection range of up to 180 mm, or a 10 - 40 mm zoom lens. Preference is given to the use of conventional non-fisheye objectives which form a proportional representation of an object plane perpendicular to the optical axis. However, it would also be conceivable to use fish eye lenses. For the image recording device, in each case (particularly preferably a total of four) cameras, preferably of the type IP67 VGA, can be used, which are particularly preferably arranged on a backpanel, with preferably 1 master and 3 slaves being provided. In this case, one, preferably a quadruple, camera system with preferably four frame grabbers and preferably a 1.6 GHz CPU can preferably be provided on a special backpanel. However, a preferred quadruple camera system with preferably two (PXC) double scrapers on a CPU and preferably a normal backpanel is advantageous.

In a further advantageous embodiment, in each case at least two, preferably exactly four, image recording devices are arranged in at least two and preferably exactly two different (preferably parallel to each other, preferably spaced apart in the vertical or in the longitudinal direction of the object / container) and the output Value device preferably generates at least one composite image for each of these planes. Preferably, a composite image is therefore generated for (at least) two different height ranges of the object. Thus, for example, a lower peripheral region of an object and an upper peripheral region of an object can be examined separately from one another and a separate composite image can be generated in each case. By arranging two (independent) image recording device systems in different planes or heights, it is possible to inspect and monitor at the same time about two superimposed areas, for example two wrap-around labels arranged one above the other, of the object.

In a further advantageous embodiment, at least one illumination device is preferred, at least four, preferably exactly four illumination devices are preferred, and an illumination device is particularly preferably provided for each image acquisition device. Preferably, RGB lighting is provided from above, particularly preferably with four modular lamps. Preferably, the light of the illumination device is directed by a Fresnel lens, which is particularly preferred to the light sources, which advantageously serve for the optimal direction of the light and, with particular preference, LED lamps are arranged upstream. Preferably, four module lamps are thus provided, whose light is directed through an upstream Fresnel lens. Preferably, the at least one lighting device has an LED light source, preferably the light sources of the lighting devices are given by LEDs.

The device preferably has a (closed) housing in which the image recording devices are arranged and which advantageously serves to protect the image recording device (s) from ambient (scattering) light and / or other disturbing environmental influences such as spray water. Preferably, the housing (particularly preferably exclusively) has an inlet opening, through which the objects can be supplied, in particular by means of the transport device, and an outlet opening, through which the objects can be removed, in particular by means of the transport device. The housing may be closed except for the inlet and the outlet opening to all sides, but it can also only, except for the inlet and the outlet opening, be completed only laterally and upward, down to be open. Such a housing design may already be sufficient as an optical privacy shield or as an optical separation, since the objects to be inspected can preferably be optically shielded from below through the transport device within the housing and the device as a whole is thus more accessible. The at least one illumination device or the illumination devices (in each case assigned to the image recording devices) is preferably likewise arranged within the housing. Preferably, an improvement of the recognition by removing the black areas on the doors and / or the inlet and / or outlet walls can be achieved Preferably, the transport device is fixedly connected to a (camera) housing.

In a further advantageous embodiment, the device has a preferably arranged upstream in the transport direction of the objects labeling device which attaches to the objects at least one label and preferably at least two, preferably exactly two, preferably at least three and more preferably exactly three labels. Preferably, the labeling device is arranged immediately in front of the optical unit for inspecting the objects, i. immediately before the at least two image recording devices or in front of the housing, within which the image recording devices are arranged. Preferably, the evaluation device takes an assessment or a check of presence and / or completeness and / or the absence of labels or label parts and / or a (correct) arrangement and / or a (correct) position of the label or the labels relative to the Object (or the container) and / or the relative arrangement of the labels with respect to each other and / or a centricity and / or the rotational position of the labels and / or the accuracy or the presence of false and / or incorrectly inserted labels (coarse identity) and or an adhesive offset, in particular in the case of an all-round label or in the case of wrap-around labels and / or an MHD imprint control or a minimum durability imprint control, and the control of an individual marking, for example a 2-D code such as QR or Data Matrix.

The evaluation device preferably evaluates contrasts, ie in this way preferably all label imprints that stand out clearly from the background can be recognized. In the case of transparent labels, it is preferred to evaluate only the imprint but not the contour of the label. Preference is given to labels provided with a width of at least 60 mm, preferably of at least 70 mm, which are arranged on a cylindrical portion of an object to be inspected. The evaluation device preferably takes its evaluation or a distinction or check or verification or identification by a label brightness and / or different text positions and / or label positions. large and / or label shape and preferred for similar labels in particular by different text position. Particularly preferably, the evaluation device takes into account in its evaluation or checking or checking characters with font size which is at least 1 mm, preferably at least 1.3 mm high and / or lines with at least 0.1 mm, preferably with at least 0.2 mm line width. The evaluation device preferably takes into account a position inaccuracy of the (label) pressure. The evaluation device preferably makes an assignment of a neck or shoulder ring to a body label. In a further advantageous embodiment, the evaluation device checks for the presence of at least one element of the object to be inspected, in particular an element arranged thereon, and / or a positioning of this element relative to the object and / or relative to at least one further element of the object to be inspected, in particular an element arranged thereon. The element is preferably a label and / or a print applied to the object and / or another optically verifiable surface treatment of the object.

The labels applied or applied to the object may be metallized or non-metallized labels. Partially metallised labels are also possible, such as paper labels with metallized writing. Also conceivable are large-area metallized labels which preferably have a metallization of an area fraction of more than 30% based on the label size, in which, depending on the degree of gloss, preferably a patterning is provided or depending on the degree of gloss of a patterning need. Preferably, as labels, transparent labels, particularly preferably after patterning, can be used and optically checked or controlled. Preferably, the labels have a rectangular shape. Also conceivable are round and / or oval labels, which, however, are preferably subjected to sampling. Preferably, the object to be inspected, and preferably also the at least one element of the object to be inspected, does not have any negative cuts, such as gripping surfaces. The evaluation by the evaluation device preferably takes place in the case of so-called no-label-look labels according to or on the basis of a sampling.

In a further advantageous embodiment, the device has a, in particular optical, sensor device which is suitable and intended to a position of inspecting object relative to at least one image pickup device, and / or having a trigger device which is suitable and determined, in the presence of a predetermined position of the object to be inspected, in particular in the presence of a predetermined relative position of the object to at least one image pickup device imaging at least an image recording device and preferably all image recording devices with a time delay or initiate at the same time. Preferably, as a sensor device or as a trigger device, a light barrier, particularly preferably arranged within the housing, is provided. It is conceivable that the light barrier is arranged approximately on the illumination device or on a lamp. As soon as the object to be inspected or the container to be inspected is at a predetermined or predetermined pickup position along the transport path, the light barrier preferably triggers a signal which preferably triggers the picking up of an image of at least one image pickup device and preferably all image pickup devices at the same time.

In a further advantageous embodiment, at least one and preferably exactly one display device is provided for displaying the composite image. However, it is also conceivable that a display device is provided for displaying the composite image for each surface area to be checked, such as for each label attached or to be checked. This offers the advantage that at the same time the different labels can be inspected by the operator on different display devices. However, it is also possible to display several labels at the same time on a display device. Preferably, a touch display is provided as the display device. In a further advantageous embodiment, the device has at least one (main) memory device, preferably at least one flash memory, with at least 4 GB memory size. Preferably, the device may comprise a memory device in which the label form and / or the label type and / or the label design is stored and which can be accessed by the evaluation device. Furthermore, the evaluation device can preferably carry out a comparison of the composite image with the stored label design when checking or checking the labels. In addition, the dimensions of the object to be inspected, such as the container height and / or the container diameter, may preferably be stored in the storage device his. Preferably, at least one image recording device or all image recording devices or cameras also has an arithmetic unit.

In a further advantageous embodiment, the object to be inspected has a cylinder-like body, preferably a cylindrical body, preferably a cylindrical body with a conical element and / or it is particularly preferred for the object to be inspected to be a rotationally symmetrical object, preferably one Object without negative cuts such as grip surfaces and / or preferably a container and preferably a bottle. The object is preferably a container with a crooked neck, the checking of which is likewise possible by the proposed method or by the proposed device. The container preferably has a medallion in the shoulder region, which can serve for the evaluation device and / or analysis device for alignment control, preferably after a sampling. Preferably, the object to be inspected or the container has a minimum height of 50 mm; if it falls below this height, the object is preferably patterned, in order to be able to guarantee a fault-free operation. Preferably, the object to be inspected or the container has a maximum height of 350 mm.

The present invention is further directed to a method of optically inspecting objects with a transport device that transports the objects along a transport path, wherein a first image capturing device captures at least a first spatially resolved image from a first surface region of the object at a first viewing angle a second image pickup device captures at least one second spatially resolved image from a second surface area of the object at a second different viewing angle from the first, wherein the first surface area and the second surface area of the object coincide at least in an overlap area.

According to the invention, an evaluation device identifies a region of the second image corresponding to at least one region of the first image, in which the overlapping region is imaged at least in regions and preferably completely, and the evaluation device generates based on this identification at least from the first spatially resolved image and from the second spatially resolved image Bild a composite image, wherein the evaluation preferably a surface curvature of at least one Ab- Section of the object and / or caused by the image recording of an image pickup compression or distortion of at least one imaged region of the image taken into account in their evaluation. The method can be equipped with one or with all possible combinations of the features described in the context of the device as well as vice versa, the device with the features described below in the context of the method or all possible combinations thereof.

It is therefore also proposed in the context of the method according to the invention that the evaluation device no longer prefer the detail of an image preferably taken by a first image recording device and / or the detail of an image preferably taken by a second image recording device and / or the further images preferably taken by image recording devices only after a (exact) position determination (and / or determination of the geometric dimensions) of the object to be determined, but preferably uses image information of more than one of these images to determine an overlap area or to select the respective sections that are joined together and / or to be superimposed to create a composite image, which preferably a possible (angular and distance) faithful (linear) image of the surface to be controlled or inspected of the to give the object to be inspected.

Preferably, the (preferably different) image recording devices preferably simultaneously record (single) images, which preferably cover the entire surface area of the object to be inspected, are transmitted to the evaluation device.

In a first step, the evaluation device preferably in each case performs an equalization of these preferably the at least two, preferably the four (single) images. In this equalization process, a distortion or compression of the pixels caused by the image recording is preferably transformed back or out of account compared to the imaged points on the object surface. In this case, both the distance of the respective image recording device from the position of the object at the time of the respective image recording (recording position) and / or the distance of the respective image recording device from the respectively recorded surface area of the object at the time of the respective image recording and / or (preferably measured by the recording device or agreed and / or given) geometric data of the object to be inspected and / or a (preferably measured by the receiving device or determined and / or predetermined) surface curvature of the object to be inspected are taken into account. In this case, preferably no information is lost because the image is preferably processed linearly in each case and preferably every available original image pixel is present in the equalized image.

Preferably, in a preferably subsequent step, a more precise examination of the rectified images takes place at the interfaces or at the edge regions or at least one overlap region. These interfaces or edge regions can preferably already be predefined or preselected in advance, at least roughly, by a preceding calibration process, depending on the type or type of object to be inspected. Preferably, in a preferably subsequent step, the search for corresponding points in the at least one overlapping region takes place between at least two preferably equalized images. For this purpose, correlation functions are preferably used, which preferably assess a similarity of a pixel group of the at least one / first (equalized) image with respect to a (preferably equal) pixel group of the least second (equalized) image. Preferably, a particularly preferred square region of the first image, such as a window of size (2n + 1) x (2n + 1), where n is a natural, preferably predetermined number, is selected here. This output range or this reference range can already be predetermined for the evaluation device. However, it can also be that the reference range of the preceding object to be inspected is used as the output range. Preferably, a similar and preferably equally large area, a search area, of the second image is selected and this area or window in the second image, the search image, preferably along a line with the same coordinate, preferably by a pixel number d shifted. Preferably, this variation along a line with the same coordinate corresponds to a rotation of the object to be inspected about an axis perpendicular to its support surface on the transport device, which preferably coincides with a longitudinal axis of the object, or preferably this variation along a line with the same coordinate, that corresponds Change of the image pixel of the (equalized) second image, which would result from a rotation of the object about its longitudinal axis. In each case, the intensities or the gray values of the search field shifted by the number of pixels d are preferred. reichs in the second picture. In this comparison, one or more of the above-mentioned correlation functions or the above-mentioned methods of image analysis are preferably used. Preferably, the at least two image recording devices, particularly preferably the at least four and in particular the four image recording devices (preferably in each case) preferably take an image of the object from different viewpoints or angles or views at the same time. This group of recorded images preferably covers (completely) at least one surface area to be inspected. Preferably, an all-around image or a 360 ° image or a panorama image of at least one region of the outer surface of the object, which preferably extends completely along a circumference of the object, is to be created from these images.

The evaluation device preferably identifies the image to be inspected (even) for a plurality of images or preferably for just taken images, preferably for the images from the above-mentioned group of recorded images, preferably in pairs, for at least one region of the one image a corresponding region at least one other image in which in each case at least partially and preferably completely an overlap region is shown. For each image of the group, the evaluation device preferably carries out at least one, preferably exactly one, above-described identification process of corresponding regions with (exactly) one further image of this group. However, the evaluation device preferably carries out at least two, preferably exactly two, of the above-described identification processes of corresponding regions with at least two, preferably exactly two further images of this group. Preferably, the corresponding regions are at least in sections at least one (preferably lateral or in the circumferential direction) edge region of an image, preferably one (preferably lateral or circumferential) edge region of the two images involved. But it is also possible that it is an upper or a lower edge area (respectively).

Preferably, therefore, a complete development of an object or container is produced from four individual recordings and is preferably aligned with a label. The finished all-around image or the composite image or the panorama image is then displayed on a display device, preferably a touch display. The control or the evaluation device preferably evaluates contrasts, ie, it is possible to recognize preferably all label imprints that stand out clearly from the background. In the case of transparent labels, the imprint, but not the contour of the label, is preferably evaluated. An evaluation of offset and / or oblique position preferably takes place via (preferably predefined) measurement lines, along which preferably contrast jumps, for example from light to dark and / or from dark to light, are determined.

Further advantages and embodiments will be apparent from the attached drawings, in which:

Fig. 1 is a side view of an embodiment of a

device according to the invention; Fig. 2 is an illustration of the embodiment of an inventive

Device of Figure 1 in the transport direction.

3 shows an illustration of the embodiment of a device according to the invention of FIG. 1 and FIG. 2 in a plan view;

4 shows a representation of the embodiment of a device according to the invention from FIG. 1 to FIG. 3 in an oblique view;

Fig. 5 is an illustration of a composite image produced by a prior art apparatus;

Fig. 6 is an illustration of a label; and

7 shows a schematic procedure of an embodiment of a device according to the invention.

FIGS. 1 to 4 show an embodiment of a device 1 according to the invention for optically inspecting objects 10 in different views. In this embodiment, the objects 10 are containers with an attached to them. brought breast label 14 and body label 12, the correct arrangement on the container 10 through the device 1 should be checked or controlled. 1 shows a side view, FIG. 2 shows a side view in the transport direction of the objects 10, FIG. 3 shows a plan view and FIG. 4 shows an oblique view of the embodiment of a device 1 according to the invention.

The device 1 has a housing 30 mounted on (four) legs 36 and having a side wall 32 which may be formed by closeable safety doors 32 (which may be opened for inspection work) and a ceiling wall 34 which houses the optical unit located in the housing , Preferably, the image pickup devices 20, protects from ambient light and other disturbing environmental influences.

The housing 30 preferably has two, preferably on opposite sides (- walls) of the housing 30, arranged locks 38 with openings 37 in the housing 30 for the objects 10, of which a lock 38 as an inlet opening for feeding the containers 10 in the housing and to the arranged within the housing 30 image pickup devices 20 out and of which the other lock 38 serves as an outlet opening for discharging the (inspected) containers 10 from the housing 30. The contour of the inlet opening and preferably also that of the outlet opening can be adapted to the contour of the object 10 or of the container 10 or can be modeled such that as little ambient light as possible penetrates into the interior of the housing. Through the locks 38 into the interior of the housing 30 and out of the housing 30 also extends the transport device 40, which preferably transported the containers 10 individually and more preferably spaced from each other or with defined gaps. This can be configured as a conveyor belt. The transport device 40 preferably also has a transport section 39 in the interior of the housing 30, in which it preferably transports the containers 10 or objects 10 in a straight line along a transport path. In the interior of the housing 30, a further housing 39 (see FIG. 2) can also be provided, which visually separates a (further) area. The transport device 40 preferably transports the containers 10 to be inspected (on the upper edge 42 of the transport device 40).

For illuminating the areas of the container 10 to be inspected, it is preferably possible for the interior of the housing 30, particularly preferably at an upper area or at the dept. kkenwandung 34, a lighting device 50 and / or a light barrier or a trigger device to be arranged. The two latter elements are preferably provided and intended to detect the achievement of a certain predetermined position of the container 10 preferably in the housing 30 or preferably along the transport path and to initiate or trigger a preferably simultaneous image acquisition of the image recording devices 20. It can thus be achieved that the image recording devices or cameras make recordings of the container 10 at the same position and also the same rotational position or orientation of the container 10 even at high transport speeds and preferably without the need for interrupting the transport. This may be advantageous in particular because, during the transport of a container 10, its orientation or its rotational position may change.

The image recording devices 20 or cameras 20 are preferably arranged equidistantly and at equal angles. Typically, in a 360 ° -tab control of preferably rotationally symmetrical containers 10 preferably four imaging devices 20 or cameras 20 are provided or installed on a conveyor belt, which are preferably arranged offset in 90 ° to each other. It could also be used a different number of cameras 20 at equidistant angular intervals, for example, six cameras 20 in the 60 ° grid. These, preferably four, cameras 20 are preferably calibrated to one another and can also share a common coordinate system. The containers to be checked are preferably moved upright and / or spaced apart on a preferably single-lane transport device 40 by the detection unit, which is preferably arranged in the housing 30. When a container 10 is in the (preferably predetermined) pick-up position within recognition, preferably all four cameras 20 are triggered simultaneously and each camera 20 takes a picture of each side of the container 20.

In the prior art, the position of the container 10 within the common coordinate system is determined based on the contour of the container 10 of each camera 20 and based on this position, a composite image E or a panoramic image containing the complete view of the container 10 created. For this purpose, a previously created SD model of the container 10 is projected onto the position in the coordinate system and then a 3D-2D transformation is performed individually by each camera and the individual images are combined to form a panoramic image. In case of inaccurate position calculation, poor calibration tion, etc., however, information is displayed twice or not at the interfaces of the individual images.

Fig. 5 shows such an illustration of an erroneous composite image E produced by a prior art apparatus of a label 12 shown in Fig. 6, which has the label "Label" and a barcode 13 including the black bars 13a-13f. 5, the arrow P now symbolizes the location of the composite image E, where it is composed of two images taken by different cameras 20. However, due to inaccurate position calculation or even poor calibration, as is shown by comparison with the label 12 of FIG. 6 can be seen, at the symbolized by the arrow P interface a section of the label, which also contains the bar 13b and 13c of the bar code 13 in addition to a part of the letter "L". Due to the missing parts of the barcode 13b and 13c, it can not be determined, for example in a subsequent analysis device or an image evaluation device, whether the barcode can be read correctly or is printed correctly on the label.

FIG. 7 schematically illustrates a procedure of an embodiment of a device 1 according to the invention, for example that described in the context of FIGS. 1-4. Four image recording devices 20, which are preferably arranged in a plane, which in turn may extend parallel to the transport plane, preferably take (as triggered, for example) an image of a different recording position and thus of a different side of the object 10. Reference numeral 10 denotes a cylindrical object 10 with a longitudinal axis L, which is transported vertically in a direction perpendicular to its longitudinal axis L. Reference number 12 denotes a label 12 to be checked, which is arranged on the object 10 and extends peripherally around the complete circumference of the object 10. Shown in Fig. 7 is also a

Top view of the object 10, which is transported in the transport direction T along its transport path and is currently in a receiving position. In this, the image recording devices 20 are preferably arranged equidistant from the object 10. Preferably, the image pickup devices 20, which are involved or provided for creating (exactly) a composite image E, are arranged in a (horizontal) plane (in this case the drawing plane). The image recording devices 20 are preferably arranged at equidistant angular distances with different viewing angles W1 and W2 (measured with respect to a selected coordinate axis y). The recording directions or viewing angles of the image recording devices 20 preferably close with the transport direction T each have a 45 ° angle. The first image recording device 20 then picks up a first spatially resolved image B1 from a first surface region 01 of the object 10 at the first viewing angle W1. The second image recording device 20 receives a second spatially resolved image B2 from a second surface region 02 of the object 10. The two surface areas 01 and 02 overlap by the arrangement of the first and second image pickup device 20 about the common overlap region U12.

For evaluation by the evaluation device 60, an equalization of the (preferably four) individual images B1 and B2 is preferably first made (not shown). In this step, preferably no information is lost, because the original image is preferably processed linearly. This means that every existing original image pixel is present in the equalized image. Thus, inaccuracies are made only negligible errors. These rectified images B1 and B2 are then examined more precisely at the interfaces. Since the individual cameras 20 are preferably always rotated by 90 ° relative to their predecessor, one can also imagine the partial views rotated by 90 °. It can be seen that an overlapping region U12 was created by processing a larger angle rotated partial views. In this area or preferably its mapping, it is now necessary to find corresponding points. Corresponding means preferably that the point P1 in image B1 and the point P2 in image B2 belong to the same point Pw in the world (or on the object 10), ie preferably on the (actual) label 12. When searching for such points, preferably at least one correlation function or correlation functions are used. In this case, a point P1 is preferably taken from a reference image (for example the image B1) and surrounds it preferably with a window of size (2n + 1) ^* (2n + 1) (where n is a given or selected natural number and Size preferably describes the respective number of pixels in an x and a y direction). This window is then preferably moved in the search image along the line with the same coordinate. In order to calculate the similarity of two points, one preferably takes the gray values of the window in the reference image, the template, and preferably subtracts the gray values of the window shifted by a preferably predetermined or selected (pixel) number d in the search image. If one now sums all absolute differences, one obtains the simplest of the correlation functions, namely Summed Absolute Differences, abbreviated SAD. Another preferred method that gives greater weight to large deviations is Summed Squared Differences, SSD for short. Here, the diffe- then squares from the gray values of the template or the reference image or the image B1 and the gray values of the search window. The images are then merged after calculating the distance function. There are also additional methods and algorithms that search for the same points in two different images. Examples include optical flow, general Hough transformation, distance transformation functions such as TrueDistance and FastMarching, object detection with hair-like features, and so on.

The Applicant reserves the right to claim all features disclosed in the application documents as essential to the invention, provided that they are novel individually or in combination with respect to the prior art. It is further pointed out that features have also been described in the individual figures, which in themselves can be advantageous. The person skilled in the art immediately recognizes that a specific feature described in one figure can also be advantageous without the assumption of further features from this figure. Furthermore, the person skilled in the art recognizes that advantages can also result from a combination of several features shown in individual or in different figures.

LIST OF REFERENCE NUMBERS

1 device

10 object, (plastic) container

12 hull label

14 breast label

13 barcode

13a - 13f bars

20 image recording device, camera

30 housing

32 side wall, safety door

34 ceiling wall

36 legs

37 opening

38 lock 39 transport section

40 transport device

42 upper edge of the transport device

50 lighting device

60 evaluation device

01 first surface area

02 second surface area

W1 first viewing angle

W2 second viewing angle

B1 first picture

B2 second picture

U12 overlap area

E composite picture

P arrow

L longitudinal axis

Y coordinate axis

P1 point in first picture B1

P2 point in second picture B2

Pw point on object or container 10

Claims

claims

1 . Device (1) for optically inspecting objects (10) with a transport device (40), which transports the objects (10) along a transport path, with a first image recording device (20) that is suitable and intended for this, at least one first spatially resolved image (B1) from a first surface area (01) of the object (10) at a first viewing angle (W1), with at least one second image pickup device (20) being suitable and intended to receive at least one second spatially resolved image (B2) from one second surface area (02) of the object (10) at a second, different viewing angle (W2) from the first, wherein the first surface area (01) and the second surface area (02) of the object (10) at least in an overlapping area (U12) to match,

characterized in that

an evaluation device is provided which is suitable and intended to identify a region of the second image (B2) which corresponds to at least one region of the first image (B1), in which the overlap region (U12) is imaged at least regionally and preferably completely, and the evaluation device is suitable and intended to generate based on this identification at least from the first spatially resolved image (B1) and from the second spatially resolved image (B2) a composite image (E), wherein the evaluation means a surface curvature of at least a portion of the object (20) and / or a compression or distortion caused by the image recording of an image recording device (20) of at least one imaged region of the image (B1, B2) is taken into account in its evaluation.

2. Device (1) according to claim 1,

characterized in that

the device (1) has a separating device, which is arranged upstream of the objects in the transport direction of the objects with respect to the image recording devices (20) and which is directly subsequently transposed by the transport device (40). isolated objects and / or spaced and / or influences or adjusts a distance between at least two immediately consecutive objects.

Device (1) according to at least one of the preceding claims, characterized in that

the evaluation means a, in particular linear, transformation of the recorded pixels of at least one region of an image (B1, B2) recorded by an image recording device (20), particularly preferably of at least one region of the first image (B1) and / or of the second image ( B1) taking into account a surface curvature and / or a distortion caused by a 3D-2D projection.

4. Device (1) according to at least one of the preceding claims,

characterized in that

the evaluation device performs an identification of mutually corresponding areas on the pixel level and / or identifies mutually corresponding pixels of at least two images (B1, B2). 5. Device (1) according to at least one of the preceding claims,

characterized in that

the evaluation device for identifying mutually corresponding regions of two recorded images (B1, B2) takes into account only a predetermined region of a captured image (B1, B2) and preferably only a predetermined region, which is preferably less than 20%, preferably less than 15%, preferred less than 10%, preferably less than 5%, and more preferably less than 1% of the total area of the recorded image occupies.

the evaluation device uses at least one evaluation method for identifying mutually corresponding regions of two recorded images (B1, B2), which is selected from a group that can analyze the contrasts, analyze includes skew and skew, measurement lines, edge filters, Laplacian filters, correlation functions, and combinations thereof.

7. Device (1) according to at least one of the preceding claims,

characterized in that

at least four image recording devices (20) are provided, which are preferably arranged at the time of image acquisition substantially equidistantly from the object to be inspected, particularly preferably in one plane, and / or at least two, preferably exactly four, image recording devices (20) in at least two and preferably exactly two different planes are arranged and the evaluation device generates at least one composite image (E) for each of these planes.

8. Device (1) according to at least one of the preceding claims,

characterized in that

at least one illumination device, preferably four illumination devices and particularly preferably one illumination device is provided for each image acquisition device (20).

9. Device (1) according to at least one of the preceding claims,

characterized in that

the device (1) has a labeling device arranged upstream, preferably in the transport direction of the objects, which attaches to the objects at least one label and preferably at least two labels.

10. Device (1) according to at least one of the preceding claims,

characterized in that

the evaluation device has a presence of at least one element of the object to be inspected (10), in particular of an element arranged thereon, and / or a positioning of this element relative to the object (10) and / or relative to at least one further element of the object to be inspected (10 ), in particular an element arranged thereon.

1 1. Device (1) according to at least one of the preceding claims, characterized in that

the device (1) has a, in particular optical, sensor device which is suitable and intended to detect a position of the object to be inspected (10) relative to at least one image recording device (20), and / or has a triggering device which is suitable and is determined, in the presence of a predetermined position of the object to be inspected (10), in particular in the presence of a predetermined relative position of the object (10) to at least one image pickup device (20) an image recording at least one image pickup device (20) and preferably all image pickup devices (20) with a time delay or initiate at the same time.

12. Device (1) according to at least one of the preceding claims,

characterized in that

the object to be inspected has a cylinder-like body, preferably a cylindrical body, preferably a cylindrical body with a conical element, and / or particularly preferably the object to be inspected is a container and preferably a bottle.

13. Device (1) according to at least one of the preceding claims,

characterized in that

the device (1) has an analysis device, preferably arranged in the transport direction of the objects (10) downstream of at least the first and second image recording device (20), which is suitable and intended to capture and record an image content of the composite image (E) to check its correct representation.

14. A method for optically inspecting objects (10) with a transport device (40) which transports the objects (10) along a transport path, wherein a first image recording device (20) comprises at least a first spatially resolved image (B1) from a first surface region (01 ) of the object (10) at a first viewing angle (W1), at least one second image pickup device (20) at least a second spatially resolved image (B2) from a second surface area (02) of the object (10) at a second, from the first ver - different viewing angle (W2), the first surface area (01) and the second surface area (02) of the object (10) coinciding at least in an overlapping area (U12),

characterized in that

an evaluation device identifies a region of the second image (B2) which corresponds to at least one region of the first image (B1), in which the overlapping region (U 12) is imaged at least regionally and preferably completely, and the evaluation device generates at least based on this identification the first spatially resolved image (B1) and from the second spatially resolved image (B2) a composite image (E), wherein the evaluation device a surface curvature of at least a portion of the object (20) and / or caused by the image recording of an image pickup device (20) compression or distortion of at least one imaged region of the image (B1, B2) is taken into account in its evaluation.