EP3211611A1 - Method for the computer-assisted detection of an empty transport container and device for computer-assisted detection of an empty transport container - Google Patents

Abstract

According to various embodiments, a method (100) for computer-aided idle recognition of a transport container (400) may comprise: acquiring (101) image data of a region of the transport container (400); Determining (103) a contour pattern representing the transport container (400) using the image data; Determining (105) a deviation amount representing a deviation of the contour pattern from at least one reference pattern, the at least one reference pattern representing an empty transport container (400); Outputting (107) a signal if the deviation quantity satisfies a predetermined criterion.

Description

The invention relates to a method for computer-aided empty detection of a transport container and a device for computer-aided empty recognition of a transport container.

In general, shipping containers can be used to transport items, e.g. Goods in the field of production or sale. In doing so, it may be necessary to detect whether and, optionally, what is in the transport container, e.g. when registering goods at a cashier's exit (this can also be referred to as "bottom of basket" recognition - BoB). Thus, costs can be reduced, which arise when unregistered goods pass the cash register unrecognized (this can also be referred to as loss prevention).

Conventionally, computer-aided methods of pattern recognition are used to identify objects placed in the transport container. In the process, distinctive patterns of the objects are recognized and compared with a database in which the patterns of known objects, e.g. Goods are stored.

Matching the patterns of items contained in the shipping container with the database and the size of the database may require considerable effort, especially if a large number of items are stored in the database with their respective patterns and / or similar ones Objects are to be detected. For example, it may be necessary to store a record for each item according to different views of the item, for example if not It can be ensured that the object is always in the same orientation. Mutually similar objects may require that a greater number of features be aligned to allow for reliable identification.

As the size of the database increases, i. As the number of records increases and / or the number of records per record increases, the amount of memory required and the computational power required or data analysis speed required can increase to ensure timely effective recognition of the items, e.g. Real time. Furthermore, as the volume of the database increases, the effort required to maintain the data sets (data care requirements) and thus the personnel required can increase in order to prevent an uncontrolled increase in the size of the database due to outdated data records. Therefore, these computer-aided pattern recognition systems can create a significant cost of acquisition and / or entertainment, especially if e.g. one system per cash register outlet is required. Also, the pattern recognition process for a very large number of objects to be discriminated is very complicated, and the pattern recognition error susceptibility also increases with increasing number of similar patterns.

Furthermore, conventional detection of the transport container contents may be limited and / or inaccurate, for example, in the case of flat objects (eg, having a small cross-section in one orientation); little or not highlighted and / or colored objects, eg with only little information, color and / or texture content (for example a nearly homogeneous layout such as black, white, gray, etc.); and / or transparent objects. Furthermore, items not stored in the database are not regularly recognized or misrecognized.

According to various embodiments, a method for computer-aided idle recognition of a transport container (ie illustrative of whether the transport container is empty, in other words, free of objects contained in the transport container) and a device for computer-aided empty detection of the transport container is provided, which have lower complexity, whereby the computational effort and storage costs can be reduced. This makes it possible to use simpler equipment, so that the cost of purchase can be reduced. Furthermore, the size of the database can be reduced, so that the effort to maintain and maintain the records can be reduced, which can lead to a cost savings.

Illustratively, in accordance with various embodiments, a method for computer-aided empty detection of a transport container and a device for computer-aided empty identification of the transport container are provided, which pattern deviations of one or more patterns, which from one or more captured by, for example, a camera images of (possibly with one or more objects filled) transport container of one or more stored patterns of the transport container in an empty state (ie without objects in the transport container) determine.

The pattern deviations are caused for example by objects in the transport container, ie they arise or become larger when the transport container is not empty. Illustratively, a captured image or a plurality of recorded images (more precisely features extracted from the image or images) of the transport container to be examined with one or more reference transport container images (more precisely with reference transport container features, which consist of one or more reference transport containers Images were extracted, for example, in a previous training method), one or more reference transport containers that are empty, and based on the pattern deviations, it is decided whether the transport container to be inspected is empty or not.

According to various embodiments, it is no longer necessary to use a database in which each possible object to be detected is stored, but the database can be reduced in scope to the one or more transport containers used. Thus, the size of the database no longer increases with the number and / or variation of items that could be transported by the shipping container and that can be recognized.

Further, according to various embodiments, depth information is obtained by means of three-dimensional (3D) image capture. The depth information may be used, for example, to detect if an item is in different areas of the shipping container, e.g. on a lower and / or upper level. Thus, it can be distinguished in which area of the transport container is an object. For example, the depth information can be obtained through one of the areas, so that it can be clearly seen from above whether something is located in a lower area of the transport container or in an area below the transport container. Thus, the number of required image sensing sensors can be reduced, since the transport container can be fully examined from one and the same perspective.

According to various embodiments, a system for computer-aided analysis of items (objects) in a (eg open) transport container (may also be referred to as a transport medium) is provided. By means of the system it will enable the state of the Transport container (eg empty, or not empty) at a given time and / or location, eg automated to determine.

According to various embodiments, a method for computer-aided idle recognition of a transport container may comprise: acquiring image data of a region of the transport container; Determining a contour pattern (e.g., by feature recognition) representing the transport container (e.g., caused by the transport container) using the image data; Determining a deviation quantity which represents a deviation of the contour pattern from at least one reference pattern, wherein the at least one reference pattern represents an empty transport container; Outputting a signal if the deviation quantity meets a predetermined criterion.

According to various embodiments, the method may further comprise: acquiring (e.g., additional) image data of an additional area of the transport container; Determining an additional contour pattern representing the additional area (e.g., caused by the additional area) based on the (e.g., additional) image data; wherein a reference pattern of the at least one reference pattern has the additional contour pattern.

According to various embodiments, the method may further comprise: storing the additional contour pattern on a data memory.

According to various embodiments, the at least one reference pattern may be part of or form a database.

According to various embodiments, the at least one reference pattern or the database may have a plurality of reference patterns (eg two or more than two, eg three, four, five, six, seven, eight, nine, ten or more than ten, eg twenty, fifty, eg one hundred or more than a hundred).

According to various embodiments, the method may further comprise: reading a reference pattern of the at least one reference pattern from a data memory (e.g., from the database).

According to various embodiments, the transport container for capturing the image data may be arranged in an image capture area, wherein the image capture area defines an image background (illustrative of the transport bin); wherein a reference pattern of the at least one reference pattern has a contour pattern representing an image background (e.g., when none or an empty transport container is disposed in the image capturing area). Illustratively, the transport container can be arranged between an image acquisition system, with which the image data are acquired, and the image background.

According to various embodiments, the deviation quantity may represent a contrast deviation of the contour pattern from the at least one reference pattern.

According to various embodiments, the deviation amount may represent a contour deviation of the contour pattern from the at least one reference pattern.

According to various embodiments, the deviation quantity may represent a surface occupation deviation of the contour pattern from the at least one reference pattern.

According to various embodiments, the method may further comprise: detecting when the transport container is in an image capture area is arranged, in which the capture of the image data takes place; wherein the image data is acquired when it has been detected that the transport container is located in the image capture area (eg, during a search phase).

The search phase may include: detecting reflections (e.g., in the IR region); and comparing (e.g., feature-based) the reflections to the reference pattern (e.g., with the features from the reference image of the transport medium).

According to various embodiments, the method may further comprise: detecting whether the transport container is located in an image capture area in which the capture of the image data occurs; Outputting another signal (may also be referred to as unrecognized signal) when no transport container is arranged in the image capture area.

According to various embodiments, the method may further comprise: detecting whether the transport container is a container type of a plurality of container types; and outputting another signal (may also be referred to as unrecognized signal) if no container type of the plurality of container types has been detected.

According to various embodiments, the predetermined criterion may represent an empty transport container and the signal may comprise an empty detected signal.

According to various embodiments, the predetermined criterion may represent a non-empty transport container and the signal may comprise a non-empty recognized signal.

According to various embodiments, the predetermined criterion may represent an error range in which a Empty detection is unreliable, and the signal may have or represent an error signal.

In accordance with various embodiments, outputting the signal may include issuing a prompt, with a POS system process transitioning to a wait state until an input to the prompt occurs.

According to various embodiments, outputting the signal may include issuing a prompt, the method further comprising: updating the at least one reference pattern based on the contour pattern when an input to the prompt represents an empty state of the transport container.

According to various embodiments, outputting the signal may include issuing a prompt, the method further comprising: forming an additional reference pattern based on the contour pattern when an input to the prompt represents an empty state of the transport container; and adding the additional reference pattern to the at least one reference pattern (e.g., to the database).

According to various embodiments, acquiring the image data of the transportation container may include: acquiring image data of an image acquisition area in which the transportation container is disposed; Determining a portion of the image data of the image capture area (may also be referred to as a "region of interest" ROI or analysis area) having the image data of the area of the transport bin.

According to various embodiments, the determination of a part of the image data can take place using one or more markers of the transport container. Clearly, one or more markers can be used to using customized detection algorithms to limit the ROI.

According to various embodiments, the transport container may include one or more markers (e.g., gray scale markers (e.g., ArUco) and / or reflection markers).

According to various embodiments, acquiring the image data of the transportation container may include: acquiring image data of an image acquisition area in which the transportation container is disposed; wherein determining the contour pattern comprises: determining a portion of the image data of the image capture area representing an image background defined by the image capture area; Comparing the portion of the image data with reference image data, the reference image data representing an empty image capture area (may also be referred to as an image-background comparison). Illustratively, the contour pattern can be determined by determining which part of the image background is obscured by the transport container (e.g., a grid structure).

According to various embodiments, the determination of the contour pattern may comprise: determining a portion of the image data (e.g., an interest region) having a predetermined contour density, determining the contour pattern using the portion of the image data having the predetermined contour density. Clearly, a feature-intensive region of the transport container can be used for empty recognition.

According to various embodiments, acquiring the image data may include: successively acquiring a plurality of image data; and determining image data of the plurality of image data representing the transport container and / or having a predetermined contour density. Illustratively, a series recording can take place wherein the images are used for empty recognition, showing the transport container. In this way, empty recognition can be facilitated if the transport container does not remain in the image capture area and / or is not always positioned immediately in the image capture area.

According to various embodiments, acquiring the image data of the transportation container may include: acquiring image data of an image acquisition area in which the transportation container is disposed; Determining a spatial location (position and / or orientation) of the transport container relative to the image capture area; Determining a portion of the image data (e.g., an area of interest) of the image capture area that has the image data of the area based on the spatial location (position and / or orientation).

According to various embodiments, the method may further comprise: detecting a container type of the transport container, wherein the transport container is a container type of a plurality of container types; and wherein the reference pattern represents an empty container of the container type.

According to various embodiments, the method may further include: selecting the reference pattern from a plurality of reference patterns, each reference pattern associated with a bin type of the plurality of bin types.

According to various embodiments, the method may further comprise: detecting a container type of the transport container, the transport container being a container type of a plurality of container types; where the criterion represents the container type.

According to various embodiments, the method may further comprise: selecting the criterion from a plurality of criteria, each criterion associated with a bin type of the plurality of bin types.

According to various embodiments, the method may further comprise: determining color information (e.g., the transport container) based on the image data; wherein the deviation amount further represents a deviation of the color information from reference color information, wherein the reference color information represents an empty transportation container.

According to various embodiments, the determination of the deviation quantity may include: weighting the deviation of the color information and the deviation of the contour pattern according to a predetermined weight characteristic.

According to various embodiments, the method may further comprise: determining topography information (e.g., the transport container) based on the image data; wherein the deviation amount further represents a deviation of the topography information from reference topography information, wherein the reference topography information represents an empty transportation container.

According to various embodiments, determining the deviation quantity may include: weighting the deviation of the topography information and the deviation of the contour pattern according to a predetermined weight characteristic.

According to various embodiments, the method may further comprise: determining depth information based on the image data; wherein the deviation quantity further represents a deviation of the depth information from reference depth information, wherein the reference depth information represent an empty transport container.

According to various embodiments, determining the deviation quantity may include: weighting the deviation of the depth information and the deviation of the contour pattern according to a predetermined weight characteristic.

According to various embodiments, determining the depth information may include capturing image information of an additional region of the transport container through the region and / or through a surface element of the transport container (which is disposed, for example, between the first region and the second region).

According to various embodiments, the method may further comprise: determining a first part of the image data of the transport container, which represents a first surface element of the transport container; and determining a second part of the image data of the transport container, which represents a second surface element of the transport container, wherein the first surface element and the second surface element at an angle to each other and / or are arranged at a distance from each other; wherein the second part of the image data is detected through the first surface element; and wherein the contour pattern represents the first surface element and / or the second surface element. Clearly, it can be distinguished according to various embodiments, which area of the transport container is examined, e.g. recognizing the surface element separating the regions, and / or e.g. by recognizing the surface elements that bound the areas.

According to various embodiments, the contour pattern may represent at least one of the following: a contour pattern of the image background (eg, a contour pattern of the image Soil structure and / or image background structure); a lattice structure of the transport container and / or a surface element of the transport container; one or more struts of the transport container and / or a surface element of the transport container; an advertising medium of the transport container and / or a surface element of the transport container; a disturbance caused by an article when the article is disposed in the transport container; and / or an image background defined by an image capture area in which the capture of the image data occurs.

According to various embodiments, the transport container may be a shopping cart (e.g., a compact shopping cart) or a shopping cart.

According to various embodiments, the transport container may have a first region (e.g., a first transport region) and a second region (e.g., a second transport region) with the first region and the second region separated by at least one surface element of the shipping container.

According to various embodiments, a surface element may comprise at least one of: a lattice structure; one or more struts; and / or an advertising medium.

According to various embodiments, the transport container may comprise a chassis. The chassis may be at least three, e.g. at least four wheels (e.g., self steering Castor wheels).

The chassis may provide two translational degrees of freedom, and optionally a rotational degree of freedom along which the shipping container is movable, according to various embodiments.

According to various embodiments, a first degree of translational freedom of the two translational degrees of freedom and a second translational degree of freedom of the two translational degrees of freedom may be different, e.g. perpendicular to each other. For example, the two translational degrees of freedom may run along a surface defined by the ground, e.g. horizontal.

According to various embodiments, the transport container may comprise plastic and / or a metal, e.g. Aluminum and / or steel.

According to various embodiments, the second area may be disposed between the first area and the chassis.

According to various embodiments, the first region may have a pivotable surface element which is set up in such a way that at least two transport containers can be pushed into one another.

According to various embodiments, the transport container may have a handle strip and / or a handle. The handle may for example be pivotally mounted.

According to various embodiments, the transport container may have an opening on a first side and a surface element (eg, a bottom or a side wall) on a second side opposite the first side, wherein a cross-sectional area of the surface element is smaller than a cross-sectional area of the opening parallel thereto; so that the surface element fits into the opening. Illustratively, a plurality of transport containers can be arranged for nesting.

According to various embodiments, an apparatus for computer-aided emptying of a transport container may comprise: an image acquisition optical system for acquiring image data; a data memory for storing at least one reference pattern and / or in which at least one reference pattern is stored, the reference pattern representing an empty transport container; a processor configured to perform the method of: acquiring image data of a region of the transportation container; Determining a contour pattern representing the transport container (e.g., caused by the transport container) using the image data; Determining a deviation quantity which represents a deviation of the contour pattern from at least one reference pattern, wherein the at least one reference pattern represents an empty transport container; Outputting a signal if the deviation quantity meets a predetermined criterion.

According to various embodiments, the image acquisition optical system may define an image capture area in which the capture of image data occurs; wherein the transport container for capturing the image data is or is arranged in the image capture area, the image capture area defining an image background of the transport container; and wherein a reference pattern of the at least one reference pattern has a contour pattern of the image background (e.g., when no and / or an empty transport container is disposed in the image capture area).

The processor may be further configured to perform one of the methods described herein, eg, as described above.

According to various embodiments, the image capture system may include at least two (optical) image capture sensors (e.g., provided by one or two cameras) for acquiring depth information (e.g., stereoscopically). In order to obtain the depth information, the image data acquired by means of two image sensing sensors can be superposed on each other, e.g. taking into account a relative spatial position (position and / or orientation) of the two image sensing sensors to each other.

Alternatively or additionally, the image acquisition system can have at least one plenoptic camera (which can also be referred to as a light field camera) for acquiring depth information.

Alternatively or additionally, the image capture system may include a projector configured to project an optical pattern into the image capture area, and having an image capture sensor configured to acquire the optical pattern for acquiring depth information.

A camera may have an (optical) image sensing sensor and at least one lens array associated with the image sensing sensor. The lens arrangement of a plenoptic camera may comprise a grid of several microlenses.

An image sensing sensor (may also be referred to as an image sensor or optical sensor) may include a plurality of photoelectrically active regions (may also be referred to as pixels) which generate an electrical signal in response to electromagnetic radiation (eg, light, eg, visible light) / or modify. The image acquisition sensor can be, for example, a CCD sensor (charge-coupled device sensor) and / or a active pixel sensor (may also be referred to as a CMOS sensor) or be formed therefrom.

According to various embodiments, an image capture sensor may be configured to be wavelength sensitive (e.g., to capture color information and / or capture a pattern projected into the image capture area).

According to various embodiments, the processor may be configured to determine a contrast deviation of the contour pattern for determining the deviation quantity.

According to various embodiments, the processor may be configured to determine a deviation of the area of coverage of the contour pattern to determine the deviation quantity.

According to various embodiments, the processor may be configured to determine a contour deviation of the contour pattern to determine the deviation quantity.

According to various embodiments, the data store may be a non-volatile data store. For example, the data store may include or be formed from a hard disk and / or at least one semiconductor memory (such as read only memory, random access memory, and / or flash memory). The read only memory may be, for example, an erasable programmable read only memory (may also be referred to as an EPROM). The random access memory may be a nonvolatile random access memory (may also be referred to as NVRAM - non-volatile random access memory).

According to various embodiments, the device may further comprise a cash register system (eg digital), which is set up to carry out one or more cash register processes. For example, a POS system process a calculation process, an inventory process, and / or a registration process.

The POS system may include at least one of: a screen (may also be referred to as a primary screen, eg, a touch screen), a printer (eg, for printing an invoice and / or a label), a scanner (eg, a bar code scanner) for registration of items, a cash drawer, a (eg programmable) cash register keypad (may also be part of the touch-sensitive screen), an electronic payment terminal (may also be referred to as an EC terminal, "EC" - electronic cash, eg for reading a debit card and or a credit card), and / or an additional screen (may also be referred to as a secondary screen), a signal output (may also be part of the screen), an operator location (where a user operating the POS system may be located ).

On the primary screen, information representing a state of a cash register process can be displayed. On the secondary screen, part of the information displayed on the primary screen can be displayed.

According to various embodiments, the signal may be output by means of the signal output, e.g. by issuing a prompt via the screen and / or e.g. by emitting an acoustic signal by means of an acoustic signal output.

The prompt may be issued, for example, such that a continuation of the registration process is only possible if an input has been made to the command prompt.

According to various embodiments, a database (may also be referred to as a reference database) having one or more data sets may be stored in the data store. Each data record can have a reference pattern. Furthermore, each data record may comprise at least one of the following information: reference depth information; Reference topography information; Reference color information; a spatial location (e.g., position and / or orientation of a shipping bin) associated with the reference pattern; a weight characteristic; and / or a type of transport container associated with the reference pattern.

Alternatively, the detection and / or detection may be done using an algorithm that is invariant to translation and / or rotation. For example, an algorithm for fast and robust recognition of image features, a so-called "robust feature recognition algorithm" (SURF), can be used. For example, the algorithm used for feature-based detection may be rotationally invariant, so that the detection of the transport container may be independent of its orientation. Then it can be dispensed with information on the spatial location of the transport container.

According to various embodiments, the image capture system may be oriented such that the image capture area (or image background) has a background (e.g., a hall floor).

According to various embodiments, a method for computer-aided idle recognition of a transport container may comprise: acquiring image data of an image acquisition area in which a transport container the image capture area defines an image background; Determining a contour pattern representing the transport container and / or the image background using the image data; Determining a deviation quantity which represents at least one deviation of the contour pattern from at least one reference pattern, wherein the at least one reference pattern represents an empty transport container; Outputting a signal if the deviation quantity meets a predetermined criterion. Clearly it can be determined whether a contour of the image background and / or the transport container is covered by an object and / or disturbed.

For example, the topography information may represent a three-dimensional profile (e.g., an area in spatial space). For example, based on the depth information, a spatial location (e.g., position, orientation, and / or distance to a reference point) in the location space may be determined. On the basis of the color information, for example, a color histogram and / or color spectrum can be determined. The deviation quantity may, for example, represent a deviation of the three-dimensional profile, the spatial position, the color spectrum and / or the color histogram from corresponding reference data.

According to various embodiments, the criterion may represent a threshold. For example, the empty detected signal may be output if the deviation amount is none than the threshold value. The non-empty detected signal may be output, for example, when the deviation amount is larger than the threshold value.

Embodiments of the invention are illustrated in the figures and are explained in more detail below.

Figur 1

ein Verfahren gemäß verschiedenen Ausführungsformen in einem schematischen Ablaufdiagram;

Figur 2

eine Vorrichtung gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 3

eine Vorrichtung gemäß verschiedenen Ausführungsformen in einer schematischen Perspektivansicht;

Figur 4

einen Transportbehälter gemäß verschiedenen Ausführungsformen in einer schematischen Seitenansicht oder Querschnittsansicht;

Figur 5

einen Transportbehälter gemäß verschiedenen Ausführungsformen in einer schematischen Seitenansicht oder Querschnittsansicht;

Figur 6

ein Verfahren gemäß verschiedenen Ausführungsformen in einem schematischen Ablaufdiagram;

Figur 7

ein Verfahren gemäß verschiedenen Ausführungsformen in einem schematischen Ablaufdiagram;

Figur 8

ein Verfahren gemäß verschiedenen Ausführungsformen in einem schematischen Ablaufdiagram;

Figur 9

eine Vorrichtung gemäß verschiedenen Ausführungsformen in einer schematischen Perspektivansicht;

Figur 10

eine Vorrichtung gemäß verschiedenen Ausführungsformen in einer schematischen Perspektivansicht;

Figur 11

ein Bilderfassungssystem gemäß verschiedenen Ausführungsformen in einer schematischen Perspektivansicht;

Figur 12

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 13

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 14

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 15

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 16

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 17

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 18

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 19

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht;

Figur 20

ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht; und

Figuren 21 bis 37

jeweils ein Verfahren gemäß verschiedenen Ausführungsformen in einer schematischen Ansicht.

Show it

FIG. 1

a method according to various embodiments in a schematic flow diagram;

FIG. 2

a device according to various embodiments in a schematic view;

FIG. 3

a device according to various embodiments in a schematic perspective view;

FIG. 4

a transport container according to various embodiments in a schematic side view or cross-sectional view;

FIG. 5

a transport container according to various embodiments in a schematic side view or cross-sectional view;

FIG. 6

a method according to various embodiments in a schematic flow diagram;

FIG. 7

a method according to various embodiments in a schematic flow diagram;

FIG. 8

a method according to various embodiments in a schematic flow diagram;

FIG. 9

a device according to various embodiments in a schematic perspective view;

FIG. 10

a device according to various embodiments in a schematic perspective view;

FIG. 11

an image acquisition system according to various embodiments in a schematic perspective view;

FIG. 12

a method according to various embodiments in a schematic view;

FIG. 13

a method according to various embodiments in a schematic view;

FIG. 14

a method according to various embodiments in a schematic view;

FIG. 15

a method according to various embodiments in a schematic view;

FIG. 16

a method according to various embodiments in a schematic view;

FIG. 17

a method according to various embodiments in a schematic view;

FIG. 18

a method according to various embodiments in a schematic view;

FIG. 19

a method according to various embodiments in a schematic view;

FIG. 20

a method according to various embodiments in a schematic view; and

FIGS. 21 to 37

each a method according to various embodiments in a schematic view.

In the following detailed description, reference is made to the accompanying drawings, which form a part hereof and in which is by way of illustration specific Embodiments are shown in which the invention can be practiced. In this regard, directional terminology such as "top", "bottom", "front", "back", "front", "rear", etc. is used with reference to the orientation of the described figure (s). Because components of embodiments can be positioned in a number of different orientations, the directional terminology is illustrative and is in no way limiting. It should be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. It should be understood that the features of the various exemplary embodiments described herein may be combined with each other unless specifically stated otherwise. The following detailed description is therefore not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

As used herein, the terms "connected," "connected," and "coupled" are used to describe both direct and indirect connection, direct or indirect connection, and direct or indirect coupling. In the figures, identical or similar elements are provided with identical reference numerals, as appropriate.

According to various embodiments, the transport container may comprise or be formed from a shopping cart, a shopping basket or another cart (and / or basket).

According to various embodiments, a recognition of remaining objects inside the transport container, for example on a lower box shelf of the shopping cart, can be made possible.

According to various embodiments, recognizing whether or not one of the following items (eg, goods may also be referred to as articles) is disposed in the transport container can be improved and / or enabled (eg, not via color histograms or depth information) can be reliably determined): a flat object (which, for example, restrict a 3D detection), a barely or not highlighted and / or colored object (eg, with only little information, color and / or texture content), a monochrome and / or homogeneous object (for example, almost homogeneous layout such as black, white, gray, etc.), a transparent object.

Detecting the article may be based, in various embodiments, on the article interfering with (or interrupting) the contour pattern (eg, a grid contour) of the shipping container and / or the image background and / or a contrast to the shipping container and / or image background ( eg underground and / or floor) causes.

According to various embodiments, computer-aided recognition of objects in an open transport container is provided. Thus, the state of the transport container (empty state, non-empty state and / or full state) can be recognized at a certain time and / or place, e.g. automated.

According to various embodiments, an image capture system (e.g., Intel RealSense F200, INTEL R200, and / or Intel SR300) may include a 2D image capture sensor and / or a 3D image capture sensor.

The processor may or may not be provided by means of an electronic (programmable) data processing system. The electronic data processing system may further comprise the data memory. For example, the electronic data processing system may comprise or be formed from a microcomputer, for example a personal computer system or a digital cash register system.

The processor may be coupled to the image acquisition system via a USB (Universal Serial Bus Interface) interface, e.g. via USB 3.0.

According to various embodiments, the data processing system may include at least one co-processor (e.g., provided by a co-processor card or external graphics card), e.g. a graphics processor (GPU). By means of the co-processor, an acceleration of the computationally intensive processes can be achieved by outsourcing them to the co-processor.

Fig.1 illustrates a method 100 according to various embodiments in a schematic flow diagram.

According to various embodiments, the method 100 in FIG. 101 may include acquiring image data of a region of the transport container. The method 100 may further include in 103: determining a contour pattern representing the transport container (eg, which is effected by the transport container) using the image data. 105, the method 100 may further include: determining a deviation quantity representing a deviation of the contour pattern from at least one reference pattern, wherein the at least one reference pattern represents an empty transport container. The method 100 may further include in 107: outputting a signal if the deviation quantity satisfies a predetermined criterion. Fig.2 illustrates a device 200 according to various embodiments in a schematic view.

The device 200 may include an optical imaging system 202, a data storage 204, and a processor 206. The processor 206 may be coupled to the optical imaging system 202 and the data memory 204, e.g. by means of a data line (i.e., so that it can be transferred between these data).

For example, the processor 206 may be configured to acquire image data by means of the optical image capture system 202. For example, the processor 206 may control the imaging system 202.

For example, the image data may be stored in a data storage 202d (may also be referred to as a cache 202d) of the image capture system 202. The image data may be detected by means of an optical sensor 202s (may also be referred to as image capture sensor 202s). At least processing of the image data by means of the processor 206, these can be read from the buffer 202d.

The optical sensor 202s may include a photoelectrically active surface 202o directed toward an image sensing area 208, e.g. in direction 208b (may also be referred to as image capture direction 208b).

Figure 3 illustrates a device 300 according to various embodiments in a schematic perspective view.

According to various embodiments, a transport container 400 may be provided in the image capture area 208 be arranged or, for example, a shopping cart, as in Figure 3 is illustrated. The transport container 400 may have a chassis 312 with a plurality of rotatably mounted wheels 312r and a transport frame 322. The transport frame 322 may be movably supported by means of the chassis 312. The transport rack 322 may provide at least one transport area 322a.

The transport container 400 may include a handle 302g by means of which the transport container 400 may be guided, e.g. from a person. The transport container 400 may stand on a substrate 208u and / or be moved. The background 208u may limit the image capture area 208. The ground 208u may be e.g. define the image background of the image capture area 208.

For example, the image capture direction 208b of the image capture sensor 202s may be directed toward the ground 208u. The image capture direction 208b may optionally be configured to change the image capture direction 208b, e.g. for changing by at least one angle 202w (e.g., a solid angle). Thus, for example, the image capture direction 208b may be adjusted, e.g. On the basis of the image data and / or a recognition of a position of the transport container in the image capture area 208. For example, the image capture direction 208b may be tracked to a moving transport container 400 and / or adapted to a non-precisely positioned transport container 400.

Alternatively or additionally, the angle 202w may define an image capture angle 202w (may also be referred to as a viewing angle). The image capture angle 202w can be understood to be that angle in the spatial space that is bounded by the edges of the recording format. With the usual rectangular image format, the image capture angle 202w can be used to match the image diagonal Meaning value. Horizontal and vertical image capture angles 202w may be less than the image diagonal value.

The imaging system 202 may be or may be held by a carrier 320, e.g. by means of a checkout counter 320. Optionally, several items of the checkout system may or may not be arranged on the checkout counter 320, e.g. Screen, printer, scanner and / or a payment terminal.

According to various embodiments, the transport container 400 may be searched in the image capture area 208 (may also be referred to as the field of view of the camera).

For capturing image data (and blank detection based thereon) of the transport container 400, a direct line of sight connection between the image capture system 202 and the transport container 400 may or may not be provided. In other words, an optical recognition of objects in the transport container 400 can take place.

Figure 4 illustrates a transport container 400 according to various embodiments in a schematic side view or cross-sectional view, for example, a transport container 400 of a first transport container type.

The transport rack 322 may include a plurality of transport areas 322a, 322b (a first transport area 322a and a second transport area 322b), each of which may be configured to receive an item (which is to be transported). By way of example, each transport region 322a, 322b can be provided and / or limited by means of a support surface element 402a, 402b (eg a bottom element).

The transport rack 322 may have a plurality of surface elements, e.g. one or more overlay surface elements 402a, 402b (e.g., bottom elements 402a, 402b), one or more side surface elements 412a, 412b (e.g., side elements 412a, 412b). Each transport area 322a, 322b may be bounded by one or more areal elements, e.g. of at least one pad surface element 402a, 402b and optionally of a plurality (with respect to four) side surface elements 412a, 412b. At least a first side panel 412b may be pivotally mounted 401, e.g. between an open state and a closed state. In an open state, an opening 402o may be formed which is configured (size and location) such that a second side panel 412b fits into the opening 402o. Thus, multiple transport containers 400 can be telescoped, e.g. in the direction of 403.

For example, the cross-sectional area (e.g., cut transverse to the direction 403) of the first transport area 322a may decrease in the direction 403 that points from the first side panel 412a to the second side panel 412b.

In at least one direction, each transport region 322a, 322b may be opened 502o (i.e., have an opening 502o) such that e.g. clearly an object can pass through the opening 502o in the transport area 322a, 322b.

According to various embodiments, the transport container 400 may comprise a surface element (eg a support surface element 402a, 402b and / or a side surface element 412a, 412b) which has a plurality of through openings (eg a grid or a grate). For example, the transport container 400 may include an incompletely closed intermediate bottom surface 402a for detecting in the second transport region 322b (illustratively lower transport region 322b) from a given perspective

(for example, from above) in which the intermediate bottom surface 402a is disposed between the image acquisition system 202 and the second transport region 322b.

According to various embodiments, a first support surface element 402a (may also be referred to as intermediate bottom surface 402a) and / or a second support surface element 402b may comprise a plurality of through openings. In other words, the first support surface element 402a and / or the second support surface element 402b may not be opaque. It can thereby be made possible for the first support surface element 402a to determine whether an object is arranged in the second transport region 322b, e.g. by means of depth information. For example, it may be determined from the depth information that certain features are associated with the first overlay tile 402a and may be discarded to form the outline pattern when e.g. only the second transport area 322b is to be examined.

Alternatively or additionally, it may be possible for contours of the second support surface element 402b and / or the image background 602h to be used to form the contour pattern through the first support surface element 402a. Analogously, it can thereby be made possible that contours of the image background can be used to form the contour pattern through the second support surface element 402b. Thus, an empty recognition can be facilitated or become (clearly because the contour pattern has more features).

Alternatively or additionally, the side surface elements 412a, 412b (eg, those facing the imaging system 202) may include a plurality of through openings. As a result, contours of the first can be made possible by a side surface element 412a, 412b Pad surface element 402b, the second pad surface element 402b and / or the image background can be used to form the contour pattern. For example, it can be determined from the depth information that certain features are associated with a page area element 412a, 412b, and can be discarded to form the contour pattern.

For example, a (non-opaque) surface element 402a, 402b, 412a, 412b (e.g., metallic or non-metallic) may have or be formed from a lattice structure. Alternatively or additionally, a (non-opaque) surface element 402a, 402b, 412a, 412b may comprise or form a coherent (eg transparent or partially transparent) surface or a combination of contiguous surfaces and a lattice structure (eg transparent and / or semi-transparent) be.

There may be a plurality of transport container types, a chassis 312 and a transport frame 322. For example, two types of transport containers may differ in at least one of: an extent of the first region 322a and / or the second region 322b, in the presence of the first region 322a and / or the second region 322b.

Figure 5 illustrates a transport container 400 according to various embodiments in a schematic side view or cross-sectional view, eg a transport container 400 of a second type of transport container.

The first type of transport container may be different from the second type of container in at least the presence of a chassis 312.

The transport container 400 may have an opened transport area 322a (with an opening 502o). The transport area 322a may be from a support surface element 402a and a plurality of side surface elements 412a, 412b.

According to various embodiments, the cross-sectional area (e.g., cut transverse to the direction 403) of the first transport area 322a may decrease in the direction 403 that points from the opening 502o to the pad surface 402a. It can thus be achieved that a plurality of transport containers 400 can be inserted into one another, e.g. along direction 403, e.g. for stacking the plurality of transport containers 400.

The transport container 400 may include a handle 502, which may optionally be pivotally mounted.

Analogously to what has been described above, contours of the image background can optionally be detected by the support surface element 402a and used to form the contour pattern, if this is e.g. has a plurality of passage opening. Thus, an empty recognition can be facilitated or become.

Analogously to what has been described above, contours of the image background and / or the pad surface element 402a may optionally be captured by a side surface element 412a, 412b and used to form the contour pattern, e.g. if this e.g. has a plurality of passage opening. Thus, an empty recognition can be facilitated or become.

Several types of transport containers can not have a chassis 312. For example, two types of transport containers may differ in at least one extent of region 322a (e.g., height and / or width).

Figure 6 illustrates a method 600 according to various embodiments in a schematic flow diagram.

According to various embodiments, image data 602 of a portion of the transport container 400 may be captured 602e. Using the image data 602, a contour pattern 604 a, 604 b, 604 c can be determined, which represents the transport container 400.

For example, a first contour pattern 604a may represent an image background 602h of the image capture area (e.g., the ground 208u) that is disturbed 612h due to the transport bin 400. Illustratively, a structure 400s (e.g., a grid) of the transport container 400 may obscure a portion of the image background 602h such that its contour is disturbed 612, e.g. interrupted 612s, will.

Alternatively, a second contour pattern 604b may represent, for example, the structure 400s (e.g., a grid 400s) of the transport container 400.

According to various embodiments, a third contour pattern 604c may represent the structure 400s (e.g., a grid) of the transport container 400 and the image background 602h of the image capture area 208.

Further, the first contour pattern 604a, the second contour pattern 604b and / or the third contour pattern 604c may be used to determine the deviation amount.

For example, the contour pattern 604 (e.g., the contour pattern 604a, the contour pattern 604b, and / or the contour pattern 604c) may be used as a reference pattern when the transport bin 400 is empty. Clearly, a reference contour pattern can be taught.

Illustratively, the optical characteristics of the transport container 400 (such as a shopping cart) or feature-intensive regions thereof, and / or the Underground / image background are initially taught. The recognized features can be stored in a reference database.

Figure 7 illustrates a method 700 according to various embodiments in a schematic flow diagram.

According to various embodiments, a determined contour pattern 604 (e.g., the contour pattern 604a, the contour pattern 604b, and / or the contour pattern 604c) may have a contour that represents an object 702 when disposed in the transport bin 400. Illustratively, the contour of the transport container 400 and / or the image background may be disturbed by the article 702, e.g. (at least partially) obscured. For example, the contour pattern 604 may include one or more perturbations 702 u (e.g., breaks) caused by the article 702.

Contour pattern 604 may be compared to a reference contour pattern 704 (may also be referred to as reference pattern 704). The reference pattern 704 may represent the image background 602h of the image capture area and / or the structure 400s of the transport bin 400 when the transport bin 400 is empty.

Based on the contour pattern 604 and the reference pattern 704, a deviation amount 706 representing a deviation of the contour pattern 604 from the reference pattern 704 can be obtained. The reference pattern 704 may include a contour pattern of the image capture area in which no or empty transport bin 400 is disposed (e.g., an empty transport bin 400 in front of the image background 602h).

The deviation amount 706 may include a contrast deviation 706a, a color deviation 706a, a contour deviation 706b and / or an area occupation deviation 706c. The contrast deviation 706a may illustratively indicate how much a deviation in contrast 706k between the reference pattern 704 and the contour pattern 604 is. The contour deviation 706b can graphically indicate how strong an interruption in a course of a contour between the reference pattern 704 and the contour pattern 604 is. The area occupation deviation 706c can graphically indicate how large a deviation in an area is that occupies the contours of the reference pattern 704 and the contour pattern 604, respectively.

The contrast deviation 706a may be determined pixel by pixel, e.g. by comparing the values 706w (e.g., contrast values 706w and / or brightness values 706w) of corresponding pixels of the reference pattern 704 and the contour pattern 604.

Alternatively or in addition to the contrast deviation 706a, a color deviation 706a, for example pixel by pixel, can be determined, e.g. by comparing the values 706w (e.g., color values 706w and / or brightness values 706w) of corresponding pixels of the reference pattern 704 and the contour pattern 604.

For the contour deviation 706b, for example, a contour 604k of the contour pattern 604 can be assigned a reference contour 704k of the reference pattern 704 and the (for example spatial and / or pixelwise) deviation 706 of the contour 604k can be determined from the reference contour 704k. The deviation 706 of the contour 604k from the reference contour 704k may be e.g. in a spatial space and / or a pixel space, which is spanned by at least two coordinates 706x, 706y (spatial coordinates and / or pixel coordinates).

The area occupation deviation 706c can be determined, for example, by a comparison of the area occupation 706f (eg number occupied pixel and / or occupied portion of the pattern) of the reference pattern 704 and the contour pattern 604 are determined.

The deviation quantity 706 may be a numerical value (eg, a surface occupation deviation 706c, a contrast deviation 706a, and / or a color deviation), a vector (eg, contour deviation, depth deviation, location coordinates, and / or pixel coordinates) / or have a matrix (eg a topography deviation).

Figure 8 illustrates a method 800 according to various embodiments in a schematic flowchart.

According to various embodiments, the deviation amount 706 may be compared to a predetermined criterion 806. Further, a signal may be output when the deviation amount 706 satisfies the criterion 806. For example, the signal may include or be formed from a visual signal (e.g., a colored signal and / or a geometric signal), a prompt, and / or an audible signal.

The predetermined criterion 806 may include a first criterion 806a (may also be referred to as a blank criterion 806a) representing an empty transport container. Then, the signal may include a first signal 802a (may also be referred to as an empty detected signal 802a) representing empty-sense. Illustratively, the first signal 802a may be output, for example, if the deviation amount 706 is none than a first threshold value represented by the first criterion 806a.

The predetermined criterion 806 may include a second criterion 806b (may also be referred to as an error criterion 806b) representing an error range. Then For example, the signal may include a second signal 802b (may also be referred to as an error signal 802b) that represents error detection. Illustratively, the second signal 802b may be output, for example, if the deviation quantity 706 is greater than the first threshold value and neither is represented as a second threshold value, which is represented by a third criterion 806c.

The predetermined criterion 806 may include the third criterion 806c (may also be referred to as non-empty criterion 806c) representing a non-empty transport container. Then, the signal may include a third signal 802c (may also be referred to as a non-idle detected signal 802c) representing non-idle detection. Illustratively, the third signal 802c may be output when the deviation amount 706 is greater than the second threshold represented by the third criterion 806c.

Illustratively, the error range may be between the empty criterion 806a and the non-empty criterion 806c and may represent a range in which a clear decision as to whether the transport container 400 is empty or not can not be made reliably.

In the case of error detection, the method 100 may start over. For example, another area of the transport container 400 and / or another analysis area may be used to form the contour pattern 604.

Figure 9 illustrates a device 900 according to various embodiments in a schematic perspective view.

The POS system may include a primary screen 802, a bar code scanner 804, a secondary screen 808, and an EC terminal 816. The screen 802 can be used for Issue the signal to be established. For example, a color signal, a geometric signal and / or a prompt may be displayed on the screen representing a state of the transport container 400 (located here outside of the image capture area 208) (eg, empty or non-empty) and / or a state of the image capture area 208 (eg with or without transport container 400).

The device 900 may include an imaging system 202 having a plurality of optical sensors 202s. Each sensor 202s may define a sensing area 208s that together define the image sensing area 208. By way of example, the detection regions 208s of at least two sensors 202s of the image acquisition system 202 may overlap. The illustrated mounting positions can be understood as exemplary. The two sensors 202s may alternatively be or may be mounted such that their detection areas are spaced or adjacent to each other.

The substrate 208u may have a contour which is e.g. represents the flooring (e.g., its flow and / or joints).

According to various embodiments, the image capture area 208 may be searched for the transport bin 400. In this case, it can be recognized whether the transport container 400 is a container type of a plurality of (known) container types, e.g. the first type of container or the second type of container. Alternatively or additionally, it can be detected whether a transport container 400 is arranged in the image capture area 208.

Further, a signal (may also be referred to as unrecognized signal) may be output if no bin type of the plurality of bin types has been detected and / or not Transport container 400 has been detected in the image capture area 208. In the event that no bin type of the plurality of bin types and / or no bin 400 has been recognized in the image capture area 208, error detection may be. Then, the unrecognized signal may be an error signal.

In the case of error detection, the method 100 may start over. For example, image data from another area of the image capture area 208 may be used to form the contour pattern 604.

According to various embodiments, the signal (e.g., the error signal) may have an input request, with a POS system process transitioning to a wait state until an input is made to the prompt. Thus, an operator (e.g., the cashier) can be made to make a state (e.g., by dislocating the transportation container 400) in which the transportation container 400 and / or its transportation container type can be recognized. Thus, unregistered articles can be further reduced.

If the transport container and / or its transport container type has been detected, the contour pattern 604 can be determined. Alternatively or additionally, a reference pattern representing the transport container type may be selected. This makes it possible to determine the deviation quantity more accurately or to reduce the error detection. Clearly, the reference pattern can be adapted to the determined transport container type, so that the reference pattern of the detected transport container type is used to determine the deviation quantity.

Optionally, the predetermined criterion can be adjusted based on the detected transport container type. For example, a first criterion for the first Transport container type can be used and a second criterion can be used for the second type of transport container. Thus, the empty recognition can be done more accurately or an error detection can be reduced.

The primary screen 802 may be configured to display information that represents the article or articles that have been registered using the bar code scanner 804. For example, the registered articles may be listed and displayed along with information associated with them (e.g., registration information) on the primary screen 802, e.g. with article number information, article name information and / or price information.

Figure 10 illustrates a device 1000 according to various embodiments in a schematic perspective view.

According to various embodiments, at least one optical sensor 202s may be attached to a holder 808h (e.g., a pole) holding the secondary screen 808. The secondary screen 808 may be configured to display a portion of the information displayed by the primary screen 802, e.g. of article name information and / or price information.

Figure 11 illustrates an image acquisition system 202 according to various embodiments in a schematic perspective view.

The image capture system 202 may include multiple (eg, or more than two) image capture sensors 202s. The imaging system 202 may include a mounting structure to which the imaging system 202 may be attached. Furthermore, the image acquisition system 202 may include a Data line 202k, by means of which the image acquisition system 202 may be coupled to the processor 206, for example.

The image capture system 202 may be configured to capture image data representing two-dimensional (2D) information, e.g. in a physical space. Optionally, the image capture system 202 may be configured to capture image data representing three-dimensional (3D) information (may also be referred to as depth information), e.g. in a physical space.

The image capture system 202 may optionally include a laser projector 202p configured to project an optical pattern into the image capture area 208, eg, by patterned laser light (ie, patterned light in the form of the pattern). In other words, the laser projector 202p can emit light according to the pattern (eg, in the form of a grating) into the image capturing area 208. The emitted light may be outside of visible light, eg in an infrared (IR) spectral range (more than about 780 nm). Alternatively or additionally, other spectral ranges can be used, for example also in the visible range of the light (approximately 380 nm to approximately 780 nm). If light from extraneous light sources is emitted into the image-capturing area, spectral ranges can be used which are, for example, outside the light emitted by the extraneous light sources. At least one image capture sensor 202s may be configured to detect the spectral region of the laser projector 202p. Thus, deformations of the pattern caused by a topography in the image capturing area and / or an object in the image capturing area can be detected by the image capturing sensor 202s (ie, topography information can be obtained). In other words, the image capture sensor 202s can capture 3D image data. The 3D image data may then be from the processor be further processed. On the basis of the optical pattern projected into the image capture area 208, alternatively or additionally, depth information can be obtained, eg by determining a distance of the imaging system 202 to contours, objects and / or structures in the image capture area and / or by evaluating the topography information.

Alternatively or additionally, the image capture system 202 may be configured to capture stereoscopic image data by means of the at least two image capture sensors 202s. The image data may then have two associated image data parts taken at different angles from the same time.

The image data may be processed by the processor, e.g. measure the distance of the imaging system 202 to contours, objects, and / or structures in the imaging area (i.e., determine depth information).

Figure 12 FIG. 12 illustrates a method 1200 according to various embodiments in a schematic view.

In a 2D mode, it can be first recognized whether a transport bin 400 is located in the image capture area 208 (ie, whether a transport bin 400 is present). For example, the detection can be done by detecting reflections (on objects), eg in the IR range. It can be recognized whether the reflections originate from a transport container, eg its structure and / or markers. This can make detection faster. The reflections can optionally be compared with the reference pattern. For this purpose, the reference pattern can have reference reflection data.

The detection of the transport container 400 may be based on the image data 602. For example, on the basis of the image data 602, pattern recognition and / or contour recognition can be performed, which is compared with the database. For example, the contour recognition can only determine whether or not a contour is present.

Alternatively or additionally, the contour pattern 604 (superimposed on the image data in the illustration) can be determined in order to detect whether a transport container 400 is arranged in the image capture area 208.

Further, in the 2D mode, e.g. when it is detected that a transport container 400 is disposed in the image capturing area 208, it is determined based on the contour pattern 604 whether an article is disposed in the transport container 400. If a deviation of the contour pattern 604 from the reference pattern 704 satisfies the criterion (eg, if all elements of the contour pattern 604 match the elements of the reference pattern 704), an empty detected signal 802a may be output, for example, on the primary screen 802. The Empty Detected For example, signal 802a may indicate information representing that the transport container 400 has been identified as empty, eg a lettering and / or a color (e.g., green).

Optionally, in the 2D mode, a deviation of the color information 1204 may be used to determine the deviation amount. For example, it may be determined whether (and how many) pixels of the image data 602 have a color value different from the reference color information. If, for example, the transport container 400 only has gray tones, it can be determined whether the image data 602 have color values.

Alternatively or additionally, it may be determined in a 3D mode whether (and if so) pixels of the image data 602 have a depth value (e.g., a distance to the image capture system 202) that differs from the reference depth information. Optionally, based on the image data 602, a topography can be determined which represents the transport container 400 (or, if appropriate, its content).

For example, first a first transport area 322a may be checked in the 2D mode and optionally thereafter a second transport area 322b behind it in a 3D mode (i.e., the first transport area 322a is disposed between the second transport area 322b and the image capture system 202).

Figure 13 illustrates a method 1300 according to various embodiments in a schematic view.

In the 2D image mode, it can be determined if the transport bin 400 is not located in the image capture area 208 (e.g., has not been parked in front of the POS system and / or is at least partially occluded). For example, the contour pattern 604 may have fewer features than specified by the criterion. Alternatively or additionally, the contour pattern 604 may include features that represent an edge of the transport container 400. Clearly, it can be determined whether the transport container 400 is not completely arranged in the image capture area 208 and / or at least partially hidden (for example, by a person).

If no transport container 400 has been detected in the image capture area 208 and / or detected that it is not completely disposed in the image capture area 208 and / or at least partially obscured, then an error signal 802b may be output, eg an error signal 802b (eg non-Detected signal). The error signal 802b may for example, on the primary screen 802. The error signal 802b can, for example, display information representing that no transport container 400 has been detected, eg a lettering and / or a color (eg yellow).

Optionally, in addition to (in 2D mode), color information 1204 and / or (in 3D mode) depth information 1206 may be used to detect if a shipping bin 400 is disposed in the image capture area 208. For example, the contour pattern 604 can not be unambiguously detected when the image capture area 208 is over-illuminated. In that case, e.g. if the detection of the transport container 400 on the basis of the contour pattern and / or the color information 1204 or the depth information 1206 leads to different results, the error signal 802b has information representing the cause of the error.

Optionally, if the transport container type has not been detected, the error signal 802b may include information representing the cause of the error "transport container type unrecognized".

If the transport bin 400 has areas that tend to error detection (e.g., billboards or highly reflective areas), these areas (error area) may be hidden (i.e., not used to determine the contour pattern 604). Alternatively or additionally, the error signal 802b may include information representing a corresponding cause of error (e.g., "error range detected").

Figure 14 illustrates a method 1400 according to various embodiments in a schematic view.

In the case that the transport container 400 has been recognized in the image capture area 208 (eg, in the 2D mode), however, no decision can be made as to whether it is empty or non-empty (ie, that an item is located in the transport bin 400) , an error signal 802b may be output. For example, the error signal 802b may be output if no contour pattern could be detected when the image data is noisy, underexposed and / or overexposed. For example, in the 2D mode, transparent objects, thin objects, and / or small objects may interfere only partially with the contour pattern to such an extent that the determination of the contour pattern is impaired.

Optionally, additional color information 1204 (in the 2D mode) and / or depth information 1206 (in the 3D mode) may be used to detect if the transport bin 400 is present. However, it may also be the case that the color information 1204 can not be uniquely assigned, e.g. when the article is low in saturation (e.g., a white color), high reflectivity or at least partially occluded by a portion of the transport container 400.

The error signal 802b may, for example, represent information representing the cause of the error "transport container type detected" and / or "no decision possible", e.g. lettering and / or color (e.g., orange).

Figure 15 illustrates a method 1500 according to various embodiments in a schematic view.

According to various embodiments, based on the contour pattern 604 (shown superimposed here with the image data 602), it may be detected when an object 702 in the Transport container 400 is disposed, for example, in the first transport region 322a.

In the 2D mode, when an item 702 (eg, an item) has been detected in the shipping container 400, a non-empty detected signal 802c may be output, for example, on the primary screen 802. The non-empty detected signal 802c may For example, display information representing that the transport container 400 has been recognized as non-empty, eg a lettering and / or a color (e.g., red). Alternatively or additionally, the non-idle recognized signal 802c may include an input request 1402. For example, prompt 1402 may require confirmation that the non-idle recognized signal 802c is true (i.e., no false alarm has been asserted). If the non-idle signal 802c is false (i.e., a false alarm has been triggered), i. If it has been erroneously recognized that an item is located in the transport container 400, optionally the criterion may be adjusted, e.g. such that a similar contour pattern 604, such as that resulting in the false alarm, will in the future trigger an error signal (e.g., the detected contour pattern falls within the error range).

Optionally, based on the color information 1204, it may be recognized and / or confirmed whether the item 702 is disposed in the transport container 400, e.g. in the first transport area 322a. For example, the contour and color detection can be triggered simultaneously.

Figure 16 illustrates a method 1600 according to various embodiments in a schematic view.

According to various embodiments, it may be recognized based on the contour pattern 604 when an article 702 is disposed in the transport container 400, eg, in the first transport region 322a. For example, the item 702 a portion of the first bearing surface element 402a (cf. Figure 3 and Figure 4 ), the chassis 312 (see Figure 3 ) and / or the image background. The covering and / or the area occupancy of the cover can be determined on the basis of the contour pattern 604.

If the object 702 is transparent, the color information 1204 may, for example, have too little deviation to satisfy the criterion.

Optionally, based on the depth information 1206, it can be recognized and / or confirmed that the item 702 is located in the transport container 400, e.g. in the first transport area 322a. For example, the contour and topography recognition can be triggered simultaneously.

Figure 17 illustrates a method 1700 according to various embodiments in a schematic view.

If an object 702 is arranged in a second region of the transport container 400, which is at least partially obscured, for example, by the first region 322a, a deviation, which is caused by article 702, can be detected on the basis of the contour pattern 604. For example, the article 702 may include a portion of the second overlay surface 402b (see FIG Figure 3 ), undercarriage 312 and / or image background 602h.

Optionally, based on the color information 1204 and / or the depth information 1206, it can be detected and / or confirmed that the item 702 is located in the transport container 400, eg in the second transport area 322b. For example, the contour recognition and simultaneously the color detection and / or the depth detection can be triggered.

For empty recognition of the transport container 400, according to various embodiments, a contour pattern 604 and optionally color information 1204 and / or depth information 1206 may be used. A database may include reference image data and / or reference color information or reference depth information representing an empty transport container 400. For idle recognition of the transport container 400 can thus be dispensed with an article database.

Figure 18 illustrates a method 1800 according to various embodiments in a schematic view.

According to various embodiments, the image data may represent a first portion 1802 of the first transport area 322a (may also be referred to as a first portion 1802).

According to various embodiments, additional image data of an additional area of the transport container may be detected. The additional image data of the additional area may represent a second part 1804 of the first transport area 322a (may also be referred to as second area 1804). The image data and the additional image data may be acquired at one time (e.g., simultaneously and / or by the same optical sensor), sequentially, and / or by various optical sensors.

Based on the additional image data, an additional contour pattern 614 (second contour pattern 614) may be determined, which may be used as the reference pattern 704. The second partial area 1804 can be used as a reference in order to determine whether an object is arranged in the first partial area 1802.

Alternatively or additionally, the additional image data of the additional area may be the second transport area 322b represent. Based on the additional image data, an additional contour pattern 624 (third contour pattern 624) can be determined. The image data and the additional image data can be recorded at once (eg simultaneously and / or by means of the same optical sensor), one after another and / or by means of different optical sensors. The deviation amount may then represent, for example, a disturbance 702u of the third contour pattern 624, which is caused by the counter sand 702.

In the case that the surface elements delimiting a transport region 322a, 322b (e.g., in the case of a basket, illustratively the basket inner surfaces if present, such as larger area gratings), e.g. For example, if the support surface element 402a, 402b (in the case of a basket, a basket tray, e.g., with braces and / or a continuous shelf) are disposed in the image capture area 208 (illustrative camera field of view), the presence of an article 702 in the shipping container 400, e.g. on the support surface element 402a, 402b resting, are determined. For this purpose, a deviation caused by the visible surface of the object, characteristic properties of a previously learned contour pattern 704 (i.e., the stored reference pattern 704), and / or a currently determined contour pattern 614 (i.e., the additional contour pattern 614) may be determined. The deviation can be represented by the deviation quantity.

Illustratively, the amount of deviation may be a measure of the absence 702u of the expected contour (eg, lattice structure or grid area) and / or disturbance 702u thereof in the inspected area of the transport container 400 caused by the article 702. For example, in the second subregion 1804 (eg, around the article 702) and / or in another part of the image data, the expected contour pattern (eg grid structure or grid area) may be present.

Alternatively or additionally, the deviation quantity may be a measure of contrast transitions and / or contrast jumps, e.g. between adjacent areas and / or areas (e.g., between the pad surface elements 402a, 402b and the article 702). This may be done based on a learned (e.g., stored) reference pattern 704 and / or on the basis of two determined contour patterns 604, 614, 624, e.g. by relating the determined (e.g., contiguous and / or contiguous) contour patterns 604, 614, 624 to each other or to the reference pattern 704.

Alternatively or additionally, a contour pattern 1814 can be determined, which represents the substrate 208u (for example, tile floor with joints, applied design elements such as advertising logos on the floor, up to cyclically trained soils, etc.). The amount of deviation may then be a measure of the absence 702u of the expected contour of the ground 208u and / or of its disturbance 702u caused by the object 702, e.g. deviating from the reference pattern 704 and / or a determined contour pattern 604, 614, 624 (clearly anticipated contours / textures).

According to various embodiments, objects that are not and / or difficult to detect by means of 3D and / or color recognition (eg bottle cases with narrow bottles and thus too small surface for the 3D projection correspondence analysis, colorless and / or poor objects) in the second transport region 322b (clearly below the basket on the lower shelf) can be recognized by using the method described herein (eg, to compare the area under investigation to the neighborhood).

According to various embodiments, a disturbance (may also be referred to as a parasitic disturbance) that is not caused by an article 702 (e.g., reflections on metal, discolorations of transport container 400 by rust, deformations of transport container 400, etc.) may be detected. The parasitic disorder (may also be referred to as a negative case) may thus be distinguished from the presence of an article 702 in the transport container 400 (which may also be referred to as a positive case). As a result, a false alarm can be avoided or reduced.

According to various embodiments, color information may optionally be provided by color recognition. For example, the deviation amount may include a deviation of the color information of the image data from reference image data representing the transportation container 400 and / or the image background 602h (e.g., the background 208u) when the transportation container 400 is empty. Clearly, a contour difference detection and color difference detection (deviating features to the reference data set) can take place. The color information of the background 208u in the image capture area 208 can be initially learned and stored in the database.

If, for example, a change from a reference color spectrum was detected in the analysis area, this can be used as an indicator that the transport container 400 is not empty.

According to various embodiments, the deviation from the reference color information in the color space may be dynamically evaluated. Thus, a false alarm can be reduced by an influence of ambient light. In other words, an illumination state of the image capture region 208 may be taken into account for determining the deviation quantity, for example based on the additional contour pattern 614.

According to various embodiments, optional 3D information (depth information) may or may not be provided by depth detection. The position of the region (eg, an adjustable region between the background 208u, above the second overlay surface element 402b, and the first overlay surface element 402a) in which the depth information is to be determined can be initially learned (can be seen as a zone to be examined To be defined). For example, this can be done on the basis of the recognized transport container type. The position of the examining zone (e.g., its limits and / or threshold) may be stored in the database, e.g. assigned to the respective transport container type.

If a deviation of the depth information from reference depth information has been detected in the zone to be examined (for example, the second transport area 322b can be detected), this can be used as an indicator that the transport container 400 is not empty.

According to various embodiments, optionally a combination variance may or may not be provided by means of a weight characteristic.

Clearly, any deviation represented by the amount of deviation may be utilized by the various non-idle state reference information (shipping bin 400 is not empty). Each active sensor may contribute to the detection of the non-empty state. By means of the weight characteristic, the data of the individual sensors can be weighted. Thus, an intelligent (eg learnable) combination can be or will be provided to enhance the quality of the evaluation. For example, the weight characteristic can be adjusted if a false alarm has been detected. Then, for example, based on the cause of the error, the Error cause associated data (contour pattern, depth information and / or color information) are weighted less. Alternatively or additionally, the weight characteristic may depend on the type of transport container. The weight characteristic and / or its dependency can be stored in the database.

According to various embodiments, the method for computer-aided emptying of the transport container 400 and the apparatus for computer-aided emptying of the transport container 400 can be inexpensively retrofitted.

According to various embodiments, a complex learning of target objects (objects to be detected 702) can be dispensed with. In other words, an object database can be dispensed with. Instead, a transport container database can be used.

According to various embodiments, simple (e.g., retrofitting) mounting of an image capture sensor 202s to an existing periphery, e.g. Holders of the elements of the cash register system (vivid the cash register furniture), be or be provided.

According to various embodiments, the image capture direction 208b may or may not be directed to the ground 208u. Thus, an impairment of the privacy of persons (e.g., customers) can be reduced since the image sensing sensor 202s is mounted in a non-critical view.

According to various embodiments, the acquisition of the additional image data and the image data may occur simultaneously. Alternatively or in addition to the additional contour pattern 614, the reference pattern 704 can be read out from a data memory.

The image data used can be provided with a sufficiently large dimension to vividly provide the highest possible signal-to-noise ratio and / or a sufficient threshold value. The image data used can be provided with a sufficiently small dimension to clearly require as little computation as possible.

Figure 19 illustrates a method 1900 according to various embodiments in a schematic view.

According to various embodiments, first of all, training of the contour pattern 614 of the transport container 400 and / or (e.g., feature-intensive regions) of the image background 602h (e.g., the bottom structure of the substrate 602h, such as texture and / or texture) may be performed, e.g. Features, contours, and / or patterns (e.g., by pattern recognition) of the contour pattern 614, and optionally depth information and / or color information of the transport container 400 and / or the image background 602h. This data can be stored in a database (reference database).

For this, image information of an empty transport container 400 may be captured by means of the image capture system 202 (e.g., in front of the background 208u). Based on the image information, a reference pattern 704, 614 may be determined. The reference pattern 704, 614 may be stored by means of a data memory 204.

fig.20 illustrates a method 2000 according to various embodiments in a schematic view.

According to various embodiments, recognition of the transport container 400 and / or its spatial position in the image capture area 208 (position determination) may be based on the image data 602. Alternatively or additionally, the detection of the transport container 400 and / or its spatial location in the image capture area 208 using an acoustic sensor (eg using sound reflection), an optoelectronic sensor (eg a light barrier) and / or a radio tag sensor (eg RFID - identification using electromagnetic waves) done. For example, the transport container 400 may have a radio tag (may also be referred to as an RFID transponder) and / or optical reflectors (optical markers).

According to various embodiments, reflections, e.g. whose distinctive shape is detected (in the form of reflection data), e.g. in 2D image space. For example, the reflections may be excited by emitting light having a defined wavelength range (e.g., IR, e.g., radiated from an IR projector) and / or at a defined angle into the image capture area 208. The reflections can be effected by a metallic lattice structure of the transport container and / or the optical reflectors. Based on a comparison of the detected reflection data (e.g., by a feature comparison) with the reference data, it can be determined whether a transport container is in the image capture area 208. This can lead to a considerably optimized reaction time.

Optionally, an analysis area 602b (ROI) can be determined based on the spatial location. The analysis area 602b can illustratively designate the area of the image data which is subsequently processed, e.g. by means of the processor 206.

According to various embodiments, the state of the contour pattern 604 can be determined by means of the processor 206, for example, whether the contours of the grid structure of the transport container 400 are complete or are interrupted or obscured by objects.

Optionally, on the basis of the image data 602 or the analysis area 602b, a color and / or texture deviation which is caused by an object 702 in the transport container 400 (visibly by existing objects in the basket or in the lower region of the transport medium) can be determined Reference pattern 704, which eg the bottom structure 208u under the transport container 400 and / or other parts of the transport container 400 represents.

Optionally, deviation data can be combined with the data of the acoustic sensor, the optoelectronic sensor and / or the radio tag sensor. This can reduce a tendency to false alarms (clearly an optimization of decision-making certainty).

Optionally, depth information may be determined based on the image data 602 and the analysis region 602b, respectively. For example, the analysis area 602b may be adjusted based on the depth information, e.g. so that it overlaps with the second transport area 322b.

According to various embodiments, a signal may be output, e.g. to the operator and / or by means of a screen 802, which represents a result of the empty recognition.

• es kann erkannt werden, ob ein ermitteltes Konturmuster, welches einen zu analysierenden Transportbehälter 400 repräsentiert, eine Störung 702u aufweist, welche von einem Gegenstand 702 in dem Transportbehälter 400 bewirkt wird. Mit anderen Worten kann eine Abweichung von einem dem System bekannten Konturmuster (welches z.B. eine freiliegende Gitterstruktur repräsentiert) ermittelt werden.
• optional kann eine Farbdifferenzanalyse gegenüber den Referenz-Bilddaten, z.B. gegenüber Bild-Hintergrundfarb- und Merkmalsinformationen, erfolgen, z.B. eingeschränkt auf den Analysebereich 602b.
• optional kann erkannt werden, ob ein Gegenstand 702 in einem (z.B. skalierbaren und/oder justierbaren) Analysebereich 602b angeordnet ist, welcher einen zumindest teilweise verdeckten Bereich des Transportbehälters 400 bewirkt;
• es kann erkannt werden, ob der Transportbehälter 400 den Zustand leer (Leer-Zustand) aufweist;
• es kann ein eines Signals (Meldesignals) ausgegeben (z.B. erzeugt) werden, wenn ein anderer Zustand als der Leer-Zustand (z.B. wenn im Transportbehälter 400 ein Gegenstand 702 angeordnet ist) und/oder kein Transportbehälter 400 erkannt wurde;
• optional kann ein Anlernen und/oder Ausblenden zeitlich variabler Objekte (mit z.B. wechselnder Oberfläche, wie z.B. Werbemittelhalter) erfolgen, um eine zuverlässige Leer-Erkennung des Transportbehälters 400 bereitzustellen (z.B. obwohl dessen Konturmuster vom Referenzmuster abweicht). Das Anlernen kann aufweisen, ein Konturmuster zu ermitteln, welches das variable Objekt repräsentiert und ein Referenzmuster und/oder die Datenbank zum Berücksichtigen des variablen Objekts zu aktualisieren.

According to various embodiments, a combinatorial processing for empty detection of the transport container 400 may take place, which comprises:

• It can be detected whether a detected contour pattern, which represents a transport container 400 to be analyzed, has a disturbance 702 u, which is caused by an object 702 in the transport container 400. In other words, a deviation from a contour pattern known to the system (which, for example, a exposed lattice structure represents) are determined.
• Optionally, a color difference analysis can take place with respect to the reference image data, for example with respect to image background color and feature information, eg limited to the analysis region 602b.
• Optionally, it can be detected whether an object 702 is arranged in an analysis area 602b (eg scalable and / or adjustable), which causes an at least partially hidden area of the transport container 400;
• it can be detected whether the transport container 400 has the state empty (empty state);
• a signal (report signal) can be output (eg generated) if a state other than the empty state (eg when an article 702 is arranged in the transport container 400) and / or no transport container 400 has been detected;
• Optionally, teaching and / or hiding of temporally variable objects (with, for example, changing surface, such as eg advertising medium holder) can take place in order to provide reliable empty recognition of the transport container 400 (for example, although its contour pattern deviates from the reference pattern). The training may include determining a contour pattern representing the variable object and updating a reference pattern and / or the database for considering the variable object.

Figure 21 illustrates a method 2100 according to various embodiments in a schematic flow diagram.

Optionally, method 2100 in 2101 (may also be referred to as initial phase 2101) may include initiating an imaging system.

Further, method 2100 may be 2103 (may also be referred to as a transport container search phase 2103). comprising: searching a transport container in an image capture area of the image acquisition system.

Further, method 2100 may include 2105 (may also be referred to as empty detection phase 2105 or detection phase 2105 or article detection phase 2105): detecting whether the transport container is empty. The blank detection phase 2105 may include, according to various embodiments: determining a deviation amount; and determining whether the deviation quantity meets a predetermined criterion.

Further, the method 2100 in 2107 (may also be referred to as evaluation phase 2107) may include: determining whether empty-detection was successful.

Further, method 2100 may include 2109 (may also be referred to as signal output phase 2109) outputting a signal representing a result of the empty detection.

Figure 22 illustrates an initial phase 2101 according to various embodiments in a schematic flow diagram.

The initial phase 2101 may optionally include in 2201 (may also be referred to as program start 2101): starting a program configured to perform a method according to various embodiments. For example, a processor may be put into a ready state. The processor may be configured to perform the method according to various embodiments, e.g. by executing the program.

Further, the initial phase 2101 may include 2203 (may also be referred to as camera initialization 2203): Initiate an image capture system, such as an image capture sensor or multiple image capture sensors of the imaging system. For example, the image capture system (eg, one of the image capture sensors or the multiple image capture sensors) may be placed in an operational state. Alternatively or additionally, at least one further (eg non-optical) sensor can be initiated, for example at least one acoustic sensor, at least one optoelectronic sensor and / or at least one radio tag sensor.

Further, the initial phase 2101 may include 2205: providing (e.g., loading and / or creating) a reference pattern or patterns. For example, the reference pattern may be loaded (i.e., called) from a reference database.

Further, the initial phase 2101 may include 2207: providing (e.g., loading and / or creating) reference data. The reference data may include or may be formed from one or more reference patterns, reference topography information, reference depth information, reference color information, and / or reference depth information. The reference data can be generated for the sensors.

Figure 23 illustrates a transport container search phase 2103 according to various embodiments in a schematic flow diagram.

The transport container search phase 2103 may include at 2301: providing (eg, loading and / or creating) image data (may also be referred to as a camera frame). The image data may represent a single image. The image data can be acquired for loading.

Further, the transport container search phase 2103 may include 2303 (may also be referred to as blank detection 2303 of the image capture area): detecting whether a transport container is located in the image capture area. To detect the transport container in the image capture area, 3D data (e.g., topography data and / or depth information) may be acquired in the image capture area. For example, for this purpose, an emitting (e.g., projecting) of an optical pattern into the image capturing area and capturing image data of the image capturing area representing the optical pattern may be performed. The acquisition of the 3D data (e.g., topography data and / or depth information) may be based on the optical pattern (may also be referred to as IR search phase 2303). The optical pattern can be clearly influenced by a transport container in the image capture area. Clearly, for detecting the 3D data, a change of the optical pattern which is caused by the transport container when it is arranged in the image capturing region can be detected. At least one image sensing sensor of the image acquisition system may be configured to detect the spectral range of the optical pattern.

Further, the transport bin search phase 2103 may include at 2305: Decide whether a transport bin has been detected in the image capture area. If no transport bin has been detected in the image capture area 2305b (decision = no 2305b), then proceeding to 2301 may be accomplished by providing additional image data. If a transport container has been detected in the image capture area 2305a (decision = yes 2305a), 2307 can proceed. The decision as to whether a transport container has been recognized in the image capture area can be based, for example, on the 3D data. For example, the 3D data may represent whether and / or how much the optical pattern is altered when the transport container is arranged in the image capture area. Illustratively, the transport container search phase 2103 may form a loop in which image data are acquired until it has been detected that a transport container is brought into the image capture area and / or already arranged therein.

In order to decide whether a transport container has been detected in the image capture area, a determination of a 3D deviation quantity representing a deviation of the 3D data from reference 3D data, wherein the reference 3D data represents an empty image capture area, can take place. Clearly, the 3D data can be compared with the reference 3D data. If the deviation satisfies a predetermined criterion, it can be decided that a transport container has been detected in the image capturing area (decision = yes 2305a). If the deviation does not satisfy the predetermined criterion, it can be decided that no transport container has been detected in the image sensing area (decision = no 2305b).

The transport container search phase 2103 may optionally include 2307: forming a plurality of image data using further image data (may also be referred to as another camera frame). The further image data can represent at least one further individual image. The plurality of image data can illustratively represent an image sequence (eg in a temporal and / or spatial sequence). To form the plurality of image data, the further image data may be acquired after it has been recognized that a transport container is brought into the image capture area and / or arranged in the image capture area. Thus, a large number of image data can be clearly captured, which represent the transport container, for example, in different layers.

When a plurality of image data are present, the transport container search phase 2103 may include at 2309: selecting the image data representing the transport container and / or having a predetermined contour density. For example, the image data can be determined from the plurality of image data (which can be clearly selected), which represent the transport container and / or which have a predetermined contour density. For selecting the image data, already processed image data of the plurality of image data may be discarded (i.e., not taken into consideration).

Further, the transport container search phase 2103 may include at 2311: detecting the transport container using a contour pattern. For this purpose, determination of a contour pattern, which represents the transport container, can take place using the image data. In 2311, for example, the transport container type and / or a position (orientation and / or position in the image capture area) of the transport container can be recognized.

Further, the transport container search phase 2103 may include at 2313: decide whether the transport container has been detected.

Figure 24 FIG. 12 illustrates a method 2400 according to various embodiments in a schematic flow diagram.

The method 2400 may include the transport container search phase 2103 and the blank detection phase 2105.

Further, the method 2400 in 2401 may include: whether there is more image data.

If it has been decided 2313 that a transport container has been detected in the image capture area 2103a (decision = yes 2103a), the empty recognition phase 2105 may proceed.

If it has been decided 2313 that no transport bin has been detected 2103b (decision = no 2103b), decision 2401 may be made as to whether there is any more image data (e.g., not yet analyzed).

If it has been decided 2401 that there is more image data 2401a (decision = yes 2401a), 2309 may select other image data from the plurality of image data. If it has been decided 2401 that there is no further image data 2401a (decision = no 2401b), then the transport container search phase 2103 may proceed, e.g. For example, in 2301, other image data may be provided (e.g., captured and / or loaded).

Figure 25 illustrates a method 2500 according to various embodiments in a schematic flow diagram.

The method 2500 may include the transport container search phase 2503 and the empty detection phase 2505.

Optionally, the method may include 2500 in 2501: determining an analysis area. Clearly, a part can be selected from the image data, which is used in the empty detection phase 2105 (empty detection of the transport container). Thus, the effort can be reduced.

Figure 26 illustrates an empty detection phase 2105 according to various embodiments in a schematic flow diagram.

The blank detection phase 2105 may include in 2601: determining a deviation amount based on a contour pattern. For this purpose, a determination of the contour pattern representing the transport container can take place using the image data (or the part of the image data); and determining the deviation quantity, which represents a deviation of the contour pattern from at least one reference pattern, wherein the at least one reference pattern represents an empty transport container.

Optionally, the blank detection phase 2105 in 2603 may include: determining the deviation amount based on color information. For this purpose, determination of the color information representing the transport container can take place using the image data (or the part of the image data); and determining the deviation amount representing a deviation of the color information from reference color information, wherein the reference color information represents an empty transportation container.

Optionally, the empty detection phase 2105 in 2605 may include: determining the amount of deviation based on depth information. For this purpose, determination of the depth information representing the transport container can take place using the image data (or the part of the image data); and determining the deviation amount representing a deviation of the depth information from reference depth information, wherein the reference depth information represents an empty transportation container.

Optionally, the blank detection phase 2105 in 2607 may include: determining the amount of deviation based on topography information. For this purpose, determination of the topography information representing the transport container can take place using the image data (or the part of the image data); and determining the deviation amount representing a deviation of the topography information from reference topography information, wherein the reference topography information represents an empty transportation container. The Topography information can be obtained, for example, using the optical pattern.

In the Figure 26 illustrated sequence can be consecutive (can also be referred to as a single thread). For example, after each determination of the deviation quantity 2601, 2603, 2605, 2607, it can be determined whether a clear decision (for example, whether the transport container is empty or not) can be made reliably (cf. error range recognition). For this purpose, a reliability of the deviation quantity can be determined.

For example, the variance may be refined (increased in reliability) by using additional information (color information, depth information, and / or topography information) to determine the variance. For example, a further determination of the deviation quantity 2603, 2605, 2607 can take place if it was determined that a clear decision can not be made reliably. Clearly, the data of the deviation quantity can be successively superimposed successively until a sufficient decision reliability is achieved.

Fig.27 FIG. 12 illustrates an empty detection phase 2105 according to various embodiments in a schematic flow diagram analogous to FIG Figure 26 ,

In the Fig.27 illustrated sequence may be at least partially parallel to each other (may also be referred to as multithreaded). For example, at least two (eg, three or four) pieces of information for determining the deviation quantity 2601, 2603, 2605, 2607 may be processed simultaneously.

Figure 28 illustrates an evaluation phase 2107 according to various embodiments in a schematic flow diagram.

The evaluation phase 2107 may include in 2801: Deciding whether an item (e.g., an item) has been detected in the shipping container. If an item has been detected in the shipping container 2801a (decision = yes 2801a), a corresponding signal may be output 2109 (e.g., a non-idle detected signal). If no item is detected in the shipping container 2801b (decision = no 2801b), optionally, a corresponding signal may be output 2109 (e.g., an empty detected signal).

If no item has been detected in the shipping container 2801b (decision = no 2801b), optionally in 2803 of the evaluation phase 2107, a determination can be made as to whether the decision by means of the image data used was not possible, for example because the deviation quantity was not reliable enough and / or because the image data was not sufficiently representative.

If it is determined that decision based on the image data used was not possible 2803a (decision = yes 2803a), a corresponding signal may be output 2109 (e.g., an error signal). If it was possible to decide by means of the image data used 2803b (decision = no 2803b), in 2805 of the evaluation phase 2107 optionally a final decision can be made as to whether no item (e.g., an item) was detected in the transport container.

When it has finally been decided that no item has been detected in the transport container 2805a (decision = yes 2805a), the transport container search phase 2103 may be started from the beginning, eg, continue with 2301, eg, by adding new image data of the transport container to be provided. Clearly, then, for example, the loop 2301, 2303, 2305 be performed until the next transport container was detected.

If it has not been definitively decided that no item has been detected in the shipping container 2805b (decision = no 2805b), a decision may be made in 2401 as to whether there is any more image data (e.g., not yet analyzed). Clearly, the decision that no object is arranged in the transport container can be checked using further image data of the plurality of image data.

According to various embodiments, final arbitration 2805 may occur when a predetermined number of decisions match based on one and the same plurality of image data. Thus, a risk of misrecognition can be reduced.

If there are other image data 2401a (decision = yes 2401a), selection 2309 of other image data may be made from the plurality of image data. If there is no further image data 2401a (decision = no 2401b), the transport bin search phase 2103 may be started from the beginning, e.g. can be continued with 2301. Then, for example, new image data from the same transport container can be detected.

Figure 29 illustrates a signal output phase 2109 according to various embodiments in a schematic flow diagram.

The signal output phase 2109 may include in 2901: issuing a prompt and / or transitioning a POS system process to a wait state (may also be referred to as a program stop and / or waiting for user input). For example, the program can be stopped. The signal output phase 2109 may include at 2903: determining whether an input to the prompt has occurred.

When an input to the prompt has been made 2903a (decision = yes 2903a), the transport bin search phase 2103 may be initiated from the beginning, e.g. can be continued with 2301. If no prompt entry is made 2903b (decision = no 2903b), the command prompt and / or the wait state may be maintained.

Fig.30 FIG. 3 illustrates a method 3000 according to various embodiments in a schematic flow diagram analogous to FIGS Fig.21 to Fig.29 ,

Fig.31 illustrates a method 3100 according to various embodiments in a schematic flow diagram.

The method 3100 may include at 3101: processing the image data (may also be referred to as process phase 3101).

Method 3100 may optionally include at 3103: forming and / or fitting a reference pattern (may also be referred to as reference pattern learning phase 3103), e.g. if there is no (e.g., matching) reference pattern and / or there are too few reference patterns.

Method 3100 may optionally include at 3105: forming and / or adjusting a criterion (may also be referred to as criterion learning phase 3105).

The method 3100 may further include at 3107: determining a deviation amount (may also be referred to as detection phase 3107).

The method 3100 may further include in 3109: outputting a result (may also be referred to as output phase 3109), e.g. Output the deviation quantity.

fig.32 illustrates a process phase 3101 according to various embodiments in a schematic flowchart.

The process phase 3101 may include in 3201: capturing image data.

The process phase 3101 may further include in 3203: determining a contour pattern using the image data. For example, the contour pattern may be formed using a Gaussian Difference algorithm (may also be referred to as a difference of Gaussian, DoG). The Gaussian Difference algorithm may include forming a blurred version of the image data based on the image data and subtracting (subtracting) it from the image data. Forming the fuzzy version can be done using a Gauss kernel, e.g. by convolving the image data using the Gauss kernel. Alternatively or additionally, the blurred version may be formed using a blur.

The process phase 3101 may further include in 3205: Requesting a reference pattern.

The process phase 3101 may further include in 3207: Deciding whether a reference pattern exists. If there is a reference pattern 3207a (decision = yes 3207a), you can proceed to 3209. If there is no reference pattern 3207b (decision = yes 3207b), with 3103 be continued, for example by a reference pattern is formed.

The process phase 3101 may optionally include in 3209: forming an analysis area for each deviation of the contour pattern from the reference pattern. For this purpose, one or more deviations of the contour pattern from the reference pattern can be determined.

fig.33 illustrates a reference pattern learning phase 3103 according to various embodiments in a schematic flow diagram.

The reference pattern learning phase 3103 may include at 3301: providing (e.g., forming) gray image data. For this, the image data can be converted to grayscale (if they are not already in grayscale).

The reference pattern learning phase 3103 may optionally include in 3303: determining a mean variation of the gray image data. The mean variation may represent a mean variation of the gray values of the gray image data. To determine the mean fluctuation, a mean deviation and / or a standard deviation can be determined.

The reference pattern learning phase 3103 may optionally include in 3305: forming a gray level matrix using the gray image data.

The reference pattern learning phase 3103 may optionally have in 3307: determining a fluctuation of the gray scale matrix, eg a mean fluctuation and / or a fluctuation of adjacent entries of the gray value matrix. The fluctuation may represent a contrast, a homogeneity and / or an energy of the gray level matrix.

The reference pattern learning phase 3103 may further include in 3309: forming a reference pattern based on the gray image data, e.g. using the fluctuation (according to the gray value matrix and / or the gray image data).

The reference pattern learning phase 3103 may optionally include in 3311: storing the reference pattern and optionally the variation, e.g. in a reference database. Alternatively or additionally, the reference pattern can be further processed directly.

Alternatively or additionally, the reference pattern learning phase 3103 may be performed when it has been finally decided that no object is disposed in the transport container, and / or when an input represents that no object is disposed in the transport container.

Fig.34 illustrates a criterion learning phase 3105 according to various embodiments in a schematic flow diagram.

The criterion learning phase 3105 may include in 3401: forming gray image data of the analysis area. For this, the analysis area of the image data can be converted to grayscale (if this is not already in grayscale).

The criterion learning phase 3105 may further include in 3403: determining a mean variation of the gray image data of the analysis area. The mean variation may represent a mean variation of the gray values of the gray image data. To determine the mean fluctuation, a mean deviation and / or a standard deviation can be determined.

Alternatively or additionally, the criterion learning phase 3105 may include in 3405: forming a gray level matrix using the gray image data of the analysis area. By doing The case may include the criterion learning phase 3105 at 3407: determining a variation of the gray level matrix of the analysis area, eg, a mean variation and / or a variation of adjacent gray level matrix entries. The fluctuation may represent a contrast, a homogeneity and / or an energy of the gray level matrix.

Fig.35 illustrates a method 3500 according to various embodiments in a schematic flow diagram.

The method 3500 may include the process phase 3101 and the output phase 3109. Optionally, the method 3500 may include the criterion learning phase 3105.

The method 3500 may further include 3501: detecting whether there is a deviation of the contour pattern from the reference pattern, e.g. another deviation if at least one deviation has already been detected.

The detection of whether a deviation of the contour pattern from the reference pattern exists can be done after the process phase 3101, e.g. after forming 3209 an analysis area for each deviation of the contour pattern from the reference pattern. Clearly, every deviation can be assigned an analysis area iteratively.

The result of whether there is a deviation of the contour pattern from the reference pattern may be output in 3109.

When a deviation of the contour pattern from the reference pattern has been detected 3605a (decision = yes 3605a), optionally, the criterion learning phase 3105 may be performed, eg proceeding with forming 3401 gray image data of the analysis area.

Fig.36 illustrates a recognition phase 3107 according to various embodiments in a schematic flow diagram.

The detection phase 3107 may include in 3601: comparing the contour pattern with the reference pattern. For this purpose, the contour pattern can be compared with the reference database, e.g. to select the corresponding contour pattern. Based on the comparison, the deviation quantity can be determined.

The detection phase 3107 may further include in 3603: comparing the deviation quantity with the criterion.

The detection phase 3607 may further include at 3603: Deciding whether an item has been detected in the shipping container. If the deviation quantity corresponds to the criterion, it can be decided that an object has been detected in the transport container. If the deviation quantity does not correspond to the criterion, it can be decided that no object has been recognized in the transport container.

If an item has been detected in the shipping container 3605a (decision = yes 3605a), 3109 may proceed. the result that an item has been detected in the transport container may be output 3109. If no item was detected in the shipping container 3605b (decision = yes 3605b), you can proceed to 3501. Then the next deviation and / or the next analysis area can clearly be processed.

If all deviations and / or the analysis areas of the image data have been processed, you can proceed to 2401.

Fig.37 FIG. 12 illustrates a method 3700 according to various embodiments in a schematic flow diagram analogous to FIGS Fig.31 to Fig.36 ,

The output 3109 result can be further processed, e.g. to output the signal.

Claims (11)

A method (100) for computer-aided idle recognition of a transport container (400), the method (100) comprising: • acquiring (101) image data of a region of the transportation container (400); Determining (103) a contour pattern representing the transport container (400) using the image data; • determining (105) a deviation quantity representing a deviation of the contour pattern from at least one reference pattern, wherein the at least one reference pattern represents an empty transport container (400); • Outputting (107) a signal if the deviation quantity meets a predetermined criterion. The method (100) of claim 1, further comprising: Acquiring image data of an additional area of the transport container (400); • determining an additional contour pattern representing the additional area based on the image data; Wherein a reference pattern of the at least one reference pattern has the additional contour pattern. Method (100) according to claim 1 or 2, Wherein the transport container (400) is arranged to capture (101) the image data in an image capture area, the image capture area defining an image background of the transport bin (400); and Wherein a reference pattern of the at least one reference pattern has a contour pattern representing the image background. The method (100) according to one of claims 1 to 3, wherein the predetermined criterion represents an empty transport container (400) and the signal comprises an empty recognized signal. The method (100) of one of claims 1 to 4, wherein outputting the signal comprises issuing a prompt, and
wherein a POS system process goes into a wait state until an input to the prompt occurs. The method (100) of any one of claims 1 to 5, further comprising: Determining color information based on the image data; wherein the deviation amount further represents a deviation of the color information from reference color information, wherein the reference color information represents an empty transport container (400). The method (100) of any one of claims 1 to 6, further comprising: Determining topography information based on the image data; wherein the deviation amount further represents a deviation of the topography information from reference topography information, wherein the reference topography information represents an empty transport container (400). The method (100) of any one of claims 1 to 7, further comprising: Determining depth information based on the image data; wherein the deviation amount further represents a deviation of the depth information from reference depth information, wherein the reference depth information represents an empty transport container (400). Method (100) according to claim 8,
wherein determining the depth information comprises acquiring (101) image information of an additional region of the transport container (400) through the region. The method (100) according to one of claims 1 to 9, wherein the contour pattern represents at least one of the following: A grid structure of the transport container (400); A disturbance caused by an article when the article is placed in the transport container (400); and or An image background which is defined by an image capture area in which the capturing (101) of the image data takes place. Device for computer-aided empty identification of a transport container (400), the device comprising: an image acquisition optical system (202) for acquiring image data; a data memory (204) for storing at least one reference pattern and / or in which at least one reference pattern is stored, the reference pattern representing an empty transport container (400); a processor (206) configured to perform the following method (100): • acquiring (101) image data of a transport container (400) by means of the image acquisition system; Determining (103) a contour pattern representing the transport container (400) using the image data; • determining (105) a deviation quantity representing a deviation of the contour pattern from at least one reference pattern, wherein the at least one reference pattern represents an empty transport container (400); and • Outputting (107) a signal if the deviation quantity meets a predetermined criterion.

Images