WO2008107150A1 - Monitoring system, in particular for analyzing the fill level of shelves - Google Patents

Monitoring system, in particular for analyzing the fill level of shelves Download PDF

Info

Publication number
WO2008107150A1
WO2008107150A1 PCT/EP2008/001674 EP2008001674W WO2008107150A1 WO 2008107150 A1 WO2008107150 A1 WO 2008107150A1 EP 2008001674 W EP2008001674 W EP 2008001674W WO 2008107150 A1 WO2008107150 A1 WO 2008107150A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image processing
preceding
computing device
contour
Prior art date
Application number
PCT/EP2008/001674
Other languages
German (de)
French (fr)
Inventor
Joachim Ihlefeld
Amer Tarraf
Original Assignee
Baumer Electric Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to DE102007010663 priority Critical
Priority to DE102007010663.9 priority
Application filed by Baumer Electric Ag filed Critical Baumer Electric Ag
Publication of WO2008107150A1 publication Critical patent/WO2008107150A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00973Hardware and software architectures for pattern recognition, e.g. modular organisation

Abstract

The invention relates to an optical monitoring device, in particular for monitoring inventories. The monitoring device comprises a plurality of interconnected digital image processing devices, said image processing devices each having one or more image sensors and one or more segmentation units, and wherein said image processing devices are interconnected to a first processing unit. The segmentation units conduct a preprocessing of the image information, with the vectorized information calculated by the segmentation units, in particular object graphics, being transmitted to the processing unit.

Description

Monitoring system, in particular for analysis of the level of shelves

description

The invention relates generally to optical monitoring systems. In particular, the invention relates to an optical system for the monitoring of stock levels.

The monitoring of large volumes is often realized according to the prior art video surveillance systems, which consist of camera systems with pan / tilt heads, controllable zoom lenses and multiplexers. Increasingly consists the demand for automation in the areas of security, point of Saie and others.

From WO 2007/006587 a merchandise shelf, merchandise containers and methods to control the removal of goods are known. With a light sensor, the amount of light is measured in a merchandise display or merchandise container. A connected to the light sensor evaluation determines the change in the amount of light. If goods are removed, the detected light amount changes. Is taken in case of theft, for example, the entire contents of a container, a significant change in the amount of light is clear and there is an alarm. However, a quantitative analysis of the level does not allow this system.

Another embodiment of the monitoring system is based on an infrared LED array. The light emitting diodes are successively driven and the associated detected light signal. The stock can be analyzed here on the basis of determining whether a product interrupts the beam path between an LED and detector.

In both arrangements, special shelf systems are required. If the system is to be introduced in a retail point of sale, a conversion of the shelf systems is accordingly required.

DE 100 13 092 Al also discloses a storage rack having a plurality of superimposed support beams for supporting storage goods carriers that are conveyed by an automatic loading and unloading device. At least one digital camera is provided that detects the recorded in the storage goods carriers stored goods and stores the digital information in a computer. The image information can then be displayed on a monitor.

To get an overview of the stock here, a visual inspection of the shelves is provided by viewing the monitor image.

It would be desirable contrast to automate statements about the stock at least partially, without an expensive remodeling of rack systems is required. This object is achieved by the subject matter of the independent claims. Advantageous embodiments and further developments of the invention are specified in the respective dependent claims. With the invention, an easily installed or nachzurüstendes system is provided. is an idea to use a crosslinked system with image sensors, wherein data reduction is performed by pre-detection sensor in proximity.

According to a first aspect of the present invention, such a monitoring system, or a monitoring device is provided which at least one, preferably a plurality of digital image processing systems or image processing devices comprising. The image processing apparatuses each include one or more image sensors and one or more segmentation units. The one or more image processing apparatuses are provided with a first

Computing means, which preferably comprises a central computer and / or multiple turn, networked computers, networked. In decentralized computing devices, especially the segmentation pre-processing units of the image information is carried out so that not the complete image data, but above all vector information, in particular object graph is transferred to the computing device. Of course, but also an image transfer of complete or partial images can also be provided, as requested by the computing device, if these data are needed. The minimum pre-recognition of contour points can be made here, at least locally. Preferably, however, further steps of image processing are performed by the networked computing device having the image processing devices. Particularly suitable for distributed computing is in particular the combination of contour points for contours. The data from these contours can then be transmitted over the network, creating a very substantial reduction of the data stream is already being achieved.

The transfer of information between the image processing devices and the computing device can be performed by a radio network, LAN, or optical network.

The corresponding method for monitoring sales storage or storage space, which is feasible by means of this monitoring device based on the fact that a plurality of networked digital

Image processing devices by one or capture images of the monitored zone plurality of image sensors, the image processing apparatuses each include one or more segmentation units, and communicate via a network with the first computing device, and wherein the segmentation units carry out a preprocessing of the image information, and wherein the image processing apparatus of the segmentation units transferred calculated vector information, in particular object graph to the computing device via the network.

Specifically, the image processing devices may also be further configured to each have a fixed-programmed or to process a transmitted from the computing device discovery job data, and then to store image sections with objects locally and finally to calculate an object graph and this to the first computing device, such as a to transfer central computer. The discovery job record firstly comprises according to a further regions having a predetermined image statistics, secondly, a geometric description of the focal points of the regions, and thirdly a priority description. As image statistics values, in particular values ​​for at least one of the attributes color, gray scale variance, directions, surface contours and contour features such as radii of curvature, contour lengths are, etc ..

According to a further aspect of the invention, an image processing device for analyzing the inventory or level of storage systems is provided which comprises at least a camera and a calculation means for evaluating the image data of the camera, the

Calculating means segmentation means by which a attributiertes contour image is generated, wherein the computing means is adapted to perform in the contour image of a search for digital lines forming the boundaries of storage areas, and wherein the computing device is further adapted to one or more limitations adjacent surfaces which free storage areas represent to determine. The search for the stretching may preferably locally but alternatively also in the central computing device in the image processing devices.

The determination of the adjacent to the contours of the edges of the bearing surfaces land may be included in a simple manner by the color of pixels or by the homogeneity of the gray or color values ​​of the land.

In order to reduce the number of cameras required for the detection of the relevant area, a pivoting or rotating device to pivot or rotate a camera may be provided to successively capture a plurality of shelf spaces with the camera.

Alternatively, or in addition to a pivoting device, a multiplexer may be provided to switch between cameras permanently installed a number. In both cases, a computing device is preferably provided which forms for the determination of contours for each image from the position of long approximately parallel digital lines and the positions of detected Labein as price tags, a grid and constituting the screen information of adjacent images to form an overall grid. Then, the length of visible free storage space can be determined and from this calculates the level of an item of goods.

Generally it is favorable to subdivide the monitoring region a of the image processing devices in a plurality of shots, spatially different areas. In particular, at positions of the sensor or sensors above the areas to be monitored a detection with a single recording would lead to large perspective distortion, which complicate the recognition of the structures of the bearing system.

Have the plurality of image processing devices

Image pickup sensors on is preferably carried out for receiving the entire detected by the sensors monitoring region a control of the sensors in the time multiplex. In this way, certain processing steps such as determining the contour points can already be carried out while images of the surveillance area are picked up by other sensors. In addition, a time-multiplexed operation is very favorable because less memory is required.

The system is also highly energy efficient due to the local pre-processing in the image processing devices. The devices can hence have through local energy supplies. The image processing devices can thereby carry out an additional resource optimization, and include a power management. Local, in particular off-grid power supplies offer the advantage of a particularly simple retrofitting of existing storage systems, since the laying of power cables to the individual decentralized devices can be eliminated. Especially maintenance such power supplies, which can independently produce their supply electricity. Thought is here, among others, power supply with solar cells. With a power management can then be loaded into rest accumulators that of the power supply during operation

Image processing devices provide or at least assist.

According to a further development of the invention it is provided that the computing means comprises means for detecting contours which contour points is calculated with subpixel accuracy from the image data. Calculating with sub-pixel accuracy by offsetting the color or gray values ​​of a plurality of adjacent pixels, and by which a determination of the location of a contour point. In this way, quantization errors are reduced. Otherwise, a per se straight contour would be caused represented by the raster of pixels as a zigzag line. By using a sub-pixel grid this can be avoided in a simple manner, which facilitates precisely the recognition of straight contours as they are typically present in an image of a shelf.

To filter out the contour points from the image data, preferably a device for determining and processing of contour points is provided which comprises at least one integrated hardware unit which is adapted to scan the data stored in the image memory data of a digital image and to determine contour points with subpixel accuracy, and as a continuous store list data in a memory. a contour can then be formed from a set of contour points. In other words, belonging to a contour contour points are identified and summarized. This can particularly preferably an arithmetic unit may be provided which is adapted to determine from the stored in the memory list data by using an arithmetic unit, the connection probabilities between each pair of contour points in consideration of the distance between the points, and wherein at least a classifier is provided in the integrated image processing apparatus, which is composed of sets of calculated connection probabilities subsets with at least three connection probabilities for possible connections between at least three adjacent contour points, one of which is a previously determined central contour point, selects and sorted those adjacent to the central contour point contour point for each subset, a possible link with the lowest probability has an adjacent contour point if the link does not connect two adjacent to the central point and the points Ansc STATEMENTS it enters with connectors in a contour point list with non-culled contour points, which characterize the remaining connections to the central point. This recognition and summary of contours is particularly simple and can therefore also good pure hardware-based, for example, in an FPGA or ASIC device are performed.

The contours detected may then by means of the computing device based on determined from the image data contours such contours, preferably be filtered out based on their length and shape, as well as to these contours perpendicular contours which bearing surfaces, or their edges and edges of the vertical structures of the shelf represent , By means of the computing device can then be checked the conformity of a grid formed with these contours with a digital model of the shelf.

An assignment of the different storage areas of a storage system, such as a display shelf, or

be able to carry luggage storage areas at an airfield on specific articles stored there, is a device for reading bar code or OCR information on Labein and a computing device for assigning the information of the label to a determined frame of a supervised camp favorable. Label may be, for example, the price and product signs on the front of a store shelf. If this information is called in a fully automatic assignment of a level of a storage area to a certain stored there or to be stored items can then be taken. When the grid of the bearing system is determined and an assignment to the content of the label on the storage system, the compartments can, or in general be associated with the various goods items stored or generally the storage locations of the storage system. The information about the level then permits a determination or at least an estimate of the quantity of goods

Based on the detectable by the monitoring device

Changes in inventories can also be determined customer current characteristic sizes for applications in the POS area. Thus, the positioning of the products within the shop premises can be optimized. The invention will be explained in more detail below with reference to exemplary embodiments and with reference to the accompanying drawings. The same reference numbers refer to like or similar parts. Show it:

Fig. 1 is a diagram of an embodiment of the system architecture of a monitoring device according to the invention,

Fig. 2 is a view of an image processing apparatus,

Fig. 3 is a recording of a shelf,

Fig. 4 is a contour point image of the receptacle shown in Fig. 3,

Fig. 5, the contour dot image with the determined frame of the shelf,

Fig. 6 is a diagram of classification of empty areas, and

Fig. 7 is a diagram of the system architecture of another embodiment of a monitoring device.

By arranging a plurality of cameras, it is possible to reach without mechanically moving parts by electronic multiplexing of the sensor chip is an enlarged scanning area. Here, each chip has its own, cost-effective in most cases, lens, for example, based on DOE (diffractive optical element). After multiplexing multiple images that are not aligned arise, that is, the images can not be made up without much effort into a large image. Another problem is the high wiring costs.

The solution now is to perform a data reduction by pre-recognition sensor in proximity. Here, first contours caused by objects that are then classified. This example is possible to detect the parked baggage at airports within each camera image. An example of a possible system architecture of a monitoring device designated as a whole by reference numeral 1 is to shown in Fig. 1. The monitoring apparatus 1 is divided into a computing device, here in the form of a central computer 10, which is connected to a plurality of image processing devices. 3 For purposes of clarity in FIG. 1, only a single image processing apparatus 3 is shown.

The image processing system are networked. For this purpose are both video, and LAN technologies and / or optical technologies. Accordingly, the

Image-processing means 3 is connected to the central computer 10 over network links. 9 These network connections 9 may be optical data links, radio links, for example via a wireless or wired connections and one LAN. Particularly advantageous is the cross-linking via wireless networks such as wireless networks, such as Blue Tooth.

Communication can take place with known network protocols. Thought is here, among others, TCP / IP. The

Communication of image processing apparatuses 3 with the central computer 10 is performed by a network controller. 7

The image processing apparatus 3 includes an image sensor unit 4 with a plurality of image sensors, such as CMOS sensors. The sensor chip of the image sensor unit 4 are staggered in time controlled by a multiplexer and successively take so that images of different, in particular overlapping portions of the detectable by the image processing device 4 portion of the surveillance area on. The image data of the image sensor unit 4 are supplied to an image segmentation unit. 6 In this preferably realized with FPGAs and / or DSPs image segmentation unit contour points are determined and summarized belonging to a contour points in contour records in vectorized form. The contour points are thereby recognized by the gradient of color or gray scale gradients. The position assignment is carried out with sub-pixel accuracy. This is possible because the calculation of a contour point more adjacent pixels are to be evaluated. The position of the edge is due to the involvement of several points with an accuracy which is generally higher than the accuracy of the grid of pixels. The detection, contour points which belong to a common contour, made according to a very simple and effective algorithm which can easily be implemented in an FPGA or ASIC without software assistance. For this purpose, the contour points are first stored as a sequential list data in a memory, the connection probabilities between each two contour points are then determined based on the distance between the points from the data stored in the memory list data, the connection probability decreases with increasing distance. It can be taken into account additional parameters such as the associated contour points toward the contrast and / or color of the points. By means of a classifier are then calculated from a set of connection probabilities subsets with at least three

Connection probabilities for possible connections between at least three adjacent contour points formed, one of which is a previously determined central contour point.

For each of the subset each one adjacent to the central contour point contour point is sorted out, the adjacent a possible link with the lowest probability of connection to a

comprises contour point if the link does not connect two adjacent points to the central point. The remaining contour points are added thereto in the terminal in a contour point list with connectors which characterize the remaining connections to the central point.

The objects to be found here are described by a search order. This includes, for example, information on the object size, color, shape features. The search order is loaded on each image processing system as a model.

For POS applications, a meaningful alert may initially carry out a scanning detection consist by the shelves are detected as long parallel digital lines.

Subsequently, the labels are detected, here by the shape feature "rectangle of a specific color" with OCR (Price). Due to the location of these objects, the labeled Raster a filled shelf is illustrated. In a third step the distance of the goods from the front edge of the shelf and thus the filling level is determined. To this end, more a priori information on the central computer 10 can be obtained and used as grid location and price are known. According to one embodiment of the invention each Bildverarbeitunssystem produced (scan) or controlled by a message sent by the computing unit, in particular of the central computer or networked computer interconnection request a respective data record, which describes the condition of the inspected environment at defined time intervals. These data sets are acquired by a central computer and processed to an overall statement. A request to create a

Record can be triggered by various events. In particular, the recording of the monitored zone can be carried out at predetermined time intervals and / or to a request, the request is triggered by a detected event from the monitoring device. Such an event can occur either by one or more of

Image processing devices independently, and / or are recognized by the computing device. triggered by the event is a search of one or more of the image processing devices may then be performed. In the simplest case, the events predetermined time points, so that is sampled at defined time intervals as described above. There are alternative or in addition, however, other events, such as the

Consignments of goods possible. Is a delivery of goods from the computing device and / or at least one of the image processing devices registered requests or search requests to the image processing devices over the network can be sent then, or will run unattended consequence of the occurrence of the image processing devices. Yet another possible event is a predicted time. So based on the detected by the computing device and / or the image-processing means changes in inventory a forecast can be created when a particular stock item is expected to be out or fill. For predicted time or at a different time in predefined time interval to the predicted time, the request for creating a data record, or a search operation by the computing device to the image processing apparatus can then be sent. For example, if the lack of a stock item for a specific time predicts an inquiry a few hours or days before that date can be sent so that it is possible to replenish the stock, such as a shelf in a shop in time.

For some applications, the statements at a relatively large time interval in the POS need such area, it is also particularly advantageous when the image processing system includes a Power Management System. Power management can be taken for example, from the network controller. 7 The Power Management System accumulates energy during the resting phase (eg solar energy) and then starts a processing procedure, for example, at intervals of one hour for about I s runs. This can be saved particularly cost all wiring.

Fig. 2 shows an embodiment of a view of an image processing apparatus 3. The

An image processing apparatus 3 is accommodated in a housing 13, for example, as shown in Fig. 2, may be spherical, the housing in this or a similar form can then be particularly vorteihaft additionally used as an advertising space.

The housing 13 is fixed to the ceiling of the storage building by means of a suspension. 11 In the upper part of the housing 13 solar cells 12 are mounted, which produce the energy for the mains-independent power supply of the image processing apparatus. 3 In the lower part, the cameras 14 are arranged so as to detect respective different, but overall overlapping surrounding regions. As cameras, for example, CMOS sensors can be used with diffractive optical lenses or other simple lenses.

Since by the distributed preprocessing of the image data is subject to an energy of generally not more than 1 Joule, preferably only 0.5 joules per image acquisition and processing in the image processing apparatus 3, an external power supply via solar cells themselves in closed storage spaces, such as inside a

Shop premises possible. In contrast, the image data would be completely transferred, so a much higher energy consumption would be generated solely by the network interface.

Subsequently, the determination of the level of shelves is described in detail. This procedure can also to other warehouse management systems, such as luggage or package storage and transport are applied accordingly in airports and logistics companies.

To determine the level of shelves in department stores, supermarkets, warehouses, and other facilities is recorded with the system for analysis of the level of shelves the removal of products by cameras and analyzed with image processing means. As cameras digital color cameras are preferably used.

For this purpose, at one or more points, preferably installed on the ceiling of a hall, cameras, which are mechanically moved around or the sample as in the example shown in FIG. 2, with a plurality of image sensors.

Each of the cameras can for example be continuously tilted and rotated so that a full frame rate of the region to be inspected. Further, the lens direction of each camera between a plurality of positions and forth. At the output of each camera thus digital image sequence with different created for each position

Zoom position and / or focusing. A single frame of a shelf in a sales area, as it is taken with a camera of the image processing apparatus shown in FIG. 3. The shelf has bearing surfaces, of which the horizontally extending front edges are visible especially 16. Of the supports of the rack vertically extending edges

18 can be seen in the shot. are at the edges 16 Label

19 attached, which characterize the products of the individual compartments of the shelf. On the shelves of the shelf goods 22 are arranged. In some places of the shelf empty spaces 20, 21 can be recognized due to a non-full level.

The digital image sequence having a structure shown in Fig. 3 picture will be a hardware-based

Image processing unit processed to a preferably subpixelinterpoliertes and attributiertes contour image. In particular, these processing may be performed in the image segmentation unit 6 of the image processing device 3 of FIG. 1. The contour image generated from the map shown in Fig. 3 Recording with contours 15 is shown in Fig. 4. This contour image, first a general search for digital lines whose inclination to the horizontal within a predetermined threshold range is located. Particularly long, nearly horizontal lines represent bearing surfaces on the shelves or their front edges to also exist long vertical lines. The summary of these lines forms a grid 23, which is shown in Fig. 5 with thick lines. This grid is matched with a digital model of the shelf, so that only the relevant contours remain.

If no grid structures are found according to one or more predefined models, then the image is discarded. This is particularly the case when the recording parameters are focused unfavorable, or if no suitable object in the field of view.

The grid elements, as shown in Fig. 5, two-dimensionally numbered, for example, show AO, Al, A2, A3, ..., Bl, B2, B3, ... horizontal raster elements. The division with vertical grid elements takes place in each case by digital lines between two adjacent horizontal grid elements. the position of the label 19 described by the search order is for the separation of the compartments, typically characterized by rates used. As can be seen from Fig. 5, the raster elements representing single trays or shelves of the rack.

The images have an overlap region. If a relationship between two successive images, for example by correlation calculation, including a

Masstabskorrektur is determined, then a inheritance of numbering. After processing of an image sequence, which describes a complete shelf or a shelf group, there is a uniform grid for this processing step. The grid is located on a shelf defined by the mold surface, hence the image processing system a list of well-defined control points (for example, by the length and height of the shelf, spacing of the shelves, OCR font size, etc.) are known. different for each subject parallax is corrected mathematically by evaluating the control points, so that standardized measurements for the level of subjects, such as the free front surface (Figure 5) can be determined.

starting from each grid detached surfaces 25, searched in the vicinity of the pitch contour 26, Fig. 4 and 5 which correspond to the color of the shelf space. The surfaces are characterized by a comparatively homogeneous color or monochrome value distribution, so that found within the surfaces 25, 26 no or few contours. These surfaces 25, 26 are referred to as gaps. The surfaces are then represented by their contours and circumscribing assignment to the raster position. As additional information the distance to the nearest vertical shelf supports can be specified.

The contour information of the gaps 25, 26 are then classified as needed, of particular importance are the surface feature and the width of the gap.

Fig. 6 shows a diagram of a classification for the two in Fig. 4 and 5 shown surfaces 25, 26. The two surfaces 25, 26 is a receiving timing assigned here for example 11:37. In addition, an allocation to the grid elements is made. Accordingly, 25 is part of the area for grid element AO and the surface 26 to the grid member BO. In addition, coordinates of the areas are determined. These location coordinates and the area may belong. The processing up to this point, as shown in Fig. 6, within the decentralized

done image processing device. 3 The thus vectorized data is then transmitted via the network connection. 9 In the computing device, here again an example of a

Central computer 10 then a further processing. Here can be included more information about the camp ( "POS Know How"), as well as statistical information that facilitates identification of the empty spaces. Relevant images can be obtained and filtered.

A further embodiment of the structure of a monitoring device according to the invention is shown in Fig. 7. The data provided by a digital color camera 14 with lens 17 are processed by a contour point processor FEX the image segmentation unit 6 and then linked to concatenated contours. The contours so generated are preferably vectorized by the hardware of FEX, or the image segmentation unit 6 with a curvature analysis (Curvature analysis). It arises from the entire image a list of digital track and circuit elements. This list is searched for approximately horizontal, long lines and long nearly vertical lines.

Subsequently, in the integrated computer of the image segmentation unit 6, the comparison of the extracted long lines with the shelf model and the projection conditions so that real parallel or perpendicular contours that correspond to the shelf model to be found. Subsequently, by contour processing, the detection of label 19, by the list of lines and circular segments corresponding to the search order for example to smaller rectangles of a predetermined size and color as well as numerals (in a predetermined by the search region, these are, for example, depending on the shelf is always in near the shelves) is sought in relation to the shelf grid. In addition, the gaps between the objects can be detected as soon as a sufficient object distance is present.

If a sufficient match is found with the shelf model, then indexing is performed and numbering of the grid in two coordinates, and the associated gaps, otherwise the image is discarded.

The List of gaps and the shelf indices (numbers) is continuously to a central computer (not shown in Fig. 7) is transmitted over a local computer network 90 (LAN, wireless, wired and / or optical transmission). In this machine all the data of cameras of the overall system, to which may still include at least one additional communicating via the local computer network 90 image processing apparatus 30, detected and assembled to the overall image of the merchandiser. Preferably, the data are transmitted in a defined in the front view of the shelves within the search order coordinate system. A record of a compartment preferably contains the location of the label, expressed by the coordinate in the grid (such as camera number., AO, compartment 5, OCR = 35.76 EUR, position of holes, free contour, time stamp), to the extent required, what is read contents of the labels (for example price), the position of the gaps and the contours of the free shelf surface 25, 26, Fig. 5, and can be supplemented by additional information on the search order.

The assignment of the rack positions for warehouse management system of the device is preferably performed by teach-in. For this, the existing in each shelf printed and legible prices are used to Labein. This list of prices so determined interactively matched with the warehouse management system on the host computer so that a correspondence list of part numbers and logical rack positions created, supplemented if necessary interactively ambiguities by additional information.

The information thus processed are in a

Shelf layout graphically displayed and output to a master control system, so that a information on current movements and the equivalence of input in the ERP system and in fact excellent prices can be checked.

Claims

claims
1. Monitoring device comprising at least one digital image processing device, wherein the
An image processing device one or more image sensors and one or a plurality of segmentation units, and wherein said image processing apparatus is crosslinked with a first computing device, and wherein the segmentation unit is arranged for pre-processing the image information, and wherein the image processing device is configured by the segmentation unit calculated vector information, in particular object graph be transferred to the computing device.
2. Monitoring apparatus according to the preceding claim, characterized in that the image processing apparatus is arranged, in each case a hard-coded or transmitted from the first computing device discovery job record comprising first regions of a predetermined image statistics, secondly, a geometric description of the focal points of the regions and thirdly
Priority Description to be processed, and then to store image sections with objects locally and finally to calculate an object graph and to transmit it to the first computing device.
3. Monitoring apparatus according to the preceding claim, characterized in that the computing device comprises a central computer or several interlinked computers.
4. Monitoring device according to one of the two preceding claims, characterized in that the search Order record image statistic values ​​for at least one of the attributes color, gray value variance directions, area contains.
5. Monitoring device according to one of the preceding claims, characterized in that the image processing device is adapted to the object graph supplemented by values ​​to the
to transmit reception conditions, feature vectors, or the recording time to the first computing device.
6. Monitoring device according to one of the preceding claims, gekennzeichnnet by a radio network, LAN, or optical network for transmitting information between the image processing device and the computing device.
7. Monitoring device according to one of the preceding claims, in particular for analysis of the inventory or level of storage systems, comprising at least one camera having a second calculating means for evaluating the image data of the camera, wherein the computing means by means of a
Segmentation means generates a attributiertes contour image, and wherein the computing means is adapted to perform a search for digital lines in the contour image, which the
form boundaries of storage areas, and wherein the computing device is further adapted to determine at one or more boundaries of adjacent homogeneous areas representing free bearing surfaces.
8. Monitoring apparatus according to one of the preceding claims, characterized by a pivoting or rotating device to pivot or rotate a camera for successively detecting a plurality of surrounding regions, or a multiplexer for switching a number of stationary cameras, wherein a computing device is provided, which for the determination of contours for composed each image from the position of long approximately parallel digital lines and the positions of detected Labein as price tags, forms a grid and the screen information of adjacent images to form an overall grid and determines the length freely visible bearing surfaces and therefrom calculates the level of an item of goods.
9. Monitoring apparatus according to one of the preceding claims, characterized in that comprising a plurality of image sensors, at least an image processing apparatus, which are driven in time division multiplex.
10. Monitoring device according to one of the preceding claims, characterized in that the local image processing apparatus, and in particular mains-independent power supplies having, preferably solar cells.
11. Monitoring device one of the preceding claims, characterized by a computing device with a device for determining contours which contour points is calculated in accordance with sub-pixel accuracy from the image data.
12. Monitoring device according to one of the preceding claims, characterized by means for determining and processing of contour points, which comprises at least one integrated hardware unit which is adapted to scan the data stored in the image memory data of a digital image and to determine contour points with sub-pixel accuracy, and as a continuous list data in a memory store, and wherein said image processing device comprises an arithmetic unit which is set up to determine from the stored in the memory list data by using an arithmetic unit, the connection probabilities between each pair of contour points in consideration of the distance between the points, and wherein at least in the integrated hardware unit a classifier is provided, which consists of sets of calculated probabilities compound subsets with at least three connection probabilities for possible connections between at least three adjacent contour points, one of which is a previously determined central contour point, selects and sorted those adjacent to the central contour point contour point for each subset, of a possible link with the lowest
having connection probability to an adjacent contour point if the link does not connect two adjacent points to the central point, and it enters the terminal into a contour point list with non-culled contour points with connectors which characterize the remaining connections to the central point.
13. Monitoring device according to one of the preceding claims, characterized by a computing device which is adapted in terms of certain of the image data contours such contours, preferably filter out based on their length, as well as to these contours perpendicular contours which bearing surfaces, or their edges and represent edges vertical structures of a shelf, and wherein the computing means is adapted to check the consistency of a grid formed with these contours with a digital model of the shelf.
14. Monitoring device according to the preceding claim, characterized in that the the
Contours of the edges of the bearing surfaces of adjacent land by the color of pixels or by the homogeneity of the gray or color values ​​of the surfaces are detected by the computing device.
15. Monitoring device according to one of the preceding claims, characterized by a device for reading bar code or OCR information on Labein and a computing means for allocating the information of the label to a determined height of a monitored bearing.
16. A method for monitoring sales storage or storage space, feasible receives by means of a monitoring device according to one of the preceding claims, wherein at least one digital image processing apparatus with one or more image sensors images of the monitor area, wherein said image processing device comprises one or more segmentation units, and a network communicating with a first computing device, and wherein the segmentation unit performs a pre-processing of the image information, and wherein the image processing device transmits the calculated segmentation units from the vector information, in particular object graph to the computing device via the network.
17. The method according to the preceding claim, carried out in which the recording of the monitored zone at predetermined time intervals, to a value determined by a forecast time or to a request, the request is triggered by a detected event from the monitoring device.
PCT/EP2008/001674 2007-03-02 2008-03-03 Monitoring system, in particular for analyzing the fill level of shelves WO2008107150A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE102007010663 2007-03-02
DE102007010663.9 2007-03-02

Publications (1)

Publication Number Publication Date
WO2008107150A1 true WO2008107150A1 (en) 2008-09-12

Family

ID=39327058

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2008/001674 WO2008107150A1 (en) 2007-03-02 2008-03-03 Monitoring system, in particular for analyzing the fill level of shelves

Country Status (1)

Country Link
WO (1) WO2008107150A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014181323A1 (en) * 2013-05-05 2014-11-13 Trax Technology Solutions Pte Ltd. System and method of retail image analysis
US10368662B2 (en) 2013-05-05 2019-08-06 Trax Technology Solutions Pte Ltd. System and method of monitoring retail units
US10387996B2 (en) 2014-02-02 2019-08-20 Trax Technology Solutions Pte Ltd. System and method for panoramic image processing

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1996031833A1 (en) * 1995-04-04 1996-10-10 Cowe Alan B Materials monitoring systems, materials management systems and related methods
JP2001088912A (en) * 1999-09-20 2001-04-03 Fujitsu General Ltd Stocktaking managing method and stocktaking system by image recognition
US20030154141A1 (en) * 2001-09-18 2003-08-14 Pro Corp Holdings International Ltd. Image recognition inventory management system
EP1422657A1 (en) * 2002-11-20 2004-05-26 Setrix AG Method of detecting the presence of figures and methods of managing a stock of components

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1996031833A1 (en) * 1995-04-04 1996-10-10 Cowe Alan B Materials monitoring systems, materials management systems and related methods
JP2001088912A (en) * 1999-09-20 2001-04-03 Fujitsu General Ltd Stocktaking managing method and stocktaking system by image recognition
US20030154141A1 (en) * 2001-09-18 2003-08-14 Pro Corp Holdings International Ltd. Image recognition inventory management system
EP1422657A1 (en) * 2002-11-20 2004-05-26 Setrix AG Method of detecting the presence of figures and methods of managing a stock of components

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BROERS ET AL.: "Architecture Study for Smart Cameras" PROC. EOS CONF. ON INDUSTRIAL IMAGING AND MACHIEN VISION, 13. Juni 2005 (2005-06-13), - 15. Juni 2005 (2005-06-15) Seiten 39-49, XP002480146 *
CORSI ET AL: "Smart Sensors" INFRARED PHYSICS AND TECHNOLOGY, ELSEVIER SCIENCE, GB, Bd. 49, Nr. 3, 9. Januar 2007 (2007-01-09), Seiten 192-197, XP005869920 ISSN: 1350-4495 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014181323A1 (en) * 2013-05-05 2014-11-13 Trax Technology Solutions Pte Ltd. System and method of retail image analysis
US10368662B2 (en) 2013-05-05 2019-08-06 Trax Technology Solutions Pte Ltd. System and method of monitoring retail units
US10387996B2 (en) 2014-02-02 2019-08-20 Trax Technology Solutions Pte Ltd. System and method for panoramic image processing

Similar Documents

Publication Publication Date Title
US8430311B2 (en) Systems and methods for merchandise automatic checkout
KR101592889B1 (en) Object matching for tracking, indexing, and search
EP1185953B1 (en) Storage system
US20050154560A1 (en) Real-time prediction and management of food product demand
US8209219B2 (en) Vision-based measurement of bulk and discrete food products
US7693757B2 (en) System and method for performing inventory using a mobile inventory robot
US20070058040A1 (en) Video surveillance using spatial-temporal motion analysis
EP3147841A1 (en) System for monitoring the condition of packages throughout transit
US8494909B2 (en) Automatic learning in a merchandise checkout system with visual recognition
EP0672993A2 (en) Automated apparatus and method for object recognition
US8430312B2 (en) Cart inspection for suspicious items
US8091782B2 (en) Using cameras to monitor actual inventory
US9727838B2 (en) On-shelf tracking system
ES2424314T3 (en) Method of allocation and deduction of the location of items detected by multiple RFID antennas
US20040260513A1 (en) Real-time prediction and management of food product demand
EP2286932A2 (en) System and method for dimensioning objects
US10192189B2 (en) Mobile pickup locations
US7246745B2 (en) Method of merchandising for checkout lanes
US8189855B2 (en) Planogram extraction based on image processing
US8630924B2 (en) Detection of stock out conditions based on image processing
US9015072B2 (en) Method and apparatus for automated inventory management using depth sensing
JP5054670B2 (en) Method and apparatus for detecting a suspicious activity by using video analysis
JP3800257B2 (en) Attention information measuring method and apparatus, and various systems using the same
US20090063306A1 (en) Determination Of Product Display Parameters Based On Image Processing
US20060283943A1 (en) Systems and methods for merchandise checkout

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08716194

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct app. not ent. europ. phase

Ref document number: 08716194

Country of ref document: EP

Kind code of ref document: A1