Classifications

• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
  • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
  • B07C5/00—Sorting according to a characteristic or feature of the articles or material being sorted, e.g. by control effected by devices which detect or measure such characteristic or feature; Sorting by manually actuated devices, e.g. switches
  • B07C5/34—Sorting according to other particular properties
  • B07C5/3404—Sorting according to other particular properties according to properties of containers or receptacles, e.g. rigidity, leaks, fill-level
  • B07C5/3408—Sorting according to other particular properties according to properties of containers or receptacles, e.g. rigidity, leaks, fill-level for bottles, jars or other glassware
• • G—PHYSICS
  • G05—CONTROLLING; REGULATING
  • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
  • G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
  • G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
  • G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
  • G05B13/027—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion using neural networks only
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F18/00—Pattern recognition
  • G06F18/20—Analysing
  • G06F18/24—Classification techniques
  • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
  • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
  • G06F18/24133—Distances to prototypes
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/0002—Inspection of images, e.g. flaw detection
  • G06T7/0004—Industrial image inspection
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/70—Determining position or orientation of objects or cameras
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/10—Terrestrial scenes
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
  • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
  • B07C2501/00—Sorting according to a characteristic or feature of the articles or material to be sorted
  • B07C2501/0063—Using robots
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10141—Special mode during image acquisition
  • G06T2207/10144—Varying exposure
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/10—Image acquisition modality
  • G06T2207/10141—Special mode during image acquisition
  • G06T2207/10152—Varying illumination
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20081—Training; Learning
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/20—Special algorithmic details
  • G06T2207/20084—Artificial neural networks [ANN]
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T2207/00—Indexing scheme for image analysis or image enhancement
  • G06T2207/30—Subject of image; Context of image processing
  • G06T2207/30108—Industrial image inspection

Definitions

• the deep neural network can be trained with a training data set with images of standing and fallen and / or damaged containers, so that the deep neural network uses the training data set to develop a model to match the standing, fallen and / or damaged containers of the container mass flow from one another distinguish.
• the deep neural network can be trained with a large number of different cases, so that the evaluation is largely independent of the container type and / or environmental influences.
• the training data set can include images of containers of different sizes, orientations or positions.
• the images of the training data set can be recorded with the at least one camera. It is conceivable that this takes place in a test system or directly on site at an operator of the beverage processing system. It is also conceivable that the manufacturer of the beverage processing system creates a database with images of standing and fallen and / or damaged containers in order to then use them with the training data set.
• the images of the training data set can be automatically duplicated in order to create additional images with additional combinations of standing and fallen and / or damaged containers. As a result, the effort involved in creating the training data set can be reduced considerably. It is conceivable that image sections with a standing or fallen and / or damaged container are created during the reproduction. The image sections can come from an original data set. It is conceivable that the image sections are rotated and / or enlarged individually during the reproduction. It is also conceivable that at least one exposure parameter is changed in the image sections during the reproduction. The image sections can then be reassembled to form the images of the training data set. As a result, a very large number of different images of the training data set can be provided using a few original images.
• the exposure parameter can mean a brightness and / or a contrast of an image section.
• the device offers the advantage of active accident and personal protection, since fallen and / or damaged containers do not have to be manually removed from the container mass flow by the operator. This is all the more true since the containers in the mass container flow are subjected to dynamic pressure among one another and intervention by the operating personnel to remove a container as a result of the sudden relief of the container flow harbors the risk of accidents such as crushing and cutting.
• the device for identifying the fallen and / or damaged container in the container mass flow can be arranged in a beverage processing system. It is conceivable that at least one container treatment machine is arranged upstream and / or downstream of the conveyor. In other words, the conveyor can connect two container handling machines to one another.
• FIG. 3 shows an exemplary embodiment of a method according to the invention for recognizing fallen containers as a flow chart
• FIG. 4 shows an exemplary embodiment of a section of the method from FIG. 3 for training the deep neural network
• the camera 6 is arranged on the conveyor 5, which detects the standing containers 2 and the containers 3 that have fallen over from above at an angle.
• the arrangement of the camera 6 is only shown here by way of example. It is also conceivable that there are several cameras that look obliquely from above in the same direction or in opposite directions. An arrangement is also conceivable directly from above, perpendicular to a transport surface of the conveyor 5.
• the neural network 71 is designed to recognize and localize the fallen containers 3. On the basis of the evaluation, the fallen containers 3 can then be removed from the conveyor 5 with a switch (not shown here) or by means of a gripping arm.
• FIG. 2 two exemplary images 11 and I2 of the image data stream output by camera 6 from FIG. 1 are shown.
• the training data set can be obtained from a set larger than 1000, preferably larger than 5000 and particularly preferably larger than 10000 images.
• step 120 the containers 2 of the container mass flow are transported standing on the conveyor 5. It can occasionally happen that one of the containers 2 falls over and then lies on the conveyor 5 as a fallen container 3.
• step 111 images of different container types and / or different lighting conditions are acquired. It is conceivable, for example, that this is done on a test system or that images of the container mass flow M of various beverage processing systems are collected in a database.
• step 112 the images are scaled to a standard size. This enables them to be evaluated uniformly.
• the fallen and standing containers 2, 3 are marked and classified. This can be done manually, semi-automatically or automatically. For example, this can be done manually by an operator on a screen or with a particularly computationally intensive image processing algorithm.
• the marking can be, for example, a surrounding box and the classification can be a container type or a container size.
• step 114 the images are automatically duplicated in order to create further images with additional combinations of standing and fallen containers 2, 3.
• image sections are first created with one standing or one overturned container 2, 3, which are then rotated and / or enlarged individually for reproduction. It is also conceivable that the exposure parameters of the image sections are changed during the reproduction. The image sections can then be put together in the most varied of combinations as further images, from which the training data set is then created in step 115.
• step 116 features are automatically extracted by means of the deep neural network 71.
• a multi-stage filtering process for the training data set is used. It is conceivable that edge filters or the like can be used to extract the outer boundary of each individual container 2, 3.
• the extraction of features here can very generally mean a method for recognizing and / or localizing distinguishing features of the overturned containers 3 compared to the standing containers 2 in the images of the training data set.
• this can also be done manually by an operator.
• the extracted features can include a container closure, a contour of a standing or fallen container 2, 3, a container label and / or the light reflections.
• the extracted features can each include a feature classification, a 2D and / or 3D coordinate.
• the deep neural network 71 is trained with the training data set.
• images of the training data set with the extracted features and the associated markings and classifications of the fallen and standing containers 2, 3 are iteratively given to the deep neural network 71. From this, the deep neural network 71 develops a model in step 118 with which the fallen and standing containers 2, 3 can be recognized.
• the model can then be verified by means of the training data set without specifying the markings and classifications. A comparison is made as to whether the deep neural network 71 actually recognizes the previously specified markings and classifications in the training data set. Likewise, further images with fallen and standing containers 2, 3 can be used for this purpose, which the deep neural network 71 was not trained.
• the substep 140 of the method 100 from FIG. 3 for evaluating the image data stream with the deep neural network 71 is shown in more detail as a flowchart.
• step 142 The features are then extracted in step 142. This takes place in the same way as described in step 116 with reference to FIG.
• the deep neural network then recognizes the orientation and the location of the respective container 2, 3 in step 143 and indicates a probability as to whether this container 2, 3 is transported lying or standing on the conveyor 5.
• This information is then visualized in step 144 and output on a screen in accordance with FIG. 2. In this way, an operator can check whether the recognition is proceeding properly.
• a signal is output in step 145 in order to remove it from the conveyor 5, for example with a switch or a gripper arm.
• the container mass flow M is recorded as an image data stream with the at least one camera 6 and the image data stream is evaluated by the image processing unit 7 with the deep neural network 71, the images of the image data stream can be evaluated on the basis of previously learned empirical values from the deep neural network 71, around the standing and to classify fallen container 2, 3 respectively. Because it is possible to train the deep neural network 71 with images of the most varied of container types and / or environmental conditions, it is no longer necessary to adapt the evaluation of the image data stream in the specific application. Consequently, the method according to the invention is particularly flexible and easy to use.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Artificial Intelligence (AREA)
• Evolutionary Computation (AREA)
• Multimedia (AREA)
• Data Mining & Analysis (AREA)
• Quality & Reliability (AREA)
• Bioinformatics & Cheminformatics (AREA)
• Bioinformatics & Computational Biology (AREA)
• Evolutionary Biology (AREA)
• Life Sciences & Earth Sciences (AREA)
• General Engineering & Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Medical Informatics (AREA)
• Software Systems (AREA)
• Automation & Control Theory (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=73455690

Family Applications (1)

Country Status (5)

Family Cites Families (14)

• 2019
  • 2019-12-03 DE DE102019132830.6A patent/DE102019132830A1/de active Pending
• 2020
  • 2020-11-16 WO PCT/EP2020/082172 patent/WO2021110392A1/de unknown
  • 2020-11-16 EP EP20808055.6A patent/EP4069440A1/de active Pending
  • 2020-11-16 CN CN202080083581.3A patent/CN114761145A/zh active Pending
  • 2020-11-16 US US17/756,744 patent/US20230005127A1/en active Pending

Also Published As

Similar Documents

Legal Events