WO2010034044A2

WO2010034044A2 - Method and system for receiving and/or processing objects

Info

Publication number: WO2010034044A2
Application number: PCT/AT2009/000360
Authority: WO
Inventors: Roland Heiml
Original assignee: Stiwa Holding Gmbh
Priority date: 2008-09-26
Filing date: 2009-09-18
Publication date: 2010-04-01
Also published as: AT507339B1; WO2010034044A3; AT507339A1

Abstract

The invention relates to a method for sequentially receiving and/or processing individual objects (2) from an object quantity (12) moved, in particular continuously, by a conveyor apparatus (3) using a handling apparatus (5), particularly an industrial robot (6), comprising an imaging method carried out on the moved object quantity (12) and an object detection method for localizing, identifying and selecting at least one object (2) which can be received and/or processed by the handling apparatus (5) using the image information of the object detection system (15) processing the imaging method, and receiving and/or processing a selected object (2) with the handling apparatus (5). An image information flow (36) continuously generated by the imaging method is divided into detection sections (28) of the object quantity (12) containing object quantity information, particularly object point coordinates and movement information, and the objects (2) which can be received and/or processed by the handling apparatus (5) are identified and selected by the object detection system (15) for individual detection sections (28).

Description

Method and system for recording and / or editing objects

The invention relates to a method for the sequential recording and / or processing of individual objects from a moving object conveyed by a conveyor device according to the preamble of claims 1, 13 and 21, further comprising a system for feeding and handling and / or processing of objects according to the preamble of Claim 25.

The handling or machining of objects, for example workpieces by means of programmable handling devices, such as industrial robots, has long been used in the manufacture of products. However, due to constant cost pressure and technical advancements, there are ongoing efforts to extend measures for the flexible automation of the handling and processing of objects to other areas of application.

An important approach in this area is the provision of camera-mounted optical camera handling devices and the use of image capture, image recognition, object recognition and other methods related to machine vision or "computer vision".

By equipping or linking a handling device with sensors, a situation-related use of the handling device can take place in addition to or as an alternative to fixed-programmed workflows, whereby the desired flexibility is achieved. When using optical sensors in the form of cameras, the basic sequence comprises the optical detection of a scene in the sphere of influence of the handling device, subsequent image processing, in particular object recognition with an analysis of the scene and an action based on the result of the analysis Non-action of the handling device.

For purposes of handling or processing of objects, it is particularly necessary to record and process their type, their position, their orientation or orientation, as well as quality features or other optically detectable attributes and features, in order to determine the actions to be taken by the handling device set. The collection and processing of the associated extensive information takes place without exception in computing units and memory units comprehensive control and regulating devices that can be structurally, spatially and logically constructed in a variety of ways.

An important field of application of the method according to the invention or of the system according to the invention consists in gripping moving objects and subsequent feeding to a specific processing point or transfer point within a production, transport or packaging installation. Depending on the degree of order of the moving object set or the objects contained therein, partial processes for separating, sorting, orienting, aligning, checking, etc. may be required before the actual feed operation. In many cases, mechanically acting devices are still often used for these subtasks of the feeding technique, but they usually have to be specially adapted to the type and properties of the objects, which is why such solutions are only slightly flexible and always involve a modification of the objects to be manipulated a great deal of effort for setting up, rebuilding, test runs and so on.

A classic example of such a relatively "inflexible" accomplishment of the subtasks of the feeding technique is the use of vibrating conveyors, in the conveying path of which so-called baffles for sorting, sorting, alignment, checking, etc. are arranged Such a feeding solution depends to a great extent on the experience and skill of the employees entrusted with the production or setting of such baffles A major problem with the use of such solutions is that the required fine adjustment in trial runs only after a sufficient number of It is therefore possible to produce and prepare these feeding objects at a later stage in the product development phase, and to thereby delay a series startup, for example Changing the parts to be manipulated, such as a slight change in geometry in many cases, a re-fine adjustment, with the disadvantages described above required.

The advantageous flexibility of a handling device, which is equipped with capabilities for machine vision, is now that the image processing used, in particular object recognition on models contained in the object recognition system to based on editing or hand-to-handle objects and these example. In CAD-based or represented by CAD data object models the object recognition system or the image processing system can be taught virtually at the touch of a button. By means of various methods, the image information supplied to the system is then compared with the object models, and based on this comparison information or statements about the real existing objects are derived, which are used to control and regulate the handling or processing operations.

DE 103 51 669 A1 discloses, for example, a method for controlling a handling device, such as a multi-axis industrial robot, wherein first an expected image of an object in a workspace of the handling device is compared with a real image of the object, then a positional deviation of the handling device is determined and then movements are performed to minimize the positional deviation. This method uses an apparatus for controlling a manipulator such as a multi-axis industrial robot, comprising imaging means for providing a real image of an object in response to a real position of the manipulator, determining means for determining an expected image of the object in response to a predetermined position of the manipulator Handling device and a comparison device for the real and the expected image for determining a positional deviation of the handling device, wherein an output signal of the comparison device for minimizing the position deviation is available.

In one embodiment, an imaging device in the form of a camera is aimed at objects moving by means of a conveyor belt and thereby generates image information which is fed to a determination device. Among other things, the expected image of an object is generated, for example, from CAD data of an object, whereby a comparison possibility is given without the need to take real images of an object. The imaging device in the form of a camera can be arranged both on the handling device, such as at the end of the kinematic chain of the robot or even independently of the handling device about above the conveyor belt. The method disclosed in the DE-Al makes it possible to accelerate the robot programming and forms a possibility of automatic position adjustment of a handling device. On questions of increasing the performance of such a system, ie the number of units per unit time - A - editable or manageable objects, this document offers no concrete approaches or suggestions.

Another system based on the linking of a robot with an image processing system is known from WO 2007/046763 A1. This describes a method and an arrangement for locating and picking up objects, for example from a container or from a conveyor belt. The area in which the objects to be recorded are located is smeared and measured with a line laser scanner, and the converted surface points of the object set are then converted into a virtual three-dimensional surface. This three-dimensional surface is filled with virtual objects that through

CAD data can be represented, whereby the individual objects can be identified and their spatial position and orientation can be determined. Before picking up individual objects, it is checked whether the robot or the handling device can pick up the selected objects without collisions, whereby collisions with the virtual surface of the object set as well as collisions with the container or the conveyor are taken into consideration.

For detecting the surface of the line laser scanner is preferably moved in a linear movement over the container the conveyor; Alternatively, the scanner can also be arranged on the robot arm and the relative movement between the scanner and the amount of object can be effected by the robot arm. By comparing the virtual image of the real object set surface with the virtual objects, the method can also be used to detect faulty objects. While WO-A1 discloses picking up objects from a conveyor belt, it does not disclose a method of picking up objects from a moving conveyor belt.

The object of the invention is to provide a method for the sequential recording or processing of individual objects by means of a handling device which allows, in particular continuously, to identify and record moving objects and in which the achievable performance when handling or editing objects is high in relation to the hardware requirements for such a system. The object of the invention is achieved by a method having the characterizing features of patent claim 1.

By dividing an image information stream continuously generated by the image acquisition method into object quantity information, in particular object point coordinates and movement information, comprehensive recognition sections of the object set and identifying and selecting the objects that can be recorded or processed by the handling device by the object recognition system for individual recognition sections, a virtual image of the continuously recorded real object quantity surface portioned into subsets so to speak, which can be processed with the known from the prior art methods for image processing or object recognition and relatively low demands on the hardware performance of the system. Since the image information stream can be subdivided into recognition sections in a variety of ways, the method can be adapted to a wide variety of requirements or applications.

Since, despite progressive performance increase in the computing speed of systems used for image processing, the computing power due to the enormous amount of data still regularly reaches its limits and thus the handling device has to wait in a sense to the calculation results of image processing or object recognition, there is the division of the image information stream in recognition sections the possibility of dividing the image recognition methods to be applied to the individual recognition sections in parallel to a plurality of arithmetic processes of one or more arithmetic units which can accomplish these relatively "small" arithmetic tasks in a short time, thereby increasing the overall performance of the handling device the Bildinformationsstroms in recognition sections open the possibility to fully evaluate the image information contained therein or even the only partially or not at all in image recognition sections contained in individual object recognition sections. This can be done in such a way that if in a recognition section after a preselectable period of a scheduled calculation period no suitable object can be identified or already a predefined number of objects could be identified, the further image processing for this recognition section is aborted and the thereby freed computing power is used immediately for image processing or object recognition in another recognition section.

The image data continuously generated by the image capture system are buffered in a storage device and are, preferably without data loss, for the purpose of

Image data preparation, subdivision into recognition sections and subsequent object recognition from the storage device readable.

In this case, the memory device can, in particular similar to what is known as a ring buffer, provide a limited number of memory registers for the individual recorded image data, wherein the memory capacity is limited by the memory size or the number of memory modules contained in the memory device to the optical resolution of the image acquisition system, the temporal resolution and the computing speed of the Image data processing is adapted so that the storage limit can only cause loss of data of the object quantity which originate from objects which are already outside the spatial and / or temporal range of the handling device, ie objects which are in any case in the course of the movement of the object quantity can no longer be reached by the handling device. The storage device is further arranged such that the writing of image data by the image acquisition system and the reading and possibly deleting of image data for the object recognition system results in no data loss or a disturbance of the data integrity. For this purpose read and write processes take place atomically.

The division of the image information stream into detection sections takes place in such a way that a defined amount of image data memory content and thus also a specific section of the real, moving object surface surface can be assigned to each recognition section. From the acquired image data or data packets, points of the object quantity surface associated with space coordinates and path information are calculated, and contiguous points of the object quantity surface form a recognition section. The division into recognition sections with the image data contained therein or the points already calculated therefrom thus also corresponds to a division into object quantity surface sections. The division of the detection sections is carried out in the object recognition system, which may be understood as a higher-level system, which also includes the image acquisition system with the camera or can be understood as a system arranged downstream of the image acquisition system via an interface. Since the image acquisition system and the object recognition system are directly linked, a rigid demarcation between these two systems is not required within the scope of the invention.

The object recognition system in turn is part of a superordinate control device with which the overall arrangement can be controlled, regulated and monitored.

The recognition of the objects can include both the recognition of the object type and the detection of position and orientation of the objects on the conveyor. In the simplest case, the object set comprises only one object type, while for other applications, also several objects of different object types in constant or variable frequency fractions can assemble the object set.

The object recognition and the handling or processing of objects based thereon takes place during, in particular, continuous, movement of the object quantity, whereby, of course, operating states can occur in which the conveying process is planned or interrupted unplanned. The object recognition can take into account stoppages or changes in the speed of the conveying process, since movement information of the conveying device can be detected and included. For example, the image acquisition, in particular the storage and / or transmission of constant image information for further processing by the object recognition system at standstill of the conveyor, so if no path pulses are transmitted to the control device, temporarily interrupt.

If the delivery rate of the conveyor device is adjustable, it is advantageous if the number per unit of time with the object quantity of transported objects is at least approximately adjusted to the input power of the handling device, in particular slightly higher than this can be set. In certain application cases of the method, it can also be advantageous if, depending on predefinable and contained in the object recognition system decision criteria for certain detection sections no identification and selection of receivable objects. The feeding performance or processing performance of the system is not affected, but positively influenced on the contrary.

If successive or adjacent detection sections are delimited by the object recognition system in such a way that they have a common overlapping section, it is ensured that each surface point of the moving object set detected by the image acquisition system can be assigned to at least one recognition section and thereby also be fed to object recognition. This avoids that at the boundaries between adjacent or successive detection sections image information and thereby information about the amount of objects are lost.

The size and / or the extent of the recognition sections can advantageously be variably adapted by the image acquisition system or by the object recognition system, whereby the recognition sections can be adapted in particular to the size of the objects to be identified and recorded or to be processed. For example, a small recognition section can be predetermined for small object dimensions, and a larger recognition section for larger object dimensions, which avoids that a recognition section that is selected too small does not contain all detectable surface data of an object, and thereby the object recognition would of course be made more difficult. The recognition sections each correspond to a specific amount of image data to which the object recognition is applied.

Although a captured with a camera from one direction or with multiple cameras from different directions image capture of the object set can detect only visible to the imaging system used part of the object set surface, so only a part of the surface of an object can be detected and the object recognition despite this Incomplete surface information must be able to enable assignment to virtually existing objects, it is advantageous if a size of the recognition sections corresponds to at least one maximum dimension of an object to be identified, since in this case a match, ie a comparison of a recognition section with known target geometries based on the fullest possible object information and very reliable results of the object recognition are obtained.

In particular, the size and / or the extent of the overlapping sections can be variably adapted by the object recognition system; for example, in the case of an object with a circular outline of known dimensions, a reliable statement about the position can already be made in the case of an only partially detected contour, while For example, in the case of an object with a rectangular outline, no reliable statement about the position or alignment can be made if, for example, only a part of a corner of the object is contained in a recognition section. The larger the overlapping portion between two adjacent recognition portions, the higher the likelihood that objects including image information in two recognition portions will be correctly identified in the object recognition.

If an overlapping section is selected such that it is assigned a subsection of the object-quantity surface that is larger than a maximum dimension of the objects, it is ensured that all image information that can be detected by a single object is completely contained in at least one recognition section. Objects that are completely contained in an overlapping section can thereby be identified in two different detection sections, and therefore measures may also be required to ensure that an object completely contained in an overlap section is not considered to be two objects identical objects located at the same position can be performed by suppressing the duplicate identification before or in the selection of the appropriate objects.

The image acquisition applied to the set of objects is based on the use of light, the term light in this connection being understood to mean any electromagnetic radiation which can be used for non-contact detection of the surface and the term light should not be restricted to the light visible to humans.

The image acquisition method is selected in particular from a group comprising 2-D image acquisition, 3-D image acquisition, stereo camera acquisition, structured light acquisition, Laser scanner detection method, laser runtime detection method, line laser scanning, and the like, wherein a 3-D image acquisition method advantageously provides depth-valued object-set surface point coordinates that enables additional and improved methods of object recognition.

Preferably, the image acquisition method used is a light-section method in which a light source, such as a line laser, emits light in one or more illumination planes onto the moving object set and a camera directed toward the illuminated object set is provided for receiving the light reflected from the object set, the detection direction of the Camera to the lighting level includes an angle. The

The surface of the object set is detected by this method in the form of vertical cutting lines, the height of which can be detected by the lateral arrangement of the camera or can be calculated therefrom by means of trigonometric relationships, wherein the juxtaposition of the vertical cutting lines is effected by the movement of the object quantity by means of the conveying device is. In addition, this motion of the object set can also be superimposed on a movement of the illumination plane, as is the case when using a laser scanner, for example.

A further advantageous method for 3-D image acquisition of the object set consists in the use of runtime-based laser measuring systems, such as a pivoting movements performing 3D laser scanner or by means of a so-called time-of-flight camera (TOF). Such TOF time-of-flight cameras can, in addition to a gray level image, also record the respective depth information for individual pixels and transmit them to the image processing or object recognition.

Since it is easily possible for more than one object to be handled or processed in a recognition section, it is advantageous if, using a single recognition section or multiple recognition sections, a ranking of at least two, in particular more identified and repudiated, ones or editable objects is created, based on the temporal sequence of the handling or processing is determined by the handling device. A wide variety of criteria can be used for the ranking of the objects, with a particularly suitable criterion representing the quality of recognition respectively assigned to the identified objects. The recognition quality corresponds approximately to the degree of correspondence of an object quantity surface calculated from the image information or an identified object extracted therefrom with a virtual object in the form of CAD model data. Deviations between optically detected surfaces and virtual target surfaces may, of course, occur due to optical aberrations of the imaging system, image noise, computational inaccuracies, or other causes, and therefore, an object recognition system must allow for certain tolerances between the computed object surface area and the virtual sol surface. The smaller the deviations between objects identified within the measured object quantity surface and the stored virtual object models, the higher is the probability that this is actually such an object.

The sequence of the objects to be photographed or objects can further be identified on one of them

Objects each associated gripping probability based, in which about the security against unintentional release of gripping members of the handling device is included. This can be determined, for example, by the number of vacuum suction elements acting on a specific object or a mathematically possible contact surface between a specific object and gripping tongs of the handling device. A further possibility of ranking can be based on a residual dwell time assigned to the identified objects in the gripping area of the handling device, wherein objects in which the residual dwell time is close to the lower limit still possible for the handling device are primarily recorded or processed. The ranking of objects can also be done using a combination of the aforementioned criteria.

Prior to the ranking and subsequent selection of a tangible object, the control device carries out a check for a possible collision between the handling device and the object quantity and / or the conveying device in order to exclude non-tangible objects from the further calculations as early as possible and thereby save computational effort. The identification and associated position detection of the objects takes place in the object recognition system in an advantageous manner by comparing object quantity information originating from the image acquisition with CAD model data of objects to be recorded or processed. For this purpose, in the object-quantity surface calculated from the image information, search is made for sub-areas that match or at least approximately agree with surface sections of the virtual object models. Various calculation algorithms can be used for this sub-method of object recognition, which is also referred to as "matching", but which will not be discussed in more detail here.

For example, a point cloud calculated from the acquired image information may serve as a basis for the "matching process" by fitting a virtual object model represented by dots into the optically detected object set surface by shifting and / or rotating until the sum of the distance squares is a minimum This is the case when the virtual object model can be brought into line with the object set surface, where the object recognition system can also compare the image information with the CAD model data of several different object types, in particular in parallel evaluation processes , whereby the time required for the positive recognition of an object can be drastically reduced, if the object set does not consist of only one object type.These parallel recognition processes can in particular be performed on several different processing units in the form of processors be assigned when using dual-core or quad-core processors.

The admissibility of the method according to the invention can be increased by comparing the position of the object determined by means of a spatially fixed camera with a position of the object determined by means of a camera connected to the handling device before deviation or processing of an object. As a result, two independent image acquisition methods or also object recognition methods are applied to each object and this redundancy causes increased reliability in the execution of the method. In this variant of the method, in particular the image acquisition by the camera connected to the handling device can take place temporally after the image acquisition by the spatially fixed camera, whereby the second image acquisition can also be used for checking whether an identified object between the first image acquisition and the second image acquisition in its position and / or orientation with respect to the rest of the set of objects has changed, so has carried out an individual movement, which is superimposed on the scheduled and predictable movement by the conveyor. This may be the case, for example, when objects which are in an unstable or unstable position are changed in their position and / or orientation by vibrations or other influences between the first and second image acquisition, that is to say they shift or tilt. This method offers a high level of reliability in the use of the handling device, since it can be used to detect and thus react to unplanned or unpredictable changes in the object scene. This additional image acquisition or additional object recognition carried out to check the object position can take place with much less computational effort and in a shorter invoice time, if only a few recognized

Object points from the first object recognition can be checked by means of the second object recognition.

Since the computing time required for the identification and selection of objects also depends on the type and complexity of the objects, it is advantageous if the number of objects moved per unit of time by the conveying device is variable, in particular continuously adjustable. The throughput of objects moved by the conveyor can thus be adjusted so that it at least approximately corresponds to the throughput of the combination of object recognition system and handling device, and not an excessively large number of objects can not be recorded or processed and in a cycle again must be returned or the handling device has to wait for objects so to speak, because a too small number is promoted.

As already mentioned above, the performance of the method can be increased by simultaneously performing the identification and selection of the objects by the object recognition system in several recognition sections, for example by assigning different recognition sections to different, simultaneously operating computing units, such as processors. The object of the invention is further achieved by a method having the features of claim 13, according to which an output of the object recognition system is returned to the conveyor or one of these upstream charging and depending on the output of the degree of order of the moving of the conveyor object amount by influencing the conveyor or a This upstream charging device is influenced.

This method can be supplemented in particular by the method measures according to one of the claims 1 to 12, whereby the advantageous effects can be combined.

Since the computational effort required for the object recognition is greatly influenced by the degree of order of the object quantity, the computational effort and any waiting time for the handling device caused thereby can be reduced by raising the degree of order of the object set in the area of the conveyor and thereby the recognition of the individual objects can be done in less time with less computational effort, whereby the throughput of the entire system can be increased.

The degree of order of the object set can be formed in particular by the number of objects promoted per unit of time and / or the degree of separation of the objects within the object set and / or the degree of geometric alignment of the objects with respect to the conveying device. Since an increased number of objects conveyed per unit of time can quickly bring the required computing power to its limits or can lead to overloading of the object recognition system, a reduction of the object throughput can nevertheless bring about increased performance in the recording or processing of objects by the handling device , Likewise, the object recognition can be carried out more easily if the objects are at a distance from one another, ie they are conveyed separately by the conveying device, since calculation algorithms in connection with overlaps of objects can be omitted and the object recognition speed can thereby be increased. Finally, the degree of alignment of the objects with respect to the conveyor device can also have a strong influence on the object recognition performance since objects which are present on the conveyor device in a few predefined orientations have a significant influence on the object recognition performance. Lich lower computational effort can be detected as completely unoriented objects or workpieces.

In return, it may be advantageous for objects with relatively simple geometry and high computing power available when the objects of the object amount of the conveyor are supplied in a disordered position, which measures to maintain or increase the degree of order of the objects of the object amount can be omitted.

For object quantities which already have a certain degree of order in front of the conveying device, it is advantageous if they are fed to the conveying device in at least partially ordered position, ie a degree of order prevailing in front of the conveying device is not unnecessarily lowered in front of the conveying device, for example by magazined objects approximately be introduced into a bulk container of a feeder. As a result, the required computing power for object recognition or measures for increasing the degree of order, such as, for example, are also reduced. mechanically acting baffles, alignment stations or separating devices.

Faster object recognition can also be effected by arranging the objects on a slide moved by the conveyor or formed by the conveyor itself, on which they have a higher degree of order than in an unordered position on a flat, smooth surface conveyor belt.

For this purpose, the slide can advantageously have regular, in particular grid-like, arranged holding means or compartment-like receiving means for the objects. Due to the regular, in particular grid-like arrangement of the individual objects, the object recognition can be significantly accelerated, since in this way, as described above, can be omitted about complex calculation algorithms for the detection of overlapping objects.

One possibility, for example, that the conveying device itself acts as a slide to increase the degree of order, consists in using an endlessly circulating conveyor belt for moving the set of objects, the surface of which has means for stabilizing the object. Jektlage and object position with respect to the conveyor belt has, ie, for example, compartmentalized subdivisions, which are formed by protruding from the conveyor belt surface elevations or on the conveyor belt surface depressions, such as webs, knobs, grooves, cup-shaped recesses, etc. This allows objects to approximately circle - Have cylindrical or spherical shape stabilized during their movement.

The holding means may be chosen, in particular from a group comprising at least magnetic elements, vacuum elements, adhesively acting adhesive elements, electrostatic elements, spring clamping elements and so on. The various holding means are suitably selected depending on the object properties; For example, magnetic elements can be used as holding means, for example for ferromagnetic objects or workpieces.

The holding means may further be formed so that a plurality of objects are held by a holding means at the same time, but preferably in a defined orientation, for example, approximately in a stacked form.

In order to facilitate the picking up of objects from a slide, such as in a handling device with a forceps or suction gripper, it is advantageous if the holding means are flush with the top of the slide or spatially protrude to the top thereof.

The object of the invention is further solved by a method having the features of claim 21, according to which the object set is subjected at least a second time to an image acquisition method and an object recognition method and determined in an object identified and selected on the basis of the first object recognition Object features identified by the second image acquisition, in particular object point coordinates, are checked for agreement with object features from the first object recognition.

This method can in particular also be supplemented by the method measures according to one of the claims 1 to 19, whereby the advantageous effects can be combined. The reliability of the method of picking up and manipulating objects by a handling device is thereby substantially increased since the redundancy of the object recognition yields more reliable data.

In particular, in the case of objects which, owing to their properties, have a relatively unstable position with regard to the conveying device, ie which can change their position and / or orientation as a result of vibrations, or which can easily be influenced by the handling or by the processing of adjacent objects, it is great advantage if immediately or promptly before the actual recording or editing process a repeated check of the position and orientation takes place.

The invention further relates to a system for feeding and handling objects with the features of claim 25, according to which the system is designed to carry out one of the above-described methods.

The described methods or the equipment suitable for their execution can be used, in particular, within a production, transport or packaging installation.

The methods according to the invention and a system suitable for their execution are explained in more detail below with reference to the exemplary embodiments illustrated in the drawings.

Each shows in a highly schematically simplified representation:

Figure 1 is a diagram of a system according to the invention for handling and / or editing of objects.

FIG. 2 shows a system according to the invention for handling objects with a sketched method sequence according to the invention; FIG. 3 shows a system for handling objects with a sketch of a variant of the method according to the invention with two object detection methods performed at a time offset;

4 shows an example of an object set with a low degree of order;

5 shows an example of an object set arranged on a slide;

FIG. 6 is a plan view of a slide with an average order of object quantity; FIG.

7 shows a plan view of a further embodiment of a slide with an object set with a high degree of order;

8 shows a system for handling objects with a return of a result or an output variable of the object recognition to the conveyor device.

By way of introduction, it should be noted that in the differently described embodiments, the same parts are provided with the same reference numerals or the same component designations, wherein the disclosures contained in the entire description can be mutatis mutandis to the same parts with the same reference numerals or component names. Also, the location information chosen in the description, such as top, bottom, side, etc. related to the immediately described and illustrated figure and are to be transferred to the new situation mutatis mutandis when a change in position. Furthermore, individual features or combinations of features from the different exemplary embodiments shown and described can also represent independent, inventive or inventive solutions.

All information on ranges of values in objective description should be understood to include these arbitrary and all sub-ranges thereof, eg the indication 1 to 10 is to be understood as including all sub-ranges, starting from the lower limit 1 and the upper limit 10 are, ie all subareas start with a lower one Limit of 1 or greater and end at an upper limit of 10 or less, eg 1 to 1.7, or 3.2 to 8.1 or 5.5 to 10.

Fig. 1 shows parts of a system 1 for processing, manufacturing, transporting or packaging of objects 2, which are conveyed by means of a conveyor 3, here in the form of a conveyor belt 4 in the working space of a handling device 5 and the objects 2 a step in In the exemplary embodiment shown, the handling device 5 is formed by a robot 6, the articulated gripping arms and at the end thereof an end effector 7 acting on the objects 2 in the form of gripping tongs or shown in Fig. 1, a vacuum gripper 8 has.

The control of the system 1 is carried out by a direction indicated by dashed lines control device 9, which is designed in particular as a programmable logic controller or as Prozessleitsys- system. The control device 9 in this case comprises input units for detecting sensor signals or signals from input devices, at least one processing unit comprising a computing unit and a memory system and output units for signal output and for controlling the components of the system 1. In the illustrated embodiment, the handling of the objects 2 in a recording of the conveying device 3 by the handling device 5 and a transfer to a second conveying device 10, on which the objects 2 are assembled by the handling device 5 by way of example with further objects 11 and transported away from the second conveying device 10. The further objects 11 can in particular also be packaging means to which the objects 2 are assigned. In this case, the system 1 can also serve as a packaging system with which a wide variety of objects 2 can be packed flexibly without costly programming work.

The objects 2 on the conveyor 3 together form an object set 12 which is moved by the conveying device 3 in the conveying direction 13 into the working space or a gripping area of the handling device 5. In order for the handling device 5 to be able to receive or process individual objects 2 of the moving object set 12, although their position on the conveying device 3 is variable and not predefined, the installation 1 comprises a sensor system 14 - indicated by dotted lines in FIG. 1 - as well as an object drawing - System 15, which together allow the localization and recognition of aufhehmbaren objects 2 from the set of objects 12. Sensor system 14 and object recognition system 15 are part of the control device 9 and can be designed as independent systems, but also as spatially associated unit.

The sensor system 14 comprises an image acquisition system 16 with a camera 17, which is directed at the moving object amount 12 on the conveying device 3 and continuously performs image recordings of the moving object amount 12. The image acquisition rate of the camera 17 preferably moves in a range of 20 to 15,000 images per second, wherein the image acquisition rate is particularly adapted to the speed of movement of the object amount 12.

The images of the object set 12 acquired by the camera 17 are stored in the image acquisition system 16 and provided to the object recognition system 15 for evaluation. For the purpose of image acquisition, a light source 18 is further provided, with which the amount of object 12 is illuminated in the receiving area of the camera 17. The light source 18 ensures that the image is captured even with changing ambient light conditions with consistent quality. The image acquisition system 16 can be designed in particular for detecting the amount of object 12 by means of a so-called light-section method in which the light source 18 emits light in an illumination plane 19 on the moving object set 12 and with the camera 17, the resulting intersection between the illumination plane 19 and the surface of the Object amount 12 is recorded. The detection direction 20 of the camera 17 is inclined by an angle 21 to the illumination plane 19 and thereby the surface of the object set 12 is detected in the form of vertical cutting lines.

From the movement of the object set 12 and the image acquisition rate of the camera 17 results in a sequence of vertical cutting lines, which represent an image of the surface of the object set 12 and can represent them. As light source 18, in particular a line laser 22 is advantageous for this light-section method, which can deliver a sharply delimited illumination plane 19 with high luminous intensity. The camera 17 for recording the scene formed by the moving object set 12 is preferably formed by a CCD camera with a resolution of at least 1024x512 pixels. The angle 21 between see illumination plane 19 and detection direction 20 of the camera 17 is preferably between 5 ° and 75 ° degrees.

In addition to the information about the moving object set 12 acquired by means of the image acquisition system 16, the installation 1 or its control device 9 can comprise a distance measuring system 23, whereby the image information of the moving object set 12 recorded by the camera 17 with path and / or Speed information about the object set 12 can be linked. The displacement measuring system 23 comprises a displacement sensor 24 - here in the form of a rotary encoder 25 arranged on the conveying device 3 - with which the movement of the delivery quantity 12 is detected. As a result of this use of a separate position-measuring system 23, the path covered in the time interval between the time of image acquisition and the predicted time of the recording of an object 2 by means of the handling device 5, and thus also the exact position of an object 2, can be calculated or predicted.

The movement of the object set 12 or of the objects 2 contained therein after the time of image acquisition could in principle also be predicted on the basis of the image information and the motion velocity calculated therefrom, but the additional displacement measuring system 23 results in a much higher accuracy in the localization or prediction of the position the male or objects to be recorded 2, by the drive of the

Conveyor 3 originating speed fluctuations of the moving object amount 12 can be included. When using a light-slit method, this additional link with path or speed data is absolutely necessary, since a speed change of the conveyor 3 can not be determined with certainty on the basis of the vertical cutting lines alone.

The displacement sensor 24 can also be integrated in a drive system 26 for the conveyor device 3, possibly the path or movement information of the conveyor device 3 can also be read out directly from the component control of the drive system 26.

The linking of the image information with the path or movement information of the path measuring system 23 is indicated in FIG. 1 by a node 27. The information collected by the sensor system 14 about the amount of object 12 moved by the conveyor device 3 is evaluated in the object recognition system 15 using object recognition algorithms, wherein the image information stream provided by the image acquisition system 15 is subdivided into recognition sections 28 that can be assigned to the concrete sections of the object set 12.

The object set information contained in the recognition sections 28 comprises in particular object point coordinates, as well as movement information, wherein the object point coordinates represent parts of the optically detected surface of the object set 12. In Fig. 1, the detection sections 28 are shown as rectangular sections within the object recognition system 15. On the individual detection sections 28 of the object recognition system 15 each an object detection method for identifying and selecting the recordable with the handling device 5 and / or editable objects 2 is applied.

The object quantity information contained in the respective recognition sections 28 is analyzed in the object recognition system 15 in such a way as to search for models of the objects 2 to be recorded or processed in the object set surface represented by object point coordinates, in accordance with models supplied to the object recognition system 15, and subsequently the identified objects 2, which should be picked up or processed by the handling device 5, are selected.

The criteria of this selection procedure will be discussed in more detail below.

The objects 2 identified or selected by the object recognition system 15 or the information representing them are converted by the control device 9 into executable instructions for the components of the system 1. This is illustrated by way of example in FIG. 1 such that object information originating from the object recognition system 15 is relayed to individual component controls 29, 30 and 31 for the conveyor device 3, the handling device 5 and the second conveyor device 10, and this takes the necessary actions to record and / or edit the Execute objects 2. The modeling of the objects 2 to be included in the object recognition system 15 is represented in FIG. 1 by a model memory 32 which contains, for example, object information in the form of CAD data of a CAD system 33. The CAD model data 34, which are frequently present anyway in the case of objects 2, can thus be stored in the model memory 32 of the object recognition system 15 and used directly for the identification, localization and selection of the objects 2 to be recorded.

As further indicated by dashed lines in FIG. 1, the camera 17 and the light source 18 can be embodied as a spatially associated camera unit 35 which contains all the necessary components in a compact housing.

FIG. 2 shows, on the basis of a likewise greatly simplified scheme of the system 1, the method according to the invention for picking up and / or processing moving objects 2 from a conveyor device 3 by means of a handling device 5. FIG. 2 shows the sensor system 14 as blocks represented by dashed lines , which is formed here by the image acquisition system 16, with which the image acquisition method is performed and further the object recognition system 15, with which the object recognition process for identifying and selection with the handling device 5 recordable and / or editable objects 2 is performed.

The moving on the conveyor 3 object set 12 comprises a variable number of objects 2, which may be identical, ie belong to an object type or which may also be different, so may belong to several types of objects. With the camera 17 of the image acquisition system 16, which is directed onto the moving object set 12, optically detectable object quantity information is detected as image information stream 36 by the image acquisition system 16 and temporarily stored in an image storage arrangement 37 for further processing by the object recognition system 15. The memory structure of the image memory 37 is indicated in FIG. 2 by way of example as a ring buffer 38, in which the individual data packets 39 of the image information stream 36 are stored. For reasons of simplicity, FIG. 2 shows a small number of data packets 39, with a system 1 that is actually executed processing a much larger number of image data, of course.

The amount of object 12 moved by the conveying device 3 past the camera 17 of the image acquisition system 16 results in a continuous image information stream 36 in the form of Data packets 39 which are stored in the image memory arrangement 37. An associated group of data packets 39 is read from the image memory arrangement 37 for carrying out the object recognition method, and an object recognition method is applied to this subset of the image information by the object recognition system 15. These groups of data packets 39 correspond to the recognition sections 28 already described with reference to FIG. 1 and contain the object quantity information acquired by the image acquisition system 16, in particular in the form of object point coordinates and optionally also movement information. Prior to the execution of the object recognition method, these object point coordinates and movement information represent information about the optically detectable surface of the object set 12, which is formed from an initially unknown arrangement of objects 2 on the conveyor device 3. These not yet analyzed object set information, if these object points define, is often referred to as a point cloud.

In FIG. 2, a first recognition section 28 is marked with a 'in the image storage arrangement 37 and comprises a specific number of data packets 39; Similarly, in the image memory arrangement 37, a second detection section 28 is labeled with b 'and this also includes a defined number of data packets 39. The detection sections a' and b 'are in sequence from the object recognition system 15 from the Bildspeicheran- order 37 - here in the form of a ring buffer 38 - read out and applied to each of the object recognition method. The calculation algorithms used for object recognition are carried out by means of a computing unit 40, such as a processor, and are thereby identified in the individual recognition sections 28-here by way of example from the recognition sections a "and b" now incorporated in the object recognition system 15, therein the identification here is in particular directed to the object type, the object position and the object orientation.

An additional result of the object recognition method may be a quality evaluation characteristic number of the object recognition, which indicates how well an identified object 2 matches a desired object model stored in the model memory 32.

At the identified objects 2, a collision check is further carried out, with which it is determined whether an identified object 2 in the determined orientation and position can be absorbed by the end effector 7 of the handling device 5 collision-free. In this case, adjacent objects 2 as well as other known obstacles, in particular the conveying device 3, must be considered. This collision check may be performed by the object recognition system 15, but may be implemented in part and may be performed by the component controller 30 of the handler 5.

As can be seen in FIG. 2, the detection sections 28 in the image memory arrangement 37 correspond to specific detection sections 28 of the object set 12 moved by the conveying device 3, which are designated here by a and b. As shown further in FIG. 2, the recognition sections 28, as here the recognition sections a and b, can be referred to as

Subsets of the sets of objects 12 overlap on the conveyor 3 and together form an overlap section 41, wherein object amount information from the overlapping portion 41 both as an object 2 in the recognition section a, and as object 2 of the recognition section b, the object recognition process undergo. The image information of the overlapping section 41 is temporarily stored by the image acquisition system 16 in the course of the continuous acquisition of the image information stream 36 as data packets 39 in the image storage arrangement 37 and are here marked 41 '.

The extent 42 of an overlap section 41 results as a difference between the size 43 of the recognition section a and a distance 44 between a front edge 45 of the first recognition section a and a front edge 46 of the second recognition section b. In the example according to FIG. 2, two objects al and a2 are located on the conveying device 3 in the recognition section a, as well as the object a2 and further objects 2 in the recognition section b. The object a2 is located entirely in the overlapping section 41. This is possible if its Maximum dimension 47 is smaller than the extent 42 of the overlap portion 41. In the illustrated constellation of the objects 2 of the object set 12 and the detection sections a and b, the image data caused by the object a2 captured in the image acquisition system only once and cached in the image storage device 37, by the division However, the image information stream 36 in detection sections 28 of the object recognition system 15 in both the first recognition section a "and in the second recognition section b" subjected to the object recognition process. The arrangement of the objects 2 in the object set 12 shown by way of example in FIG. 2 causes the object al to be recognized by the object recognition system 15 as an object al "and the object a2, which is located entirely in the overlapping section 41, both from the recognition section a is recognized as object a2 "as well as being recognized as object bl" from the recognition section b ", since in reality the objects a2" and bl "recognized by the object recognition system 15 are only a single object a2, The object recognition system 15 or the object recognition method carried out therewith further comprises a test method for recognizing objects 2 which are located in an overlapping section 41 and thereby identified and located several times as multiple identifiers and the following component controller 30 of the handling device 5 as a single one to forward to be picked or editable object a2.

Since the system 1 should be suitable for receiving and / or processing objects 2 of very different sizes, the size 43 of the supplementary sections 28 and / or the extent of the overlapping sections 41 can be variably programmable, for example by reading out per detector section 28 adaptable numbers of data packets 39 , As a result, in particular the extent 42 of the overlapping sections 41 can be chosen such that it is greater than a maximum dimension 47 of the objects 2 to be recorded and / or processed. If several object types are contained in the object set 12 at the same time, the maximum dimension 47 of the largest can in particular be included Object type for determining the extent 42 of the overlapping sections 41 are used.

3 shows a further embodiment of a system 1 for exporting the method according to the invention for picking up and / or processing objects 2, which is executed on an object set 12 moved by a conveyor device 3.

In this variant of the method, the object quantity 12 moved in the conveying direction 13 is optically detected in a first detection area 48 by means of the camera 17, and the image information stream 36 generated thereby already described with reference to FIGS. 1 and 2

Method for locating, identifying and selecting an object 2 which can be raised and / or processed by the handling device 5. The image information stream 36 is thereby from the image acquisition system 16 and the object recognition system 15 in Detection sections 28 divided and applied to this the algorithms for object recognition. In addition, in this process variant, in a second detection area 49, which lies in the gripping area 50 of the handling device 5, a second image detection and a second object detection are carried out by means of a second camera 51 on the moving object set 12. For this purpose, the system 1 comprises a second camera 51, a second image acquisition system 52 and a second object recognition system 53. The second image acquisition method and object recognition method applied at a time interval to the first image acquisition and object detection become the objects 2 of the moving object set 12 in the region of the gripping region 50 the handling device 5 detected at least one more times. Thereby, a state of the object set 12 closer in time to receiving and / or processing of the objects 2 by the handling device 5 is detected, and this second image detection and object recognition can be used, the results of the first image detection and object detection based on the first Image information stream 36 of the first camera 17 to check for possible changes of the individual objects 2 within the object set 12. This second image capture and subsequent object recognition can be used in particular to be subjected to the first image acquisition and object recognition located, identified and selected objects 2 shortly before the actual action of the end effector 7 of the handling device 5 once again a review of the expected position. This second image detection and object recognition can be carried out with considerably less effort, since such a positional check of previously known objects 2-namely, objects 2 known from the first image detection and object recognition-can be restricted to a few points on the surface of an object 2. As a result, the second object detection can be carried out with a significantly lower computation effort, and the resulting short computation time makes it possible to perform this second image acquisition shortly before the actual recording and / or processing of an object 2. The second object detection method used by the object recognition system 53 can use the results of the first object recognition method by means of the first object recognition system 15, which is indicated in FIG. 3 by an arrow 54 between the first object recognition system 15 and the second object recognition system 53, which in turn can save computing power or perform a completely separate object detection, without the results of the first object recognition method to use. The image acquisition systems 16, 52 and the object recognition systems 15, 53 are part of the Control device 9, and do not have to be structurally separate units as shown, but may well be implemented in a common unit.

FIG. 3 shows a comparator unit 55 for the comparison of the results of the first object recognition and the second object recognition, which can likewise be part of the object recognition system 15 or 53 and this comparison between the objects 2 identified using the second image acquisition method and that by means of the first image acquisition method performed objects 2 performs. In the event that the second object recognition confirms the position of an object 2 precalculated for the gripping area 50, the command for picking up and / or editing this object 2 can be given to the component controller 30 of the handling device 5. As further indicated in FIG. 3, it is also possible for a second camera 51 not to be stationary in order to carry out the second image capturing method and object recognition method, but optically to capture or grasp the gripping region 50 of the handling device 5 by a second camera 51 'on the handling device 5 . can be monitored. Even with this arrangement of the camera 51 ', it is possible to check the objects 2 identified and selected on the basis of the first image acquisition method and the first object recognition shortly before recording and / or editing to their predicted, correct position.

If objects 2 are subject to the movement of position changes during movement on the conveyor device 3, the sequence of disturbing changes in the positions of the objects 2 can be detected by this double image acquisition and object recognition and a unsuccessful or possibly even causing damage gripping operation and / or processing operation be avoided. In this execution of the method, it is also advantageous if the second object detection method of the second

Camera 51, 51 'supplied image information stream also divided into detection sections and these detection sections are placed so that they at least approximately match or correspond to the detection sections 28 in the first image detection method and object recognition method.

4 shows a greatly simplified section of a conveying device 3 with an object set 12 moved thereon in conveying direction 13, which consists of different object types and in which the objects 2 are moved in an unordered position by the conveying device 3. If, in the case of such an object set 12 with a low degree of order, a recognition section 28 is selected in the course of image acquisition and object recognition, it is possible for objects 2 of different types of objects and objects 2 to be contained in this recognition section 28 in any desired orientation. In this case, the requirements for the object recognition system 15 are very high, since the object quantity surface detected by means of the image acquisition has to be compared with different object types or has to be searched in this object quantity surface for matches with different object types by means of the algorithms. The segmentation of the image data within a recognition section 28, ie the division of the image data into parts regions 2 which are highly likely to contain individual objects 2, is much more complicated than in the case of isolated objects 2 in the case of such a disordered object set 12.

FIG. 5 shows a likewise greatly simplified section of a conveying device 3 with an object set 12 moved thereon in the conveying direction 13, wherein here the objects 2 are arranged with a higher degree of order than in the object set 12 in FIG. 4. The objects 2 are here within the Object set 12 isolated, so have a distance to each other, which in the embodiment of FIG. 5, for example by a slide 56, which carries a portion of the object set 12 so a limited number of objects 2 and is moved by the conveyor 3, is effected. In this case, the object carrier 56 can conceive subdivision means 57 which bring about or facilitate the separation of the objects 2 on the object carrier 56 or maintain it. Such subdivision means 57 may be realized, for example, by a bin division 58, as shown in FIG. 5. Such a compartments 58 may be designed so that exactly one object 2 is contained in the slide 56 in each receptacle. The object set 12 composed of the objects 2 contained in the receiving compartments, which is moved by means of a slide 56 from the conveyor 3, has a much higher degree of order than in the disordered position of the objects 2 according to FIG. 4, whereby the object recognition simpler algorithms and can be performed with significantly lower computational effort. The slide 56 for increasing the degree of order can also be formed by the conveyor 3 itself, such as by the object 12 moving conveyor belt itself elements such as subjects, webs, knobs, recesses, bars, etc. and the objects 2 on the one hand to a certain extent one Be subjected to separation and also in their be stabilized position relative to the conveyor belt. Thus, for example, a stabilized movement of difficult-to-convey objects 2, such as cylindrical or spherical objects is possible by a conveyor belt with knob-like elevations or trough-shaped depressions.

By increasing the degree of order, such as in FIG. 5, by separating the objects 2 within the object set 12, the system components required to perform the image capture process and the object recognition process have lower hardware requirements, or this object detection facilitation can be used for a shorter time period Having to provide the object recognition, that is, about the throughput of the promoted objects 2 per unit time can be increased or the time interval between image acquisition on the one hand and recording and / or editing of the objects 2 on the other hand can be reduced, whereby overall the performance and / or the reliability of such a system 1 can be increased.

6 shows a detail of a plan view of a conveying device 3, on which an object quantity 12 is moved by means of a specimen slide 56 in the conveying direction 13. The object set 12 consists in this embodiment of two different types of objects - here with a triangular and circular base - the individual objects 2 with distance to each other, so sporadically arranged on the slide 56, however, their orientation about the perpendicular to the surface of the conveyor vertical axis is arbitrary. The separation of the objects 2 on the slide 56 can be ensured by regularly, in particular grid-like arranged holding means 59, each of which can hold an object 2 in an at least approximately separated from adjacent objects 2 position. As FIG. 6 furthermore shows, a slide 56 for the sensor system 14, in particular the image capture system 16 and the object recognition system 15, can have recognizable localization markings 60 which simplify the localization and recognition of the slide and consequently also the objects 2 arranged thereon like a grid allows.

Similar to FIG. 6, FIG. 7 shows a plan view of a slide 56 moved by a conveying device 3, with which the quantity of object 12 is moved in the conveying direction 13. In this embodiment, the individual objects 2 of the object set 12 are only one Object type and at the top of the slide all arranged with the same orientation. The object set 12 thus has a very high degree of order, which can further simplify the image acquisition to be carried out by the system 1 and subsequent object recognition. The fixation of the objects 2 on the slide 56 is effected in this embodiment by holding means 59 in the form of support strips 61 which have at the top of each other spaced apart magnetic elements 62, with which ferromagnetic objects 2 can be fixed in position on the support bars 61. The shape of the support strips 61 allows good accessibility of the objects 2 for an end effector 7 of the handling device 5, for example for picking up by means of a forceps gripper, since the support bars 61 raised relative to the base body of the object carrier 56 facilitate gripping of objects 2. Of course, instead of magnetic elements 62, it is also possible to provide vacuum suction elements, adhesively acting adhesive elements, electrostatic elements, clamping elements or similarly acting holding means 59.

FIG. 8 shows a further embodiment of the system 1, which may be independent of itself, with the same reference numerals or component designations being used again for the same parts as in the preceding FIGS. 1 to 7. In order to avoid unnecessary repetition, reference is made to the detailed description in the preceding Figs. 1 to 7 or reference.

In this embodiment of the system 1, similar to FIG. 1, it comprises a conveying device 3, here again in the form of a conveyor belt 4, with which an object set 12 composed of objects 2 is conveyed in the conveying direction 13 into the gripping region 50 of a handling device 5, for instance in the form of a conveyor Robot 6 is moved. As already described with reference to FIG. 2, the image information stream 36 generated by the camera 17, which contains the optically detected information about the moving object set 12, is subdivided into recognition sections 28 for performing the object recognition method in the object recognition system 15, to which the algorithms for Object recognition can be applied. The object quantity information contained in the individual recognition sections 28 in the form of object point coordinates are again compared in this exemplary embodiment with CAD model data 34 contained in a model memory 32, and thereby locatable, identified and selected receivable objects 2. Objects 2 not picked up by the handling device 5 are transferred in the system 1 to a return device 63, with which these objects 2 can be moved back to the beginning of the conveyor device 3, whereby an object cycle is formed.

In the Ausftihrungsbeispiel according to FIG. 8, the return conveyor 63 does not end directly on the conveyor 3 but on a feeder 64, with which a feed of the conveyor 3 can be done with objects 2, such as these from a parts stock 65, a number of male and / or or processing objects 2, abgescha separated subsets and passes to the conveyor device 3. This charging device 64 is shown here by way of example as a climbing conveyor 66, which comprises a part bunker 67 and a high conveyor 68. The high conveyor 68 is, for example, with an endless revolving

Traction means equipped, which separates by means of drivers from the parts supply 65 subsets of objects 2 and passes after an obliquely increasing high transport by means of the high conveyor 68 to the conveyor belt 4 of the conveyor 3. Since the return device 63 and the loading device 64 also serve to move the objects 2, they can also be regarded as part of the conveying device 3.

Since, as already described with reference to FIGS. 4 to 7, the degree of order of the objects 2 within the moving object set 12 influences or strongly correlates with the performance requirements of the object recognition system 15, the method according to the invention can also be characterized in that a method generated by the object recognition system 15 Output quantity 69 is transmitted to the conveying device 3 or to the associated component control 29 and the degree of order of the object amount 12 moved by the conveying device 3 is influenced by influencing and / or adjusting the conveying device 3. The output variable 69 may be an index of the utilization of the computing power of the control device 9 or a signal indicating an overload or a poor utilization of the computing power.

The result of the object recognition process, which is dependent on the flow rate 12, in particular on its degree of order and thereby also on the settings of the conveying device 3, is fed back by the object recognition system 15 to the conveying device 3 or its component control 29, whereby a control loop is formed. This feedback of the output quantity 69 can, in particular, bring about that of the conveying device 3 into the detection range of the sensor system 14 and consequently into the gripping area 50 the amount of separation of the objects 2 within the flow rate 12 is increased or lowered by changing conveyance speeds or a feed amount, and thereby to the control unit 9, in particular the object recognition system 15 achievable Performance can be optimally adjusted and thereby the overall performance of the system 1 is increased. The control signals 70 output by the component controller 29 to the subcomponents of the conveyor device 3 in this case, the loading device 64, the conveyor belt 4 and the return device 63, can be used as a control loop directly from the output variable 69, ie from the result of the object recognition method be made dependent.

Means to increase the degree of order of the amount of object 12 in the area of the conveying device 3 may consist, in particular, in the application of measures known from the prior art, such as singulating devices, alignment stations, baffle sections and the like, the effect of which can be adjusted to the degree of order and based on the output 69 can be influenced.

The embodiments show possible embodiments of the method for recording and / or editing of objects 2 and a suitable for its application Appendix 1 wherein it should be noted at this point that the invention is not limited to the specifically illustrated embodiments thereof, but rather also various combinations of individual variants are mutually possible and this variation possibility due to the doctrine of technical action by objective invention in the skill of those working in this technical field is the expert. So there are all conceivable variants, by combinations of individual

Details of the illustrated and described embodiment are possible, includes the scope of protection.

For the sake of order, it should finally be pointed out that, for a better understanding of the structure of the system 1, these or their components have been shown partly out of scale and / or enlarged and / or reduced in size. The task underlying the independent inventive solutions can be taken from the description.

Above all, the individual in Figs. 1; 2; 3; 4; 5; 6; FIGS. 7 and 8 form the subject of independent solutions according to the invention. The relevant objects and solutions according to the invention can be found in the detailed descriptions of these figures.

S u b e c u s e c tio n a tio n s

1 system 40 arithmetic unit

2 object

3 conveyor

41 Overlap section

4 conveyor belt

42 extension

5 handling device

43 size

44 distance

6 robots

45 leading edge

7 end effector

8 vacuum grippers

46 leading edge

9 control device

47 maximum dimension

10 conveyor

48 detection area

49 coverage area

11 object

50 gripping area

12 object set

13 conveying direction

51 camera

14 sensor system

52 image capture system

15 Object recognition system 53 Object recognition system

54 arrow

16 image capture system

55 Comparator unit

17 camera

18 light source

56 slides

19 lighting level

57 subdivision means

20 detection direction

58 Division

59 holding means

21 angles

60 localization mark

22 line lasers

23 position measuring system

61 carrying strip

24 displacement sensor

62 magnetic element

25 encoders

63 return conveyor

64 loader

26 drive system

65 parts stock

27 node

28 recognition section

66 ascending conveyor

29 component control

67 part bunker

30 component control

68 high conveyors

69 output size

31 component control

70 control signal

32 model memories

33 CAD system

34 CAD model data

35 camera unit

36 picture information stream

37 image memory arrangement

38 ring buffers

39 data packet

Claims

Patent claims

1. A method for the sequential recording and / or processing of individual objects (2) from a by a conveyor device (3), in particular continuously, moving object amount (12) by means of a handling device (5), in particular an industrial robot (6) comprising an image acquisition method carried out on the moving object set (12) and an object recognition method for locating, identifying and selecting at least one object (2) which can be recorded and / or processed by the handling device (5) by means of the object recognition system (15) processing the image information of the image acquisition method ), and recording and / or processing of a selected object (2) with the handling device (5), characterized in that an image information stream (36) continuously generated by the image acquisition method includes recognition sections (28) containing object information, in particular object point coordinates and movement information If Is (12) is divided and the identification and selection of the handling device (5) recordable and / or editable objects (2) from the object recognition system (15) for individual detection sections (28).

2. The method according to claim 1, characterized in that successive or adjacent detection sections (28) of the object recognition system (15) are limited so that they have a common overlap portion (41).

3. The method according to claim 1 or 2, characterized in that the size (43) of the detection sections (28) of the object recognition system (15) are variably adjustable.

4. The method according to claim 2 or 3, characterized in that an overlap section (41) has an extension (42) which exceeds a maximum dimension (47) of the objects (2).

5. The method according to any one of claims 1 to 4, characterized in that the

Image acquisition method uses a light section method in which a light source (18) emits light in a lighting plane (19) on the moving object set (12), and a Camera (17) is provided, the detection direction (20) to the illumination plane (19) includes an angle (21).

6. Method according to one of claims 1 to 5, characterized in that a series of at least two, in particular a plurality of identified and receivable objects (2) is created on the basis of a single recognition section (28) or on the basis of several recognition sections (28) of which the temporal sequence of the handling or processing of objects (2) by the Handhabungsvoπϊchtung (5) is set.

7. Method according to claim 6, characterized in that the ranking is based on a recognition quality respectively assigned to the identified objects (2).

8. Method according to claim 1, characterized in that the identification and an associated position detection by comparison of object quantity information determined from image information with the object recognition system (15) supplied CAD model data (34) of male and / or or to be processed objects (2).

9. Method according to one of claims 1 to 8, characterized in that, before picking up an object (2), the position of the object (2) determined by means of a spatially fixed camera (17) is correlated with a position determined by means of a handling device (5). NEN camera (51 ') determined position of the object (2) is compared to deviation.

10. The method according to claim 9, characterized in that the image acquisition by the with the handling device (5) connected to the camera (51 ') after the time

Image acquisition by the spatially fixed camera (17) takes place, in particular temporally close to the time of receiving an object (2).

11. The method according to any one of claims 1 to 10, characterized in that the per unit time by the conveyor device (3) moving number of objects (2) variable, in particular continuously, is adjustable.

12. The method according to any one of claims 1 to 11, characterized in that the

Identification and selection of the objects (2) from the object recognition system (15) in a plurality of recognition sections (28) is carried out simultaneously.

13. Method for sequentially handling and / or editing individual

Objects (2) from a by a conveyor device (3), in particular continuously, moving object set (12) by means of a handling device (5), in particular an industrial robot (6), comprising an on the moving object set (12) executed image acquisition method and an object detection method for identifying and selecting an object (2) which can be raised and / or processed by the handling device (5) by means of the object recognition system (15) processing the image information of the image acquisition method, and recording and / or processing a selected object (2) with the handling device (5), characterized in that an output variable (69) generated by the object recognition system (15) is returned to the conveying device (3), and the degree of order of the object quantity (12) moved by the conveying device (3) is influenced and / or adjusted by the conveying device (3). being affected.

14. The method according to claim 13, characterized in that the method is carried out according to one of claims 1 to 12.

15. The method according to claim 13 or 14, characterized in that the degree of order of the object set (12) by the number of per unit time promoted objects (2) and / or the degree of separation of the objects (2) within the object set (12) and / or the degree of alignment of the objects (2) with respect to the conveying device (3) is formed.

16. The method according to any one of claims 1 to 15, characterized in that the objects (2) of the object amount (12) of the conveying device (3) are fed in a disordered position.

17. The method according to any one of claims 1 to 15, characterized in that the

Objects (2) of the object amount (12) to the conveyor device (3) are supplied in at least partially ordered position.

18. The method according to any one of claims 1 to 17, characterized in that the

Objects (2) on the conveyor device (3) are arranged on a moving of this slide (56).

19. The method according to claim 18, characterized in that individual objects (2) on the slide (56) on regularly, in particular grid-like, arranged holding means (59) are arranged.

20. The method according to claim 19, characterized in that the holding means (59) are selected from a group comprising at least magnetic elements (62), vacuum elements, adhesively acting adhesive elements, electrostatic elements, clamping elements.

21. A method for the sequential recording and / or processing of individual objects (2) from a by a conveyor device (3), in particular continuously, moving object amount (12) by means of a handling device (5), in particular an industrial robot (6) comprising an image acquisition method performed on the moving object set (12) and an object recognition method for identifying and selecting at least one object (2) that can be picked up and / or processed by the handling device (5) by means of image recognition system (15) processing image information of the image acquisition method; / or processing of a selected object (2) with the handling device (5), characterized in that the object set (12) is again subjected to an image acquisition method and an object recognition method at a time interval and to an object identified and selected based on the first object recognition(2) certain object features determined on the basis of the second image capture, in particular object point coordinates, are checked for agreement with object features from the first object recognition.

22. The method according to claim 21, characterized in that the method is carried out according to one of claims 1 to 20.

23. The method according to claim 22, characterized in that the second image acquisition and object recognition takes place immediately before the recording or editing of an object (2).

24. The method according to any one of claims 1 to 23, characterized in that the

Image capture is a 3-D acquisition method selected from a group comprising stereo camera capture, pattern light capture, laser runtime measurement.

25. Appendix (1) for supplying and handling and / or editing objects

(2), comprising a conveying device (3) for moving an object set (12) with objects (2) to be supplied and / or processed, a handling device (5), in particular an industrial robot (6), for sequentially picking up individual objects (2 from the moving object set (12), an image acquisition system (16) for generating an automatically processable image information stream (36), an object recognition system (15) for identifying and selecting recoverable objects (2) based on the image information stream (36) Programmable control device (9) for controlling the handling device (5), characterized in that the system (1) is suitably designed for carrying out the method according to one of the preceding claims.