EP4149727A1

EP4149727A1 - Method for controlling the operation of an industrial robot

Info

Publication number: EP4149727A1
Application number: EP21725132.1A
Authority: EP
Inventors: Volker Haaf; Daniel GREB
Original assignee: Gerhard Schubert GmbH
Current assignee: Gerhard Schubert GmbH
Priority date: 2020-05-15
Filing date: 2021-05-10
Publication date: 2023-03-22
Also published as: DE102020113278A1; US20230192421A1; WO2021228773A1

Abstract

Method for controlling the operation of an industrial robot (3), in particular configured to carry out pick-and-place or selection tasks.

Description

Method for controlling the operation of an industrial robot

The invention relates to a method for controlling the operation of an industrial robot configured in particular to carry out pick-and-place or isolation tasks, the industrial robot being a handling device which has at least one handling element for handling one of a first orientation and / or position in comprises a second orientation and / or position to be relocated object.

Corresponding methods for controlling the operation of industrial robots are basically known from the prior art.

The control of the operation of corresponding industrial robots has hitherto typically been carried out via control devices that can be assigned or assigned to the respective industrial robots. These control devices are set up to feed these data - this can be, for. B. to act around different acquisition or sensor data -to process the generation of control information. The control information generated in this way is used as a basis for controlling the operation of the respective industrial robot or robots.

Although these known methods enable reliable control of the operation of industrial robots on the basis of corresponding control information, they are in need of improvement as the data processing resources required to generate the corresponding control information, i.e. H. in particular the required memory and computing resources are typically very high.

In contrast, there is a need for a data processing resource, in particular with regard to the data processing resources required to generate the corresponding control information, i. H. in particular the required memory and computing resources, improved methods for controlling the operation of an industrial robot.

Proceeding from this, the present invention is based on the object, in particular with regard to the data processing resources required for generating the corresponding control information, ie. H. in particular the required memory and computing resources to specify an improved method for controlling the operation of an industrial robot.

The object is achieved by a method for controlling the operation of an industrial robot according to claim 1. The dependent claims relate to possible embodiments of the method.

A first aspect of the invention relates to a method for controlling the operation of at least one industrial robot. A corresponding industrial robot typically comprises a handling device which has at least one handling element, ie, for example, a Gripping element, suction element, etc., for handling an object to be transferred from a first spatial orientation and / or position into a second spatial orientation and / or position. A corresponding industrial robot is therefore in particular an industrial robot configured to carry out pick-and-place or transfer or separation tasks. Correspondingly, the method is, in particular, a method for controlling the operation of at least one industrial robot configured to carry out pick-and-place or transfer or separation tasks.

A corresponding handling device of a corresponding industrial robot can optionally also be referred to or regarded as an end effector device. A corresponding handling element can therefore optionally also be referred to or regarded as an end effector element.

In all embodiments, a corresponding industrial robot can be a collaborative industrial robot (“cobot”). The method for controlling a collaborative industrial robot (“cobot”) can therefore be or will be implemented.

The method comprises the steps described in more detail below:

In a first step of the method, a plurality of objects located in a first orientation and / or position is detected and detection data describing the detected plurality of objects located in the first orientation and / or position is generated. In the first step of the method, acquisition data are generated which contain or describe a plurality of acquired objects located in a first orientation and / or position. For this purpose, the objects in the first alignment and / or position are typically detected via one or more detection devices; consequently, one or more detection devices is or are typically used to carry out the first step of the method. The objects located in the first alignment and / or position are typically located in a detection area of the respective detection device (s). A corresponding detection area can e.g. B. a defined section or area of a, z. B. belt-like or chain-like conveying element of a conveying device conveying the respective objects located in the first orientation and / or position. Specifically, a corresponding detection area can be a defined section or area of a, z. B. belt-like or chain-like feed element of a feed device feeding the respective objects in the first orientation and / or position into an action area of a handling element of a handling device of the industrial robot.

The objects detected in the first step of the method can in principle be aligned and / or positioned at least partially, possibly completely, in an ordered or disordered manner.

A first step is used in a second step of the method Data processing measure for processing the acquisition data, the application of the first data processing measure supplying a selection of exactly one object from the plurality of objects in the first orientation and / or position described by the acquisition data, and generating exactly one of the objects described by the acquisition data Selection information describing a plurality of objects that are selected objects in the first orientation and / or position. In the second step of the method, a first data processing measure is applied in order to process the acquisition data generated in the first step of the method. The acquisition data generated in the first step of the method are accordingly processed in the second step of the method on the basis of a first data processing measure. The application of the first, carried out in particular by a data processing device implemented in hardware and / or software

As a result, the data processing measure provides a selection of precisely one object from the plurality of objects in the first alignment and / or position described by the detection data. The result of the application of the first data processing measure is therefore generated selection information which contains or describes exactly one object selected from the plurality of objects in the first orientation and / or position described by the detection data. The selection information generated therefore contains or describes (precisely) the object which was selected from the plurality of objects in the first orientation and / or position described by the detection data by using the first data processing measure.

As can be seen below, one or more selection criteria can be taken into account when applying the first data processing measure; consequently, exactly one object can be selected from the plurality of objects located in the first alignment and / or position described by the detection data on the basis of or taking into account at least one selection criterion.

The application of the first data processing measure can involve the application of at least one selection algorithm, possibly forming a component of a selection software. B. involve an image processing algorithm. A corresponding selection or image processing algorithm can be set up to select precisely one object from the plurality of objects in the first alignment and / or position described by the detection data. As mentioned, one or more selection criteria can be taken into account when applying the first data processing measure. A corresponding selection or image processing algorithm can be set up accordingly to take into account one or more corresponding selection criteria when selecting precisely one object from the plurality of objects in the first orientation and / or position described by the detection data.

The selection algorithm can be used as part of a machine learning process, ie via a Method of machine learning. As part of a corresponding machine learning process, one or more artificial neural networks with one or more intermediate layers implemented between an input and an output layer can be or have been used.

In a third step of the method, a second data processing measure is used to process the selection information, the application of the second data processing measure providing at least one coordinate for handling the precisely one object described by the selection information by means of at least one handling element of the handling device of the industrial robot, and generating one the coordinate information describing at least one coordinate. In the third step of the method, a second data processing measure is applied in order to process the selection information generated in the second step of the method. The selection information generated in the second step of the method is accordingly processed in the third step of the method on the basis of a second data processing measure. The application of the second data processing measure carried out in particular by the or another data processing device implemented in hardware and / or software provides as a result at least one coordinate for handling the precisely one object described by the selection information by means of at least one handling element of the handling device of the industrial robot. The result of the application of the second data processing measure is therefore generated coordinate information which contains or describes at least one coordinate for handling the precisely one object described by the selection information by means of at least one handling element of the handling device of the industrial robot. The generated coordinate information therefore contains or describes one or more coordinates - these are typically coordinates related to the respective object - for handling the precisely one object described by the selection information by means of at least one handling element of the handling device of the industrial robot. The coordinates described by the coordinate information can be, for. B. act around world coordinates.

As can be seen below, one or more determination criteria can be taken into account when using the second data processing measure; consequently, one or more coordinates for handling the precisely one object described by the selection information can be determined by means of at least one handling element of the handling device of the industrial robot based on or taking into account at least one determination criterion.

The application of the second data processing measure can include the application of at least one determination algorithm, possibly forming a component of a determination software. B. involve an image processing algorithm. A corresponding determination or image processing algorithm can be set up to determine suitable coordinates on the object described by the selection information, at which the object can be handled, ie, gripped, by means of one or more handling elements of the handling device of the industrial robot. As mentioned, one or more determination criteria can be taken into account when applying the second data processing measure. A corresponding determination or image processing algorithm can be set up accordingly to take into account one or more corresponding determination criteria when determining corresponding coordinates.

The determination algorithm can be used as part of a machine learning process, i. H. via a method of machine learning. As part of a corresponding machine learning process, one or more artificial neural networks with one or more intermediate layers implemented between an input and an output layer can be or have been used.

In a fourth step of the method, the operation of the industrial robot for performing a task, in particular a pick-and-place or isolation task, is controlled on the basis of the coordinate information. The coordinate information is therefore included in corresponding control information for controlling the operation of the industrial robot or corresponding control information is generated on the basis of or taking into account the coordinate information. In the fourth step of the method, the coordinate information generated in the third step of the method is used as a basis for controlling the operation of the robot, so that the industrial robot attaches the respective handling element or elements of the handling device to the coordinate or coordinates described by the coordinate information around the respective object to convert from the first orientation and / or position into a second orientation and / or position. The second alignment and / or position can be, for. B. be specified by a user or programmer of the industrial robot or will be.

The control of the operation of the industrial robot typically includes handling precisely the one object that is described by the selection information at the coordinate or coordinates that are described by the coordinate information, or results in one.

The main advantage of the method described here is that in the second step of the method a corresponding selection of exactly one object is selected from the plurality of detected objects described by the detection data. In this way, the amount of data to be processed further to determine the respective coordinates can be significantly reduced or limited to a single object, namely the respectively selected object, which also significantly reduces the data processing resources required to generate the corresponding control information, ie in particular the memory and computing resources reduced. In particular in comparison to approaches known from the prior art for generating corresponding control information to be used as a basis for the control of corresponding industrial robots, in which considerable amounts of data have to be processed, the method described here makes it possible already in the second step of the method and thus comparatively early to perform the to significantly reduce the amount of data to be processed to generate corresponding coordinate information by using the first data processing measure to select exactly one object from the plurality of objects in the first orientation and / or position described by the detection data. As a result of the selection made in the second step of the method from exactly one object from the plurality of objects in the first orientation and / or position described by the detection data, the amount of data to be processed further is reduced to precisely the selected object; the data to be processed further defined by the selection information are thus restricted or concentrated to the respective precisely one selected object, so that further processing of the data is limited or concentrated only to the respective precisely one selected object.

Due to the reduced amount of data, the method can be carried out significantly faster than the required data processing resources, i.e. H. in particular the required memory and computing resources are significantly reduced. Alternatively or additionally, less powerful data processing devices can be used to carry out the method, in particular with regard to storage and computing power.

Overall, there is an improved method for controlling the operation of an industrial robot.

Controlling the operation of the industrial robot on the basis of the coordinate information can in all embodiments include at least one interaction of the at least one handling element of the handling device of the industrial robot with the object described by the respective selection information and thus selected at or in the area of the at least one coordinate described by the coordinate information. A corresponding interaction can be a, z. B. realized by gripping (“picking”) the object described by the respective selection information and thus selected at the coordinate (s) described by the respective coordinate information. In particular, controlling the operation of the industrial robot on the basis of the coordinate information can include converting the object described and thus selected by the respective selection information from a first orientation and / or position to a second orientation and / or position by means of the at least one handling element of the handling device of the industrial robot .

The detection of the plurality of objects located in the first orientation and / or position in the first step of the method can be carried out by means of at least one optical Detection device are carried out. Corresponding detection devices can accordingly be designed as optical detection devices, ie, for example, as (digital) camera devices, or comprise such devices. Corresponding detection devices can have one or more detection elements, ie, for example, optical sensor elements, such as e.g. B. CCD and / or CMOS sensors include. The objects located in the first alignment and / or position can therefore be detected optically in all embodiments.

In particular, in the first step of the method, at least one optical detection device can be used, which is connected to the industrial robot that can be controlled or is to be controlled according to the method, ie. H. is arranged or formed in particular on an immovable or immovable section of the industrial robot, in particular on a section of a housing device of the industrial robot which is set up to accommodate functional and / or supply components of the industrial robot. In other words, at least one optical detection device arranged or embodied in a fixed position or stationary on the industrial robot can be used, which is advantageous because the fixed position or fixed arrangement enables a defined optical detection area to be implemented in a simple manner. A corresponding positional or stationary arrangement of a corresponding optical detection device is expediently selected in an area that is elevated relative to the objects to be detected, so that the detection area resulting from the arrangement of the optical detection device relative to the objects to be detected provides a kind of overview of at least a part of the objects to be detected, possibly across all objects to be detected. For this purpose, an arrangement of the optical detection device on a housing device of the industrial robot of the industrial robot has proven to be expedient. A corresponding housing device can, for. B. arranged or formed on a vertically extending arranged or formed base support of the industrial robot. Functional and / or supply components of the industrial robot, possibly implemented in hardware and / or software, can be arranged or formed on or in a housing device, which is sometimes also to be referred to or to be regarded as a “head”.

In all embodiments, at least one optical detection device can be used which has a defined optical detection area within which objects can be or are detected by means of the detection device. A corresponding optical detection area can, for. B. a defined section or area of a, z. B. belt-like or chain-like conveying element of a conveying device conveying the respective objects located in the first orientation and / or position. Specifically, a corresponding optical detection area can be a defined section or area of a, z. B. band-like or chain-like feed element of a feed device feeding the respective objects in the first orientation and / or position into an action area of a handling element of the handling device of the industrial robot. There is typically at least one qualitative relationship between the size of the optical detection area and the amount of data generated by the detection data. In all embodiments, in particular with regard to its size, an optical detection area can be selected in such a way that the amount of data generated by the detection data is limited to a specific limit value. It is of course conceivable that an optical detection device with a variable optical detection area is used; consequently, an optical detection device can be used which has a first optical detection area and at least one of these, in any case, with regard to the size of the detection area in comparison to the first optical detection area, different further optical detection areas.

Basically, the objects to be recorded can be recorded dynamically or statically; the objects can therefore be detected when they are in motion or when they are not in motion.

The first data processing measure can be implemented by at least one single or multi-layered first artificial neural network. A corresponding first artificial neural network has at least one intermediate layer located between an input and an output layer. The degree of complexity and the associated performance of the respective first artificial neural network (s) can in particular be defined by the number of respective intermediate layers. In principle, artificial neural networks configured in a (comparatively) simple or (comparatively) complex manner can be used to implement the first data processing measure. Surprisingly, the use of corresponding artificial neural networks showed special advantages in connection with the processing of the respective acquisition data to generate the respective selection information.

Alternatively or in addition, the second data processing measure can be implemented by at least one single or multi-layered second artificial neural network. A corresponding second artificial neural network has at least one intermediate layer located between an input and an output layer. The degree of complexity and the associated performance of the respective second artificial neural network (s) can in particular be defined by the number of respective intermediate layers. In principle, artificial neural networks configured in a (comparatively) simple or (comparatively) complex manner can be used to implement the second data processing measure. Surprisingly, the use of corresponding artificial neural networks showed special advantages in connection with the processing of the respective selection information to generate the respective coordinate information.

As mentioned, at least one for applying the first and second data processing measures can be used to apply the first and second data processing measures set up hardware and / or software implemented data processing device can be used. A respective data processing device can be set up to implement at least one first artificial neural network and / or at least one second artificial neural network or be or will be implemented by corresponding first and / or second artificial neural networks. A respective data processing device can form part of a control device for controlling the operation of the industrial robot.

As mentioned, one or more selection criteria can be taken into account when applying the first data processing measure. The selection made by means of the first data processing measure can therefore be carried out on the basis of at least one selection criterion from the plurality of objects in the first orientation and / or position described by the detection data. A corresponding selection criterion can represent a boundary condition to be taken into account when selecting a respective precisely one object.

As a corresponding selection criterion, for. B. absolute alignment information and / or absolute position information describing an absolute alignment and / or position of at least one object of the objects located in the first alignment and / or position can be used. Absolute alignment information and / or absolute position information can e.g. B. be specified in angles and / or in world coordinates or contain such. Thus, an absolute alignment and / or position of at least one object of the objects located in the first alignment and / or position can be recorded and used as a selection criterion and taken into account or taken as a basis for the selection of precisely one object.

Alternatively or in addition, a selection criterion such as B. relative orientation information and / or relative position information describing a relative orientation and / or position of at least one object of the objects in the first orientation and / or position to at least one further object of the objects in the first orientation and / or position can be used. A relative orientation information and / or relative position information can, for. B. be specified in angles and / or in world coordinates or contain such. Thus, a relative alignment and / or position of at least one object of the objects in the first alignment and / or position to at least one further object of the objects in the first alignment and / or position can be detected and used as a selection criterion and accordingly when selecting the exactly one object can be taken into account or used as a basis.

Alternatively or in addition, a corresponding selection criterion can be an approach movement or approach vector of a handling element of the handling device of the industrial robot for approaching at least one object in the first orientation and / or position Approach information describing the objects located can be used. A starting movement or starting vector of a handling element of the handling device of the industrial robot, in particular required by an ACTUAL position and / or ACTUAL orientation, can therefore be recorded and used as a selection criterion and taken into account or taken as a basis for the selection of exactly one object will.

Alternatively or additionally, dimensional information describing at least one geometrical-constructive dimension of at least one object of the objects located in the first orientation and / or position can be used as a corresponding selection criterion. Consequently, at least one dimension of at least one object of the objects located in the first orientation and / or position can be detected and used as a selection criterion and accordingly taken into account or taken as a basis for the selection of precisely one object.

As an alternative or in addition, form information describing at least one geometrical-constructive shape (spatial shape) of at least one object of the objects located in the first orientation and / or position can be used as a corresponding selection criterion. Consequently, at least one shape of at least one object of the objects located in the first orientation and / or position can be detected and used as a selection criterion and accordingly taken into account or taken as a basis for the selection of precisely one object.

As an alternative or in addition, color information describing the coloring of at least one object of the objects located in the first alignment and / or position can be used as a corresponding selection criterion. The coloring of at least one object of the objects located in the first alignment and / or position can therefore be recorded and used as a selection criterion and accordingly taken into account or taken as a basis for the selection of precisely one object.

Alternatively or in addition, the corresponding selection criterion can include the surface, in particular the surface texture, i.e. H. in particular the mechanical and / or optical surface properties of at least one object of the objects located in the first orientation and / or position are used. The surface, in particular the surface quality, of at least one object of the objects located in the first orientation and / or position can therefore be recorded and used as a selection criterion and taken into account accordingly or taken as a basis for the selection of precisely one object.

Alternatively or in addition, mass information describing the mass, in particular a center of gravity, of at least one object of the objects located in the first orientation and / or position can be used as a corresponding selection criterion. Consequently, the mass, in particular a center of mass, of at least one object in the Objects located in the first alignment and / or position are detected and used as a selection criterion and accordingly taken into account or taken as a basis for the selection of precisely one object.

As an alternative or in addition, a corresponding selection criterion can be used as a corresponding selection criterion, which is sometimes also referred to as “format” in pick-and-place applications. The type of at least one object of the objects located in the first orientation and / or position can therefore be recorded and used as a selection criterion and accordingly taken into account or taken as a basis for the selection of precisely one object.

As mentioned, one or more determination criteria can be taken into account when applying the second data processing measure. The determination of the at least one coordinate for handling the precisely one object described by the selection information using the second data processing measure can therefore be carried out on the basis of at least one determination criterion. A corresponding determination criterion can represent a boundary condition to be taken into account when determining the respective coordinate (s).

As a corresponding determination criterion, for. B. absolute orientation information and / or absolute position information describing an absolute orientation and / or position of the respectively selected object can be used. Absolute alignment information and / or absolute position information can e.g. B. be specified in angles and / or in world coordinates or contain such. An absolute alignment and / or position of the respectively selected object can therefore be recorded and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or taken as a basis.

Alternatively or in addition, z. B. relative orientation information and / or relative position information describing a relative orientation and / or position of the respectively selected object to at least one further object of the objects located in the first orientation and / or position can be used. A relative orientation information and / or relative position information can, for. B. be specified in angles and / or in world coordinates or contain such. A relative orientation and / or position of the respectively selected object to at least one further object of the objects located in the first orientation and / or position can therefore be recorded and used as a determination criterion and taken into account or taken as a basis when determining the coordinate (s) will.

Alternatively or in addition, a corresponding determination criterion can be an approach movement, in particular required from an ACTUAL position and / or an ACTUAL orientation or approach information describing the approach vector of a handling element of the handling device of the industrial robot can be used for approaching the respectively selected object. An approach movement or approach vector of a handling element of the handling device of the industrial robot for approaching the respective selected object can therefore be recorded and used as a selection criterion and accordingly when determining the coordinate (n ) taken into account or taken as a basis.

Alternatively or in addition, dimensional information describing at least one geometrical-constructive dimension of the respectively selected object can be used as a corresponding determination criterion. Consequently, at least one dimension of the respectively selected object can be recorded and used as a determination criterion and accordingly taken into account when determining the coordinate (s) or used as a basis for this.

Alternatively or in addition, shape information describing at least one geometric-constructive shape (spatial shape) of the respective selected object can be used as a corresponding determination criterion. A shape of the respectively selected object can therefore be recorded and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or used as a basis for this.

As an alternative or in addition, color information describing the coloring of the respective selected object can be used as a corresponding determination criterion. The coloring of the respective selected object can therefore be recorded and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or used as a basis for this.

Alternatively or in addition, the corresponding determination criterion can include the surface, in particular the surface quality, i.e. H. in particular the mechanical and / or optical surface properties of the respective selected object descriptive surface information can be used. The surface, in particular the surface quality, of the respectively selected object can therefore be recorded and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or used as a basis.

Alternatively or in addition, mass information describing the mass, in particular a center of gravity, of the respective selected object can be used as a corresponding determination criterion. The mass, in particular a center of mass, of the respectively selected object can therefore be recorded and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or taken as a basis.

Alternatively or in addition, the type of Handling information describing handling of the respectively selected object can be used by at least one handling element of the handling device of the industrial robot. The type of handling of the respectively selected object can therefore be detected by at least one handling element of the handling device of the industrial robot and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or taken as a basis.

Alternatively or in addition, handling information describing the handling surface of the respectively selected object by at least one handling element of the handling device of the industrial robot can be used as a corresponding determination criterion. The handling surface of the respectively selected object can therefore be detected by at least one handling element of the handling device of the industrial robot and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or used as a basis.

As an alternative or in addition, a genre describing the particular selected object can be used as a corresponding determination criterion. The genus of the respectively selected object can therefore be recorded and used as a determination criterion and taken into account accordingly when determining the coordinate (s) or used as a basis for this.

The method typically does not end after performing the steps described for a first selected object which, within the framework of or after performing the steps of the method described above, for example. B. selected from the plurality of objects located in the first orientation and / or position and removed as a selected object in the course of controlling the industrial robot from the plurality of objects located in the first orientation and / or position and transferred to a second orientation and / or Position has been implemented. Correspondingly, after corresponding selection information and coordinate information for a first object selected from the plurality of objects located in the first orientation and / or position has been generated, corresponding selection information and coordinate information is typically generated for at least one further object from the plurality of objects in the first orientation and / or position. or position remaining objects. The method is typically carried out until appropriate selection information and coordinate information has been successively generated for each object from the large number of objects located in the first orientation and / or position. Of course, the process can be interrupted or stopped when a termination condition is fulfilled or present. A corresponding termination condition can e.g. B. be fulfilled or present when there is only a certain number of objects in the first orientation and / or position or when there is no longer any object in the first orientation and / or position.

The method may prior to the step of detecting the plurality of in a first orientation and / or position objects and the generation of acquisition data describing the acquired plurality of objects in the first orientation and / or position, a step of providing a plurality of objects in a first orientation and / or position on a stationary or moving surface, in particular a base of a conveyor device, preferably a feed conveyor device for feeding objects. In any case, the provision, which can include feeding the respective objects into an action area of a handling element of the handling device of the industrial robot, can therefore optionally also represent a step of the method.

In all embodiments, the corresponding objects can be objects to be packaged in a package. The type of packaging results from the type of objects to be packed. Merely by way of example, food that can be separated, such as. B. candy, referenced as possible objects. The method can in principle be carried out with all objects to be separated from a plurality of objects located in a first orientation and / or position.

A second aspect of the invention described herein relates to an industrial robot, in particular a collaborative industrial robot (“cobot”). The industrial robot comprises a handling device which has at least one handling element that can be moved in at least one degree of freedom of movement for handling an object to be converted from a first orientation and / or position to a second orientation and / or position and a control device implemented in particular in terms of hardware and / or software Control of the operation of the industrial robot. The control device is set up to carry out the method according to the first aspect of the invention. The control device therefore typically has machine-readable instructions for performing the steps of the method according to the first aspect of the invention. All statements in connection with the method according to the first aspect of the invention apply analogously to the industrial robot according to the second aspect of the invention and vice versa.

A third aspect of the invention described herein relates to an arrangement for converting objects from a first orientation and / or position into a second orientation and / or position. The arrangement, which may also be designated or considered as a machine, comprises at least one industrial robot according to the second aspect of the invention. For this purpose, the arrangement comprises at least one peripheral device. A corresponding peripheral device can be or comprise a feed device for feeding objects, in particular objects located in a first orientation and / or position, into an action area of at least one handling element of the handling device of the industrial robot. Alternatively or in addition, a corresponding peripheral device can be or comprise a discharge device for discharging objects, in particular objects that have been converted into the second orientation and / or position by means of the industrial robot. All statements in connection with the Methods according to the first aspect of the invention apply analogously to the arrangement according to the third aspect of the invention and vice versa.

A corresponding arrangement can be a packaging machine for packaging objects or form part of such a machine. A corresponding packaging machine can, for. B. be set up to objects such. B. food, cosmetic articles, pharmaceutical articles, technical articles, from a first orientation and / or position to a second orientation and / or position, d. H. z. B. in a carrier-like, receiving device to implement.

The invention is explained in more detail using exemplary embodiments in the drawings. It shows:

1 shows a basic illustration of an arrangement for converting objects from a first orientation and / or position into a second orientation and / or position according to an exemplary embodiment;

2 shows a basic illustration of an industrial robot according to an exemplary embodiment;

3 shows a basic illustration of a plurality of objects located in a first orientation and / or position, detected by a detection device, according to a

Embodiment;

FIG. 4 shows an object selected from the plurality shown in FIG. 3; FIG. and

5 shows a block diagram to illustrate a method for controlling the operation of an industrial robot according to an exemplary embodiment.

1 shows a schematic diagram of an arrangement 1 for converting objects 2 from a first orientation and / or position into a second orientation and / or position according to an exemplary embodiment in a top view. The arrangement 1 can also be referred to or viewed as a machine.

The arrangement 1 comprises one, e.g. B. as collaborative industrial robots ("cobots"), trained industrial robots and several peripheral devices. In the case of corresponding peripheral devices, the exemplary embodiment is a z. B. designed as a feed belt feed device 4 for feeding objects 2, in particular objects 2 located in a first orientation and / or position, in an action area 5 of a, z. B. designed as a gripping or suction element, end effector or handling element 6 of an end effector or handling device 7 of the industrial robot 3 and a z. B. a discharge device 9 designed as a discharge belt for discharging objects 2, in particular objects 2 which have been moved into a second orientation and / or position by means of the industrial robot. The end effector or handling element 6 of the end effector or handling device 7 is, as indicated purely schematically by the double arrow, mounted movably in one or more degrees of freedom of movement.

The dashed illustration indicates that the arrangement 1 can also include several corresponding peripheral devices and several corresponding end effector or handling devices 7 together with the associated end effector or handling element 6.

FIG. 2 shows a basic illustration of an industrial robot 3 according to an exemplary embodiment in a side view. Based on Fig. 2 is again the z. B. designed as a robot arm or such a comprehensive end effector or handling device 7 together with the associated end effector or handling element 6 shown. The industrial robot 3 shown in FIG. 2 can correspond to the industrial robot s shown in FIG. 1.

A housing device 3.1 arranged or constructed on a vertically extending arranged or constructed base support 3.2 of the industrial robot 3 can be seen from FIG. 2. Functional and / or supply components (not shown) of the industrial robot 3, possibly implemented in hardware and / or software, can be arranged or formed on or in the housing device 3.1, which is also to be referred to or to be regarded as a “head”.

Based on Fig. 2 is also a z. B. designed as a camera device or such a comprehensive optical detection device 10 can be seen, which is arranged or formed on the housing device 3.1. The optical detection device 10 is accordingly arranged or formed on a section of the housing device 3.1 and thus on an immovable or immovable section of the industrial robot 3. A corresponding positional or stationary arrangement of the optical detection device 10 is therefore selected in the exemplary embodiment on an area that is elevated relative to the objects 2 to be detected, so that the optical detection area resulting from the arrangement of the optical detection device 10 relative to the objects 2 to be detected 12 enables a kind of overview of at least some of the objects 2 to be detected, possibly of all objects 2 to be detected.

The dashed illustration again indicates that the industrial robot 3 can also include several corresponding end effector or handling devices 7 together with associated end effector or handling elements 6 as well as several corresponding optical detection devices 10.

Returning to FIG. 1, a control device 11 implemented in hardware and / or software can be seen, which is set up to control the operation of the industrial robot 3. The control device 11 shown purely by way of example in FIG. 1 as a structural component of the industrial robot 3 is therefore set up to generate control information Basis of which the operation of the industrial robot 3 is controlled to carry out certain tasks. With corresponding tasks it can be, for. B. to pick-and-place or separation tasks of objects 2 act.

The control device 11 is set up accordingly to carry out a method for controlling an industrial robot 3, which is explained in more detail below using an exemplary embodiment, also with reference to FIGS. 3-5.

The method comprises the steps described in more detail below:

In a first step of method S1 of the method, detection data are generated which contain or describe a plurality of detected objects 2 located in a first orientation and / or position. The z. For this purpose, objects 2 located, for example, on a surface of the feed device 4 in the first orientation and / or position are detected via one or more optical detection devices 10; consequently, one or more optical detection devices 10 are used to carry out the first step of the method. The objects 2 located in the first orientation and / or position are located in a detection area 12 shown in dashed lines in FIG. 1 and shown separately in FIG. 3 at least one optical detection device 10. z. B. band-like or chain-like, feed element 4.1 of the respective objects 2 located in the first orientation and / or position in the action area 5 of a handling element 6 of the handling device 7 of the industrial robot 3 affect the feed device 4.

The objects 2 detected in the first step of the method can in principle be aligned and / or positioned at least partially, possibly completely, in an ordered or disordered manner.

In a second step of method S2 of the method, a first data processing measure is used in order to process the detection data generated in the first step of method S1. The acquisition data generated in the first step of the method S1 are therefore processed in the second step of the method S2 on the basis of a first data processing measure. The application of the first data processing measure, carried out in particular by a data processing device implemented in hardware and / or software, provides as a result a selection of exactly one object 2.1 from the plurality of objects 2 in the first orientation and / or position described by the detection data (cf. 3). The result of the application of the first data processing measure is therefore generated selection information which contains or describes exactly one object 2.1 selected from the plurality of objects 2 in the first orientation and / or position described by the detection data. The selection information generated therefore contains or describes (precisely) the object 2.1, which by applying the first data processing measure from the A plurality of objects 2 located in the first orientation and / or position, described by the detection data, has been selected.

The application of the first data processing measure can involve the application of at least one selection algorithm, possibly forming a component of a selection software. B. involve an image processing algorithm. A corresponding selection or image processing algorithm can be set up to select precisely one object 2.1 from the plurality of objects 2 in the first alignment and / or position described by the detection data.

The selection algorithm can be used as part of a machine learning process, i. H. via a method of machine learning. As part of a corresponding machine learning process, one or more artificial neural networks with one or more intermediate layers implemented between an input and an output layer can be or have been used.

In a third step of method S3 of the method, a second data processing measure is used in order to process the selection information generated in the second step of method S2. The selection information generated in the second step of the method S2 is accordingly processed in the third step of the method S3 on the basis of a second data processing measure. The application of the second data processing measure carried out in particular by the or another data processing device implemented in hardware and / or software provides at least one coordinate K for handling the precisely one object 2.1 described by the selection information by means of a handling element 6 of the

Handling device 7 of the industrial robot 3 (see. Fig. 4). The result of the application of the second data processing measure is therefore generated coordinate information which contains at least one coordinate K for handling the precisely one object 2.1 described by the selection information by means of a handling element 6 of

Handling device 7 of the industrial robot 3 contains or describes. The coordinate information generated therefore contains or describes one or more coordinates - these are, for example, world coordinates related to the respective object 2.1 - for handling the precisely one object 2.1 described by the selection information by means of a handling element 6 of the handling device 7 of the industrial robot 3.

The application of the second data processing measure can include the application of at least one, possibly a component of a determination software,

Determination algorithm, this can be, for. B. involve an image processing algorithm. A corresponding determination or image processing algorithm can be set up to determine suitable coordinates on the object 2.1 described by the selection information, at which the object 2.1 can be handled by means of a handling element 6 of the handling device 7 of the industrial robot 3, i. h e.g. grab, lets.

In a fourth step of method S4 of the method, the coordinate information generated in the third step of method S3 is used as a basis for controlling the operation of the industrial robot, so that the industrial robot 3 detects the respective handling element 6 of the handling device 7 at the coordinate or coordinates K described by the coordinate information starts to convert the object 2.1 from the first orientation and / or position into a second orientation and / or position. The second alignment and / or position can be, for. B. be specified by a user or programmer of the industrial robot or will be.

The control of the operation of the industrial robot 3 accordingly typically includes handling precisely the one object 2.1, which is described by the selection information, at the coordinate or coordinates K which are described by the coordinate information, or results in such.

The control of the operation of the industrial robot 3 on the basis of the coordinate information can include at least one interaction of a handling element 6 of the handling device 7 of the industrial robot 3 with the object 2.1 described and thus selected by the respective selection information at or in the area of the coordinate (s) K described by the coordinate information include. A corresponding interaction can be a, z. B. realized by gripping (“picking”) the object 2.1 described by the respective selection information and thus selected at the coordinate (s) K described by the respective coordinate information. In particular, controlling the operation of the industrial robot 3 on the basis of the coordinate information can convert the object 2.1 described and thus selected by the respective selection information from the first orientation and / or position to a second orientation and / or position by means of a handling element 6 of the handling device 7 of the Include industrial robots 3.

The main advantage of the method is that already in the second step of the method S2 a corresponding selection of exactly one object 2.1 is selected from the plurality of detected objects 2 described by the detection data. In this way, the amount of data to be processed further to determine the respective coordinates K can be significantly reduced or limited to a single object, namely the respectively selected object 2.1, which also reduces the data processing resources required to generate the corresponding control information, ie in particular the memory and Computing resources, significantly reduced.

The method therefore makes it possible in the second step of method S2, and thus comparatively early, to considerably reduce the amount of data to be processed to generate the corresponding coordinate information by applying the first data processing measure from the plurality described by the acquisition data to the first alignment and / or position located objects 2 exactly one object 2.1 is selected. As a result of the selection made in the second step of the method S2 from exactly one object 2.1 from the plurality of objects 2 in the first orientation and / or position described by the detection data, the amount of data to be processed subsequently is based on precisely the selected object 2 -1 reduced; the data to be processed further defined by the selection information are thus limited or concentrated to the respective precisely one selected object 2.1, so that further processing of the data is limited or concentrated only to the respective precisely one selected object 2.1.

The first data processing measure can be implemented by a single or multi-layered first artificial neural network. A corresponding first artificial neural network has at least one intermediate layer located between an input and an output layer. In principle, artificial neural networks configured in a (comparatively) simple or (comparatively) complex manner can be used to implement the first data processing measure.

As an alternative or in addition, the second data processing measure can be implemented by a single or multi-layered second artificial neural network. A corresponding second artificial neural network has at least one intermediate layer located between an input and an output layer. In principle, artificial neural networks configured in a (comparatively) simple or (comparatively) complex manner can be used to implement the second data processing measure.

Respective data processing devices for applying the first and second data processing measures can therefore be set up to implement at least one first artificial neural network and / or at least one second artificial neural network or be or will be implemented by corresponding first and / or second artificial neural networks. The respective data processing device can form part of the control device 11 for controlling the operation of the industrial robot 3. When using the first data processing measure, at least one selection criterion can be taken into account. The selection of exactly one object 2.1 from the plurality of objects 2 in the first orientation and / or position described by the detection data using the first data processing measure can therefore be carried out on the basis of at least one selection criterion.

As a corresponding selection criterion, for. B. absolute alignment information and / or absolute position information describing an absolute alignment and / or position of at least one object 2 of the objects 2 located in the first alignment and / or position can be used.

Alternatively or in addition, a selection criterion such as B. a relative orientation and / or position of at least one object 2 of the objects 2 in the first orientation and / or position to at least one further object 2 of the objects 2 in the first orientation and / or position describing relative orientation information and / or relative position information be used.

Alternatively or in addition, an approach movement or approach vector of a handling element 6 of the handling device 7 of the industrial robot 3 for approaching at least one object 2 of the type shown in the first Approach information describing alignment and / or position of objects 2 can be used.

Alternatively or additionally, dimensional information describing at least one geometrical-constructive dimension of at least one object 2 of the objects 2 located in the first orientation and / or position can be used as a corresponding selection criterion.

Alternatively or additionally, shape information describing at least one geometric-constructive shape (spatial shape) of at least one object 2 of the objects 2 located in the first orientation and / or position can be used as a corresponding selection criterion.

As an alternative or in addition, color information describing the coloring of at least one object 2 of the objects 2 located in the first alignment and / or position can be used as a corresponding selection criterion.

Alternatively or in addition, a corresponding selection criterion can use surface information describing the surface, in particular the surface properties, ie in particular the mechanical and / or optical surface properties, of at least one object 1 of the objects 2 located in the first orientation and / or position will.

Alternatively or in addition, mass information describing the mass, in particular a center of gravity, of at least one object 2 of the objects 2 located in the first orientation and / or position can be used as a corresponding selection criterion.

Alternatively or in addition, a corresponding selection criterion can be used as a corresponding selection criterion describing the genre of at least one object 2 of the objects 2 located in the first orientation and / or position, sometimes also referred to as “format” in pick-and-place applications.

When using the second data processing measure, at least one determination criterion can be taken into account. The determination of the at least one coordinate for handling the precisely one object 2.1 described by the selection information using the second data processing measure can therefore be carried out on the basis of at least one determination criterion.

As a corresponding determination criterion, for. B. absolute alignment information and / or absolute position information describing an absolute alignment and / or position of the respectively selected object 2.1 can be used.

Alternatively or in addition, z. B. relative orientation information and / or relative position information describing a relative orientation and / or position of the respectively selected object 2.1 to at least one further object 2 of the objects 2 located in the first orientation and / or position can be used.

Alternatively or in addition, an approach movement or approach vector of a handling element 6 of the handling device 7 of the industrial robot 3 describing approach information for approaching the respectively selected object 2.1 can be used as a corresponding determination criterion, in particular from an ACTUAL position and / or ACTUAL orientation .

Alternatively or in addition, dimensional information describing at least one geometrical-constructive dimension of the respectively selected object 2.1 can be used as a corresponding determination criterion.

Alternatively or in addition, form information describing at least one geometrical-constructive shape (spatial shape) of the respectively selected object 2.1 can be used as a corresponding determination criterion.

As an alternative or in addition, a corresponding determination criterion can be the color of the selected object 2.1 descriptive color information can be used.

Alternatively or in addition, the corresponding determination criterion can include the surface, in particular the surface quality, i.e. H. in particular the mechanical and / or optical surface properties of the respectively selected object 2.1 descriptive surface information can be used.

Alternatively or in addition, mass information describing the mass, in particular a center of gravity, of the respective selected object 2.1 can be used as a corresponding determination criterion.

Alternatively or additionally, handling information describing the type of handling of the respectively selected object 2.1 by a handling element 6 of the handling device 7 of the industrial robot 3 can be used as a corresponding determination criterion.

Alternatively or in addition, handling information describing the handling surface of the respectively selected object 2.1 by a handling element 6 of the handling device 7 of the industrial robot 3 can be used as a corresponding determination criterion.

As an alternative or in addition, a genus of the respective selected object 2.1 describing genre information can be used as a corresponding determination criterion.

The method typically does not end after the described steps S1-S4 have been carried out for a first selected object 2.1 which, within the framework of or after carrying out the above-described steps of the method, e.g. B. selected from the plurality of objects 2 located in the first orientation and / or position and removed as the selected object in the course of controlling the industrial robot from the plurality of objects 2 located in the first orientation and / or position and transferred to a second orientation and / or position has been implemented. Correspondingly, after corresponding selection information and coordinate information for a first selected object 2.1 has been generated, corresponding selection information and coordinate information for at least one further object 2 is typically generated from the plurality of objects 2 remaining in the first orientation and / or position. The method is typically carried out until appropriate selection information and coordinate information has been successively generated for each object 2 from the large number of objects 2 located in the first orientation and / or position. The process can be interrupted or stopped when a termination condition is fulfilled or present. A corresponding termination condition can e.g. B. be fulfilled or present when there is only a certain number of objects 2 in the first orientation and / or position or when there is no longer any object 2 in the first orientation and / or position. Before step S1 of acquiring the plurality of objects 2 located in a first orientation and / or position and generating acquisition data describing the acquired plurality of objects 2 located in the first orientation and / or position, the method can include a step of providing a plurality on objects 2 in a first orientation and / or position on a stationary or moving base, in particular a feed conveyor device 4. In any case, the provision, which can include feeding the respective objects 4 into the action area 5 of a handling element 6 of the handling device 7 of the industrial robot, can therefore also optionally represent a step of the method.

In all exemplary embodiments, the corresponding objects 2 can be objects to be packaged in a package. The type of packaging results from the type of objects to be packed. Merely by way of example, food that can be separated, such as. B. candy, referenced as possible objects 2.

Claims

PATEN TAN SPEECH

1. A method for controlling the operation of an industrial robot (3) configured in particular to carry out pick-and-place or isolation tasks, the industrial robot (3) having a handling device (7) which has at least one handling element that can be moved in at least one degree of freedom of movement (6) for handling an object (2) to be converted from a first orientation and / or position into a second orientation and / or position, characterized by the following steps:

- Detecting a plurality of objects (2) located in a first orientation and / or position and generating detection data describing the detected plurality of objects (2) located in the first orientation and / or position;

- Applying a first data processing measure for processing the acquisition data, the application of the first data processing measure providing a selection of exactly one object (2.1) from the plurality of objects (2) in the first orientation and / or position described by the acquisition data, and generating a the selection information describing exactly one object (2) selected from the plurality of objects (2) in the first orientation and / or position described by the detection data;

- Applying a second data processing measure for processing the selection information, the application of the second data processing measure supplying at least one coordinate for handling the precisely one object (2.1) described by the selection information by means of at least one handling element (6) of the handling device (7) of the industrial robot (3) , and generating coordinate information describing the at least one coordinate;

- Controlling the operation of the industrial robot (3) to carry out a task, in particular a pick-and-place or separation task, on the basis of the coordinate information.

2. The method according to claim 1, characterized in that the detection of the plurality of objects (2) located in the first orientation and / or position is carried out by means of at least one optical detection device (10).

3. The method according to claim 2, characterized in that at least one optical detection device (10) is used, which on the industrial robot (3), in particular on an immovable or immobile portion of the industrial robot, in particular on a portion of a for receiving functional and / or supply components of the industrial robot (3) set up housing device (3.1) of the industrial robot (3), is arranged or designed.

4. The method according to claim 2 or 3, characterized in that at least one optical detection device (10) is used, which a defined optical detection area (12) within which objects (2) can be detected by means of the optical detection device (10) or detected will have.

5. The method according to claim 4, characterized in that the defined optical detection area (12) of the at least one optical detection device (10) at least in sections, optionally completely, within a defined action area (5) within which the at least one handling element (6) of the Handling device (7) is movable.

6. The method according to any one of the preceding claims, characterized in that the first data processing measure is implemented by at least one single or multi-layered first artificial neural network.

7. The method according to any one of the preceding claims, characterized in that the second data processing measure is implemented by at least one single or multi-layered or -stepped second artificial neural network.

8. The method according to claim 6 and 7, characterized in that at least one data processing device set up for applying the first and second data processing measures is used to apply the first and second data processing measures.

9. The method according to any one of the preceding claims, characterized in that the selection made by means of the first data processing measure exactly one object (2.1) from the plurality of objects (2) in the first orientation and / or position described by the detection data is based on at least a selection criterion is carried out.

10. The method according to claim 9, characterized in that the selection criterion used is an absolute orientation and / or position of at least one object (2) of the objects (2) in the first orientation and / or position describing absolute orientation information and / or absolute position information; and / or a relative alignment and / or position of at least one object (2) of the objects (2) in the first alignment and / or position to at least one further object (2) of the objects in the first alignment and / or position ( 2) descriptive relative orientation information and / or relative position information; and / or an approach movement or approach vector of a handling element required in particular from an ACTUAL position and / or ACTUAL orientation Handling device of the industrial robot for approaching at least one object (2) of the objects (2) in the first orientation and / or position, describing approach information; and / or dimension information describing at least one dimension of at least one object (2) of the objects (2) located in the first orientation and / or position; Shape information describing at least one shape of at least one object (2) of the objects (2) located in the first orientation and / or position; and / or color information describing the coloring of at least one object (2) of the objects (2) located in the first alignment and / or position; and / or surface information describing the surface, in particular the surface quality, of at least one object (2) of the objects (2) located in the first orientation and / or position; and / or mass information describing the mass, in particular a center of mass, of at least one object (2) of the objects (2) located in the first orientation and / or position; generic information describing the genus of at least one object (2) of the objects (2) located in the first orientation and / or position can be used.

11. The method according to any one of the preceding claims, characterized in that the determination of the at least one coordinate for handling the precisely one object (2.1) described by the selection information, which is carried out by means of the second data processing measure, is carried out on the basis of at least one determination criterion.

12. The method according to claim 11, characterized in that an absolute alignment and / or position of the selected object (2.1) describing absolute alignment information and / or absolute position information is used as the determination criterion; and / or a relative orientation and / or position of the selected object (2.1) to at least one further object (2) of the objects (2) in the first orientation and / or position describing relative orientation information and / or relative position information; and / or an approach movement or approach vector of the at least one handling element (6) of the handling device (7) of the industrial robot (3) describing approach information for approaching the selected object, which is required in particular from an ACTUAL position and / or ACTUAL orientation (2.1); and / or dimension information describing at least one dimension of the selected object (2.1); shape information describing at least one shape of the selected object (2.1); and or color information describing the coloring of the selected object (2.1); and / or surface information describing the surface, in particular the surface texture, of the selected object (2.1); and / or mass information describing the mass, in particular a center of mass, of the selected object (2.1); and / or handling information describing the type of handling of the selected object (2.1) by at least one handling element (6) of the handling device (7) of the industrial robot; and / or handling information describing the handling surface of the selected object (2.1) by at least one handling element (6) of the handling device (7) of the industrial robot (3); and / or a genre information describing the genre of the selected object (2.1) can be used.

13. The method according to any one of the preceding claims, characterized in that the control of the operation of the industrial robot (3) on the basis of the

Coordinate information comprises an interaction of the at least one handling element (6) of the handling device (7) of the industrial robot (3) with the object (2.1) described by the respective selection information at or in the area of the at least one coordinate described by the coordinate information.

14. The method according to claim 13, characterized in that controlling the operation of the industrial robot (3) on the basis of the coordinate information converting the object (2.1) described by the respective selection information from a first orientation and / or position into a second orientation and / or position by means of the at least one handling element (6) of the handling device (7) of the industrial robot (3).

15. The method according to any one of the preceding claims, characterized in that after generating corresponding selection information and coordinate information for a first object (2) from the plurality of objects (2) remaining in the first orientation and / or position, corresponding selection information and coordinate information is generated for a further object (2) from the plurality of objects (2) located in the first orientation and / or position.

16. The method according to any one of the preceding claims, characterized in that a collaborative industrial robot (“cobot”) is used as the industrial robot (3).

17. Industrial robot (3), in particular collaborative industrial robot (“cobot”), comprising at least one handling device (7) which has at least one in at least one Manipulation element (6) that can be moved to a degree of freedom of movement for handling an object (2) to be converted from a first orientation and / or position to a second orientation and / or position and a control device (11) implemented in particular in terms of hardware and / or software to control operation of the industrial robot (3), characterized in that the control device (11) is set up to carry out the method according to one of the preceding claims.

18. Arrangement (1) for converting objects (2) from a first orientation and / or position into a second orientation and / or position, comprising:

- At least one industrial robot (3) according to claim 17; and

- At least one feed device (4) for feeding objects (2), in particular objects (2) located in a first orientation and / or position, into an action area (5) of at least one handling element (6) of the handling device (7) of the industrial robot (3), and / or

- At least one removal device (5) for removing objects (2), in particular objects (2) which have been converted into the second orientation and / or position by means of the industrial robot (3).