EP3850545A1

EP3850545A1 - Method and apparatus for identifying an article

Info

Publication number: EP3850545A1
Application number: EP19773741.4A
Authority: EP
Inventors: Manuel Kiefer; Willi Pönitz
Original assignee: Trumpf Werkzeugmaschinen SE and Co KG
Current assignee: Trumpf Werkzeugmaschinen SE and Co KG
Priority date: 2018-09-12
Filing date: 2019-09-03
Publication date: 2021-07-21
Also published as: US11972600B2; US20230118305A1; WO2020053023A1; DE102018215538A1; CN112703508A; JP2022500757A

Abstract

The invention relates to a method and an apparatus (10) for recognizing an article. This involves at least one shot (14), in particular in the form of a photograph, of the article being produced. The shot (14) is used to ascertain shot features (22) by means of a shot extraction algorithm (20). Stored article data (12) are used to ascertain article features (28) and to compare them with the shot features (22) in order to output association information (36). There is in particular provision according to the invention for a user rating (42) to be provided to improve both the shot extraction algorithm (20) and the article extraction algorithm (26). Alternatively or additionally, there is in particular provision according to the invention for both the shot extraction algorithm (20) and the article extraction algorithm (26) to be produced on the basis of interconnected, preferably weighted, data aggregation routines.

Description

Method and device for identifying an object

Background of the Invention

The invention relates to a method and a device for identifying an object. The invention further relates to the use of such a method or the use of such a device in industrial production.

In the production of objects, in particular in the production of sheet metal parts, an object, in particular a manufactured sheet metal part, is often found that cannot be assigned to a specific order. Disposing of the item is a waste of resources and also carries the risk that the item will be missing from a particular job. However, the completely manual identification of the object is complex and can often only successfully carried out by experienced employees who are not always present or who have to be released from other tasks in order to identify the object. A fully automated identification of an object, on the other hand, often fails due to inadequate recordings of the object or inadequate identification programs.

Object of the invention

It is therefore an object of the invention to provide a method or a device which enables an effective assignment of an object found by chance. Furthermore, it is an object of the invention to provide a corresponding use of the method or the device.

Description of the invention

This object is achieved according to the invention by a method according to patent claim 1, a device according to patent claim 12 or a use according to patent claim 16. The subclaims represent preferred developments.

The invention thus relates to a method for identifying an object. The object can be embodied in the form of an essentially two-dimensional object, for example in the form of a flat sheet metal part, or in the form of a three-dimensional object, for example in the form of a sheet metal part with deformations. Identification is done by assigning a recording of the object to the stored object data. This object data can be stored in the form of CAD data. The process has the following process steps:

A) creating the recording or recording of the object with an image capture device, wherein the image capture device can be designed in the form of a camera;

B) deriving recording features from the recording using a recording extraction algorithm;

C) deriving object features from the stored object data using an object extraction algorithm; D) comparing the recording features with the object features using a comparison algorithm; and

E) Outputting assignment information of the recording to the stored object data.

According to the invention, it is therefore provided to output at least one item of information relating to the assignment of the object to be identified to stored object data. The information is preferably given in the form of probability information, so that it can be seen with what probability the object corresponds to the stored object data. The output can e.g. on a screen, data glasses, or as data transfer. Advantageously, a plurality of assignment information items can be output with an indication of the probability. According to the invention, features are extracted from the image as well as from the stored object data. This enables a particularly reliable assignment of the recording to the stored object data and thus a particularly high probability of identifying the object.

The method preferably has the following method step:

F) Reading out a user rating of the assignment information.

The method then provides for an evaluation of a user with regard to this assignment information. The evaluation by the user can take place, for example, by data transmission or by an input device, for example a button, touch screen, voice recording, speech recognition or similar input devices. If, for example, the user is shown two or more identification matches, sorted by probability, the user can select the one that they really agree with. He has given a user rating. This can make it possible to improve future allocation information. A recording extraction algorithm can insert data of the recording, process it depending on predefined recording parameters and output recording characteristics in the form of the processed data. The predefined recording parameters can have so-called weighted variables. Their function and determination is explained below.

The method can further preferably have the following method step:

H) changing, in particular improving, in particular optimizing, parameters of the exposure extraction algorithm, the object extraction algorithm and / or the comparison algorithm on the basis of the

User rating.

The parameters can have so-called weighted variables. Their determination and function is explained below.

Before, during and / or after method step H), the method step is preferably carried out as follows:

G) storing the user rating of the assignment information in a user rating result memory.

Saving the user rating enables many user ratings to be collected, as a result of which the exposure extraction algorithm, the object extraction algorithm and / or the comparison algorithm can be significantly improved.

The user evaluation result memory can be cloud-based. Cloud-based here means a storage device, in particular a locally remote, preferably anonymized, storage device in which user ratings of more than one, advantageously several, or several thousand different users are stored. As a result, different users can contribute to the optimization of the process regardless of the manufacturing location. It was recognized that the described method was a resounding success, that is to say assignment information with the correct assignment to the Most likely to get it only after tens of thousands, especially hundreds of thousands of user reviews have been read out. Such an amount of data is often not available for a single manufacturing facility in a year. Accordingly, the procedure would probably have remained uninteresting.

The recording can be made or recorded in the wavelength range visible to the human eye. Alternatively or additionally, the recording can be made in the wavelength range that is not visible to the human eye, for example in the IR range, in the UV range and / or in the ultrasound range.

In method step A), several images of the object can be recorded with the image capturing device. In method step B), the recording features can be extracted from several recordings. In method step E), assignment information can be output from several recordings of the stored objects. It has been shown that the creation and processing of a plurality of recordings considerably improves the quality of the assignment information for a user, since in particular the influence of artifacts caused by the recording position is reduced.

The image extraction algorithm, the object extraction algorithm and / or the comparison algorithm preferably have an algorithm with a plurality of data aggregation routines. A data aggregation routine can be designed to aggregate several “determined data” into a new data packet. The new data packet can have one or more numbers or vectors. The new data packet can be made available in whole or in part to other data aggregation routines as "determined data". "Determined data" can be, for example, recording data, object data, or data packets made available by one of the data aggregation routines. The recording / extraction algorithm, the object extraction algorithm and / or the is / are particularly preferred Comparison algorithm in the form of an algorithm with several connected data aggregation routines. In particular, several hundred, in particular several thousand, of such data aggregation routines can be connected to one another. This significantly improves the quality and speed of the algorithm / algorithms. The acquisition extraction algorithm, the object extraction algorithm and / or the comparison algorithm can have a function with weighted variables. One, in particular several, particularly preferably all data aggregation routines can be designed to combine, in particular multiply, a plurality of "determined data" with a weighted variable, and thus to convert the "determined data" into combined data "then the combined data 'aggregate to a new data packet, in particular to add. The change in the weighting, in particular the change in the weighted variables, is particularly preferably carried out on the basis of the user rating. In order to determine suitable weighted variables, the exposure extraction algorithm, the object extraction algorithm and / or the comparison algorithm can be run through with data, in particular exposure data and / or object data, the relationship of which is known in each case. The weighted variables can be determined, preferably in a first phase, separately for the exposure extraction algorithm, the object extraction algorithm and the comparison algorithm.

The recording features and object features can in this case themselves be data packets, in particular a plurality of structured data, in particular data vectors or data arrays, which in turn themselves contain "determined data" e.g. for the comparison algorithm, in particular for the data aggregation routines of the comparison algorithm. The precise structure of these recording features and object features can change, in particular improve, preferably optimize, through the mechanical evaluation of the user rating.

By changing, in particular improving, preferably op- timed, weighted variables are managed cloud-based, other users can use them in their algorithms and benefit from the method. The algorithms mentioned or a further secondary or higher-level algorithm can be designed to monitor and recognize when one or all of the algorithms, with a predetermined cluster, output association information that the user assesses as bad, and then output a negative message. The output can be visual, for example on a screen or in another suitable form, for example as data output. The monitoring algorithm can also be designed to respond to the output of such a negative message with an improvement routine that changes further properties or the interaction of one or more of the algorithms mentioned.

In a further preferred embodiment of the method, the object features are in the form of construction data, material data, surface texture data and / or thermal conductivity data. The comparison algorithm can include forming a dot product and / or forming a difference between the recording features and the object features. The aforementioned measures have proven to be particularly effective in assigning the recording characteristics to the object characteristics.

The output of the assignment information of the recording further preferably comprises the output of several assignment probabilities to different stored objects. A user can choose from various assignment options. The identification of the object can thus be carried out with a very high probability of success. The object according to the invention is further achieved by a device for identifying an object by assigning the object to stored object data, the device having the following:

a) an image capture device for taking a picture of the object;

b) a recording extraction unit with a recording extraction algorithm for extracting recording features from the recording;

c) an object extraction unit with an object extraction algorithm for extracting object features from the stored object data;

d) a comparison unit with a comparison algorithm for comparing the recording features with the object features;

wherein the recording extraction algorithm, the object extraction algorithm and / or the comparison algorithm is designed to be optimizable on the basis of the user rating; and

e) an output unit for outputting assignment information of the image created with the image capture device to the stored object data.

The device according to the invention is preferably designed to carry out a method described here.

The image capture device can be in the form of a camera, in particular for visible light.

The device can also have the following:

f) an input unit for reading out a user rating with regard to the assignment information.

Furthermore, the device can have the following: g) A user rating result storage for storing the user rating of the association information. The user evaluation result memory can be cloud-based. The recording extraction algorithm, the object extraction algorithm and / or the comparison algorithm can be designed in the form of an algorithm with a plurality of connected data aggregation routines.

The method and / or the device can be used particularly advantageously in industrial production with a computer-based production control for processing reflective objects, in particular sheet metal parts. In the past, the recordings of sheet metal parts with an image capturing device were often not of sufficient quality for identification, because contours are very difficult to distinguish from the background and light reflections simulate incorrect contours. Because of this, processes and devices which compared the recording data with object data without the algorithms described have so far not led to any success in industrial production. By reflective objects are meant objects with a smooth surface that reflect the light in such a way that when taking the picture, in addition to the contours, undesired light reflections of other objects can also occur. Examples of such reflective objects are metals, glass, plastics with a smooth surface, coated materials, such as coated plates made of plastic, wood, metal, glass, etc. It is particularly advantageous if the production control is at least partially cloud-based. Then parameters, in particular the weighted variables, for changing, in particular improving, in particular optimizing, the algorithms from a first production site can also be used in other production sites and vice versa. This means that a much larger database is available and the identification for each individual manufacturing company can be significantly improved. Further advantages of the invention result from the description and the drawing. Likewise, according to the invention, the features mentioned above and those which have been elaborated further can be used individually or in combination in any combination. The embodiment shown and described is not to be understood as conclusive, but rather has an exemplary character for the description of the invention.

Detailed description of the invention and drawing

FIG. 1 shows a schematic representation of an embodiment of the method according to the invention or the device according to the invention.

FIG. 1 shows a device 10 for identifying an object found without being assigned. The object can be in the form of a sheet metal part. Item data 12 are stored for the item. In order to assign the object found to the object data 12 and thereby identify it, at least one image 14 is created.

The object data 12 are preferably stored in the form of CAD data, in particular in an object database 16 in the form of a CAD database.

The image 14 is created with an image capture device 18. The image capturing device 18 can be designed in the form of a camera.

The receptacle 14 shown in FIG. 1 indicates that the assignment of such a receptacle 14 can be very difficult. This is particularly the case if - as shown - the receptacle 14 only shows a part of the object, the receptacle 14 is made against an uneasy background and / or the surface properties of the object make the receptacle 14 difficult to create. A recording extraction algorithm 20 is applied to the recording 14 in order to extract recording features 22. This is indicated schematically in FIG. 1 on the basis of the area framed in dash-dotted lines in the receptacle 14. The recording extraction algorithm 20 is in a recording Extraction unit 24 deposited. The recording extraction algorithm 20 can have data aggregation routines which are connected to one another, in particular weighted to one another. An item extraction algorithm 26 is applied to the item data 12 to extract item features 28. This is indicated schematically in FIG. 1 on the basis of the area framed in dot-dash lines in the item data 12. The object extraction algorithm 26 is stored in an object extraction unit 30. The object extraction algorithm 26 can have interconnected, in particular mutually weighted, data aggregation routines.

The recording features 22 and the item features 28 are fed to a comparison algorithm 32. The comparison algorithm 32 is stored in a comparison unit 34. The comparison algorithm 32 can have interconnected, in particular mutually weighted, data aggregation routines. The comparison algorithm 32 is preferably designed to form a scalar product or a difference between the recording features 22 and the object features 28.

Assignment information 36 is output as a result of the comparison algorithm 32. The assignment information is output in an output unit 38. As shown in FIG. 1, a plurality of item data (here three; not shown in FIG. 1 with a reference symbol) can be displayed, the assignment probability in each case (here 60%, 35% and 5%). for the respective item data. This considerably simplifies the assignment of the found object to object data for a user. In order to improve future assignments, ie to increase the quality of future assignment information, the device 10 has an input unit 40. The input unit 40 is designed to read out a user rating 42. The user rating 42 is then used to Extraction algorithm 20 to optimize the object extraction algorithm 26 and / or the comparison algorithm 32 or its parameters. The user rating 42 can be stored in a user rating result memory 44, so that the method or the device 10 can be optimized with a large number of user ratings 42. It is particularly preferred that the user evaluation result memory 44 is cloud-based. In this way, user evaluations 42 can be incorporated into the optimization of the method or device 10 across devices.

In other words, the invention relates to a method and a device 10 for recognizing an object. At least one picture 14, in particular in the form of a photograph, is made of the object. On the basis of the recording 14, 20 recording features 22 are determined by means of a recording extraction algorithm 20. On the basis of stored object data 12, object features 28 are determined and compared with the recording features 22 in order to output assignment information 36. According to the invention, it is provided in particular to provide a user rating 42 both for improving the exposure extraction algorithm 20 and for the object extraction algorithm 26. As an alternative or in addition to this, it is provided according to the invention in particular to design both the exposure extraction algorithm 20 and the object extraction algorithm 26 based on interconnected, preferably weighted, data aggregation routines.

Reference list

10 device

12 item data

14 recording

16 Item database

18 image capture device

20 recording extraction algorithm

22 recording characteristics

24 recording extraction unit

26 Object extraction algorithm

28 Item Features

30 item extraction unit

32 comparison algorithm

34 comparison unit

36 Allocation information

38 output unit

40 input unit

42 user rating

44 user evaluation result memory

Claims

1 Method for identifying an object by assigning a

Recording (14) the object for stored object data (12), with the process steps:

A) Create the recording (14) of the object with a

Image capture device (18);

B) extracting recording features (22) from the recording (14) using a recording extraction algorithm (20);

C) extracting object features (28) from the stored object data (12) using an object extraction algorithm (26); wherein step C) before, during and / or after

Process steps A) and B) can be carried out and the following process steps are then carried out:

D) comparing the recording features (22) with the object features (28) using a comparison algorithm (32);

E) Output of assignment information (36) of the receptacle (14) to the stored object data (12).

2 Method according to claim 1 with the further step:

F) Reading out a user rating (42) of the

Association information (36).

3 The method of claim 1 or 2 with the further step:

H) Optimizing the recording extraction algorithm (20), the

Object extraction algorithm (26) and / or the comparison algorithm (32) based on the user rating (42).

4. The method according to claim 3, in which the following method step is carried out before, during and / or after method step H):

G) storing the user rating (42) of the

Assignment information (36) in a user evaluation result memory (44).

5. The method of claim 4, wherein the user evaluation result memory (44) is cloud-based.

6. The method according to any one of the preceding claims, wherein the

Creation of the recording (14) takes place in the wavelength range visible to the human eye.

7. The method according to any one of the preceding claims, in which

Method step A) several pictures (14) of the object are taken with the image capture device (18), in

Process step B) the recording features (22) are extracted from the plurality of recordings (14) and, in process step E), assignment information (36) of the plurality of recordings (14) to the stored objects is output.

8. The method according to any one of claims 2 to 7, wherein the recording-extraction algorithm (20), the object extraction algorithm (26) and / or the comparison algorithm (32) has / have interconnected data aggregation routines.

9. The method according to any one of claims 2 to 8, wherein the object features (28) in the form of construction data, material data,

Surface quality data and / or thermal conductivity data are available.

10. The method according to any one of claims 2 to 9, wherein the comparison algorithm (32) comprises forming a dot product and / or forming a difference between the recording features (22) and the object features (28).

11. The method according to any one of the preceding claims, wherein the

Outputting the assignment information (36) of the receptacle (14) comprises outputting assignment probabilities to various stored objects.

12. Device (10) for identifying an object by assigning the object to stored object data (12), the device (10) comprising the following: a) an image capturing device (18) for creating an

Receiving (14) the object; b) a recording extraction unit (24) with a recording extraction algorithm (20) for extracting recording features (22) from the recording (14); c) an object extraction unit (30) with an object extraction algorithm (26) for extracting object features (28) from the stored object data;

d) a comparison unit (34) with a comparison algorithm (32) for comparing the recording features (22) with the object features (28); wherein the recording extraction algorithm (20), the object

Extraction algorithm (26) and / or the comparison algorithm (32) can be optimized based on a user evaluation (42);

e) an output unit (38) for outputting a

Association information (36) with the

Image capture device (18) created image (14) for the stored object data (12).

13. The apparatus of claim 12, wherein the apparatus (10) further

Has the following:

f) an input unit (40) for reading out a user rating (42) of the assignment information (36).

14. The apparatus of claim 12 or 13, wherein the device (10)

also has:

g) a user rating result memory (44) for storing the user rating (42) of the assignment information (36).

15. The device according to one of claims 12 to 14, in which the image extraction algorithm (20), the object extraction algorithm (26) and / or the comparison algorithm (32) have / have data aggregation routines connected to one another.

16. Use of a method or a device according to one of the

preceding claims in an industrial production, wherein in particular reflective objects, preferably sheet metal parts, are processed in industrial production.