DE102022204297A1 - Method for training a machine learning algorithm - Google Patents

Abstract

The invention relates to a method for training a machine learning algorithm, which maps input data into output data, the input data being image data, and the method (1) having the following steps: providing labeled image data, the labeled image data from at least one optical sensor, and wherein the labeled image data represents training data for training the machine learning algorithm (2) and training the algorithm of the machine learning algorithm based on the training data, wherein training the machine learning algorithm involves applying a first cost function which compares predictions with ground truth information, and wherein training the machine learning algorithm involves applying a second cost function which, for each of the at least one optical sensor, has a similarity of predictions from the corresponding optical sensor with respect thereto Scene generated image data evaluated (3).

Description

The invention relates to a method for training a machine learning algorithm, wherein training the machine learning algorithm includes evaluating the prediction symmetry.

Machine learning algorithms are based on the fact that statistical methods are used to train a data processing system so that it can carry out a specific task without it having originally been explicitly programmed for this purpose. The goal of machine learning is to construct algorithms that can learn from data and make predictions. These algorithms create mathematical models that can be used, for example, to classify data.

For example, intelligent or smart camera systems are increasingly being used to monitor highways, buildings, fences or offices. Smart camera systems differ from conventional camera systems in that the image data recorded is evaluated directly by the camera, for example based on a machine learning algorithm, and the results of the image processing are output.

In order to be able to use such smart camera systems, it is important to first calibrate or set them up. In addition to the image data, further information is required for this, for example information about intrinsic calibration, orientation, or recognized objects or boundaries. This information is often generated and entered manually, although the installation time of smart camera systems can be reduced and they can be made insensitive to potential changes in orientation if this process were automated.

Machine learning algorithms trained in deep learning are particularly suitable for the automated estimation of this information, especially since they are able to interpret even complex scene content. However, the disadvantage here is that it is difficult to check the correctness of the corresponding information used for calibration, since there is usually no highly accurate calibration information available. Rather, many parameters must be derived based on a few clues in image data, which then form ground truth information or basic knowledge, i.e. knowledge about a correct classification, although this information can be very uncertain and does not reflect the actual information.

From the publication US 6,985,780 B2 A smart camera system is known which provides focused images to an operator at a host computer by processing digital images at the imaging site before sending them to the host computer, wherein the smart camera includes a resident digital signal processor for pre-processing digital images transmitting the images to the host, and wherein the preprocessing includes image feature extraction and filtering, convolution and deconvolution methods, parallax and perspective image error correction, and image compression.

The invention is therefore based on the object of specifying an improved method for training a machine learning algorithm, with which the influence of false or inaccurate ground truth information can be weakened.

The task is solved by a method for training a machine learning algorithm according to the features of patent claim 1.

The task is also solved by a control device for training a machine learning algorithm according to the features of patent claim 8.

Disclosure of the invention

According to one embodiment of the invention, this object is achieved by a method for training a machine learning algorithm which maps input data into output data, the input data being image data, and the method comprising providing labeled image data, the labeled image data from at least one optical sensor, and wherein the labeled image data represents training data for training the machine learning algorithm, and training the algorithm of the machine learning algorithm based on the training data, wherein training the machine learning algorithm comprises applying a first Cost function that compares predictions with ground truth information, and wherein training the machine learning algorithm involves applying a second cost function that, for each of the at least one optical sensor, has a similarity of pre dictions based on image data supplied by the corresponding optical sensor regarding the same scene.

A sensor, which is also referred to as a detector, (measurement or measuring) sensor or (measuring) sensor, is a technical component that records certain physical or chemical properties and/or the material properties of its environment qualitatively or quantitatively as a measured variable can. Optical sensors refer in particular to sensors that are designed to record or capture data about their surroundings, which can be reproduced as an image or graphic using a special program.

Image data also refers to the data that can be reproduced as an image or graphic using a special program. Labeled image data is understood to mean further labeled or known image data, that is, image data in which labels and/or features have already been identified.

A cost function or loss is also understood to mean a loss or an error between the determined output values of the deep learning-based machine learning algorithm and reference values, for example corresponding actual conditions or actually measured data.

Predictions are understood to be predictions made by the machine learning algorithm regarding individual labels and/or features or output data that the algorithm outputs based on predetermined input data.

The fact that the training of the machine learning algorithm is based not only on the classic first cost function but also on an additional second cost function, which evaluates the predictions on image material from the same camera with regard to the similarity or symmetry of the predictions, has the advantage that the method is more robust against high noise levels and variances in ground truth information.

Overall, an improved method for training a machine learning algorithm is provided, with which the influence of false or inaccurate ground truth information can be weakened.

In one embodiment, the method further comprises generating additional image data by at least one augmentation method, the training data comprising the labeled image data and the additional image data.

Augmentation methods are understood to be methods for generating further training data, with augmentation methods in particular being known which generate new data by slightly modifying known labeled data.

This means that the method can also be used in applications where large amounts of data are required to train the corresponding machine learning algorithm, for example when calibrating smart camera systems, without first having to manually determine new ground truth information, with the influence on this Training can be weakened by repeatedly introducing incorrect or inaccurate parameters as part of the augmentation.

In addition, the second cost function can have a weighting which is designed to prevent very frequently generated training data from being taken into account correspondingly frequently when training the machine learning algorithm.

Weighting refers to the evaluation of individual influencing variables of a mathematical model, for example in terms of their importance and reliability.

This can prevent an example from gaining the upper hand when training the machine learning algorithm, which has been introduced very frequently, for example through augmentation or augmentation methods, which in turn can reduce the risk of overfitting.

Furthermore, the second cost function can be designed to compare the predictions with an average value formed from at least some of the predictions.

The fact that the average value is formed from at least some of the predictions means that a corresponding average value is formed from at least some of the corresponding predictions for all training data.

This allows resources required to execute the second cost function and thus to train the algorithm, for example Memory and/or processor capacities can be reduced. In addition, machine learning algorithms are usually trained in batches, i.e. on parts of the training data, so that all examples or training data are usually never evaluated at the same time. By comparing the predictions with the average value, it can be ensured that the predictions do not have to be compared directly with all examples, but only with the average value.

The first cost function and the second cost function can also be combined into a common cost function, with training the machine learning algorithm comprising applying the common cost function.

The fact that the first cost function and the second cost function are combined into a common cost function means that a single cost function is formed based on the first cost function and the second cost function, for example by expanding the classic first cost function by a further term with respect to the second cost function becomes.

By combining the first cost function and the second cost function into a common cost function, the resources required to execute the method or to train the machine learning algorithm can be reduced and at the same time the time required to train the machine learning algorithm can also be shortened.

Further, the training data may be divided into multiple parts, wherein the step of training the machine learning algorithm for each part of the training data comprises sequentially applying the first cost function and the second cost function to the corresponding part of the training data.

The fact that the first cost function and the second cost function are applied successively to a portion of the training data means that the first cost function and the second cost function are applied successively to at least the portion of the training data.

This means that both cost functions can also be evaluated alternately and exclusively.

A further embodiment of the invention also provides a method for calibrating at least one optical sensor, wherein the method comprises calibrating the at least one optical sensor based on a machine learning algorithm, and wherein the machine learning algorithm is carried out by a method described above Training a machine learning algorithm has been trained.

Thus, a method for calibrating at least one optical sensor is specified, which is trained on a machine learning algorithm that is trained by an improved method for training a machine learning algorithm with which the influence of false or inaccurate ground truth information can be attenuated was based. In particular, the fact that training the machine learning algorithm is based not only on the classic first cost function but also on an additional second cost function, which evaluates the predictions on image material from the same camera with regard to the similarity or symmetry of the predictions, has the advantage that the method is more robust against high Noise levels and variances in ground truth information is.

With a further embodiment of the invention, a control device for training a machine learning algorithm, which maps input data into output data, the input data being image data, is also specified, the control device being a provision unit which is designed to provide labeled image data, wherein the labeled image data comes from at least one optical sensor, and wherein the labeled image data represents training data for training the machine learning algorithm, and a training unit which is designed to train the machine learning algorithm based on the training data, wherein the Training the machine learning algorithm includes applying a first cost function that compares predictions with ground truth information, and wherein training the machine learning algorithm includes applying a second cost function that, for each of the at least one optical sensor, Similarity of predictions on image data supplied by the corresponding optical sensor regarding the same scene is evaluated.

An improved control device for training a machine learning algorithm is thus specified, with which the influence of incorrect or inaccurate ground truth information can be weakened. The fact that the machine learning algorithm is trained not only on the classic first cost function but also on an additional second cost function that bases the predictions on the same image material Camera based on the similarity or symmetry of the predictions, has the advantage that the method is more robust against high levels of noise and variances in ground truth information.

In one embodiment, the control device further has an augmentation unit which is designed to generate additional image data using at least one augmentation method, the training data comprising the labeled image data and the additional image data. This means that the control device can also be used in applications where large amounts of data are required to train the corresponding machine learning algorithm, for example when calibrating smart camera systems, without first having to manually determine new ground truth information, with the influence on this Training can be weakened by repeatedly introducing incorrect or inaccurate parameters as part of the augmentation.

The second cost function can in turn have a weighting, which is designed so that very frequently generated training data is taken into account correspondingly frequently when training the machine learning algorithm. This in turn can prevent one example from gaining the upper hand when training the machine learning algorithm gains, which was very often introduced, for example, through augmentation or augmentation processes, which in turn can reduce the risk of overfitting.

In addition, the second cost function can in turn be designed to compare the predictions with an average value formed from at least some of the predictions. As a result, resources required for executing the second cost function and thus for training the machine learning algorithm, for example memory and/or processor capacities, can be reduced. In addition, machine learning algorithms are usually trained in batches, i.e. on parts of the training data, so that all examples or training data are usually never evaluated at the same time. By comparing the predictions with the average value, it can be ensured that the predictions do not have to be compared directly with all examples, but only with the average value.

The first cost function and the second cost function can also be combined into a common cost function, with the training unit being designed to train the machine learning algorithm by applying the common cost function. By combining the first cost function and the second cost function into a common cost function, the resources required to train the machine learning algorithm can be reduced and at the same time the time required to train the machine learning algorithm can also be reduced.

Furthermore, the training data can in turn be divided into several parts, wherein the training unit can be designed to successively apply the first cost function and the second cost function to the corresponding part of the training data for each part of the training data. This means that both cost functions can also be evaluated alternately and exclusively.

A further embodiment of the invention also provides a system for training a machine learning algorithm, wherein the system has at least one optical sensor for generating image data and a control device described above for training a machine learning algorithm, and wherein the control device is formed is to train the machine learning algorithm based on image data generated by the at least one optical sensor.

Thus, a system for training a machine learning algorithm is specified, which is based on an improved control device for training a machine learning algorithm with which the influence of false or inaccurate ground truth information can be mitigated. The fact that the training of the machine learning algorithm is based not only on the classic first cost function but also on an additional second cost function, which evaluates the predictions on image material from the same camera with regard to the similarity or symmetry of the predictions, has the advantage that the method is more robust against high noise levels and variances in ground truth information.

With a further embodiment of the invention, a control device for calibrating at least one optical sensor is also specified, wherein the control device is designed to calibrate the at least one optical sensor based on a machine learning algorithm, and wherein the machine learning algorithm is characterized by the above described control device was trained to train a machine learning algorithm.

Thus, a control device for calibrating at least one optical sensor is specified, which is trained on a machine learning algorithm, which is trained by an improved control device for training a machine learning algorithm with which the influence of false or inaccurate ground truth information can be attenuated was based. In particular, the fact that training the machine learning algorithm is based not only on the classic first cost function but also on an additional second cost function, which evaluates the predictions on image material from the same camera with regard to the similarity or symmetry of the predictions, has the advantage that the method is more robust against high Noise levels and variances in ground truth information is.

In summary, the present invention provides a method for training a machine learning algorithm, wherein training the machine learning algorithm includes evaluating prediction symmetry.

The configurations and further developments described can be combined with one another as desired.

Further possible refinements, further developments and implementations of the invention also include combinations of features of the invention described previously or below with regard to the exemplary embodiments that are not explicitly mentioned.

Brief description of the drawings

The accompanying drawings are intended to provide further understanding of embodiments of the invention. They illustrate embodiments and, in connection with the description, serve to explain principles and concepts of the invention.

Other embodiments and many of the advantages mentioned arise with regard to the drawings. The illustrated elements of the drawings are not necessarily shown to scale to one another.

• 1 ein Flussdiagramm eines Verfahrens zum Trainieren eines Algorithmus des maschinellen Lernens gemäß Ausführungsformen der Erfindung; und
• 2 ein schematisches Blockschaltbild eines Systems zum Trainieren eines Algorithmus des maschinellen Lernens gemäß Ausführungsformen der Erfindung.

Show it:

• 1 a flowchart of a method for training a machine learning algorithm according to embodiments of the invention; and
• 2 a schematic block diagram of a system for training a machine learning algorithm according to embodiments of the invention.

In the figures of the drawings, the same reference numerals designate the same or functionally identical elements, parts or components, unless otherwise stated.

1 shows a flowchart of a method for training a machine learning algorithm 1 according to embodiments of the invention.

For example, intelligent or smart camera systems are increasingly being used to monitor highways, buildings, fences or offices. Smart camera systems differ from conventional camera systems in that the image data recorded is evaluated directly by the camera, for example based on a machine learning algorithm, and the results of the image processing are output.

In order to be able to use such smart camera systems, it is important to first calibrate or set them up. In addition to the image data, further information is required for this, for example information about intrinsic calibration, orientation, or recognized objects or boundaries. This information is often generated and entered manually, although the installation time of smart camera systems can be reduced and they can be made insensitive to potential changes in orientation if this process were automated.

Machine learning algorithms trained in deep learning are particularly suitable for the automated estimation of this information, especially since they are able to interpret even complex scene content. However, the disadvantage here is that it is difficult to check the correctness of the corresponding information used for calibration, since there is usually no highly accurate calibration information available. Rather, many parameters must be derived based on a few clues in image data, which then form ground truth information or basic knowledge, i.e. knowledge about a correct classification, although this information can be very uncertain and does not reflect the actual information.

1 shows a method for training a machine learning algorithm 1, which maps input data into output data, the input data being image data, and the method 1 includes a step 2 of providing labeled image data, the labeled image data being from at least one optical sensor, and wherein the labeled image data represents training data for training the machine learning algorithm, and a step 3 of training the machine learning algorithm based on the training data, wherein training the machine learning algorithm involves applying a first Cost function that compares predictions with ground truth information, and wherein training the machine learning algorithm involves applying a second cost function that, for each of the at least one optical sensor, relates a similarity of predictions from the corresponding optical sensor evaluates image data supplied from the same scene.

The fact that the training of the machine learning algorithm is based not only on the classic first cost function but also on an additional second cost function, which evaluates the predictions on image material from the same camera with regard to the similarity or symmetry of the predictions, has the advantage that method 1 is more robust against high Noise levels and variances in ground truth information is.

Overall, an improved method for training a machine learning algorithm 1 is specified, with which the influence of false or inaccurate ground truth information can be weakened.

The machine learning algorithm can be, for example, an artificial neural network.

In addition, the second cost function can be designed to generate similarities in predictions on image data generated by the same optical sensor at different times and / or under different lighting conditions with respect to the same scene or characterizing the same scene.

How 1 shows, the method also has a step 4 of generating additional image data by at least one augmentation method, the training data comprising the labeled image data and the additional image data.

The at least one augmentation method can include, for example, generating new image data by mirroring and/or shifting image data.

This means, for example, that a neural network can be trained with only a few available examples or training data.

According to the embodiments of 1 the second cost function further has a weighting which is designed to prevent very frequently generated training data from being taken into account correspondingly frequently when training the machine learning algorithm.

In addition, the second cost function is designed to compare the predictions with an average value formed from at least some of the predictions.

The average value can be formed, for example, by low-pass filtered averaging of at least some of the predictions.

Furthermore, a separate average value can be formed for each optical sensor and each possible scene.

According to the embodiments of 1 The first cost function and the second cost function are also combined to form a common cost function, whereby training the machine learning algorithm involves applying the common cost function.

In particular, the classic cost function is expanded to include a term regarding the similarity of predictions.

wobei p_i die Ground-Truth-Informationen der i-ten Bilddaten und p'_i die Prädiktion auf den i-ten Bilddaten sind, p die Gesamtheit aller in den Bilddaten enthaltenen Ground-Truth-Informationen und p' die Gesamtheit aller aus den Bilddaten abgeleiteten Prädiktionen sind, L_c die erste Kostenfunktion ist, L_s die zweite Kostenfunktion ist, α und β Gewichtungen sind und λ_ij die Gewichtung, welche ausgebildet ist, zu verhindern, dass sehr häufig erzeugte Trainingsdaten beim Trainieren des Algorithmus des maschinellen Lernens entsprechend häufig berücksichtigt werden, ist. 1_ij ist weiter eine Funktion, welche den Wert 1 aufweist, wenn die Bilddaten i und j beziehungsweise die entsprechenden Beispiele dieselbe Ground-Truth aufweisen beziehungsweise von derselben Kamera stammen, und ansonsten den Wert 0 aufweist.In particular, the common cost function L can be defined as follows: $$L\left(p\text{'},p\right)=\alpha {\displaystyle {\sum }_i{L}_c\left(p{\text{'}}_i,p_i\right)+\beta {\displaystyle \sum {}_i{\displaystyle \sum {}_{j,j>i}{\lambda }_{ij}{1}_{ij}{L}_s\left(p{\text{'}}_i,p{\text{'}}_j\right)}}}$$

where p _i is the ground truth information of the ith image data and p' _i is the prediction on the ith image data, p is the totality of all ground truth information contained in the image data and p' is the totality of all of the image data derived predictions, L _c is the first cost function, L _s is the second cost function, α and β are weights and λ _ij is the weight, which is designed to prevent training data generated very frequently when training the machine learning algorithm correspondingly frequently be taken into account. 1 _ij is also a function that Has a value of 1 if the image data i and j or the corresponding examples have the same ground truth or come from the same camera, and otherwise has the value 0.

The classic cost term is thus expanded to include a component relating to the symmetry of the predictions.

According to the embodiments of 1 The training data is divided into several parts, and for each part of the training data the first cost function and the second cost function, or the classic cost term and the additional cost term, are also applied one after the other to the corresponding part of the training data.

The corresponding algorithm can then be used, for example, to calibrate smart camera systems, with information about the orientation of a smart camera based on the machine learning algorithm, for example about how high the camera is suspended above a level, how far down the camera looks , or a focal length of the camera can be obtained, and the smart camera can then be calibrated based on the information obtained.

Furthermore, the machine learning algorithm can be trained based on appropriately labeled training data, for example also for segmenting image data or for object recognition.

2 shows a schematic block diagram of a system for training a machine learning algorithm 10 according to embodiments of the invention.

How 2 shows, the system 10 has optical sensors 11 for generating image data and a control device for training a machine learning algorithm 12, and the control device 12 is designed to assign the machine learning algorithm based on image data generated by the optical sensors 11 train.

According to the embodiments of 1 The optical sensors 11 are each smart cameras. Furthermore, the optical sensors can also be other sensors which are suitable for generating image data relating to a scene or image data characterizing a scene.

The control device 12 further has a provision unit 13, which is designed to provide labeled image data, the labeled image data coming from at least one of the optical sensors 11, and the labeled image data representing training data for training the machine learning algorithm, and a training unit 14, which is designed to train the algorithm of the machine learning algorithm based on the training data, wherein training the machine learning algorithm comprises applying a first cost function that compares predictions with ground truth information, and wherein the training of the machine learning algorithm involves applying a second cost function which, for each optical sensor 11, evaluates a similarity of predictions on image data supplied by the corresponding optical sensor with respect to the same scene.

The provision unit can be, for example, a receiver which is designed to receive image data generated by the optical sensors. The training unit can also be implemented, for example, based on code stored in a memory and executable by a processor.

How 2 shows, the control device 10 also has an augmentation unit 15, which is designed to generate additional image data by at least one augmentation method, the training data comprising the labeled image data and the additional image data.

The augmentation unit can in turn be implemented, for example, based on code stored in a memory and executable by a processor.

According to the embodiments of 2 The second cost function also has a weighting which is designed to prevent very frequently generated training data from being taken into account correspondingly frequently when training the machine learning algorithm.

In addition, the second cost function is again designed to compare the predictions with an average value formed from at least some of the predictions.

According to the embodiments of 2 the first cost function and the second cost function are in turn combined to form a common cost function, with the training unit 14 being designed to train the machine learning algorithm by applying the common cost function.

In addition, the training data is divided into several parts, with the training unit 14 shown being designed to successively apply the first cost function and the second cost function to the corresponding part of the training data for each part of the training data.

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 6985780 B2 [0006]

Claims (15)

Method for training a machine learning algorithm which maps input data into output data, the input data being image data, and the method (1) comprising the following steps: - Providing labeled image data, wherein the labeled image data comes from at least one optical sensor, and wherein the labeled image data represents training data for training the machine learning algorithm (2); and - Training the machine learning algorithm based on the training data, wherein training the machine learning algorithm comprises applying a first cost function that compares predictions with ground truth information, and wherein training the machine learning algorithm includes Applying a second cost function, which, for each of the at least one optical sensor, evaluates a similarity of predictions on image data generated by the corresponding optical sensor with respect to the same scene (3). Procedure according to Claim 1 , wherein the method (1) further comprises generating additional image data by at least one augmentation method (4), and wherein the training data includes the labeled image data and the additional image data. Procedure according to Claim 2 , wherein the second cost function has a weighting which is designed to prevent very frequently generated training data from being taken into account correspondingly frequently when training the machine learning algorithm. Procedure according to one of the Claims 1 until 3 , wherein the second cost function is designed to compare the predictions with an average value formed from at least some of the predictions. Procedure according to one of the Claims 1 until 4 , wherein the first cost function and the second cost function are combined into a common cost function, and wherein training the machine learning algorithm (3) comprises applying the common cost function. Procedure according to one of the Claims 1 until 5 , wherein the training data is divided into a plurality of parts, and wherein the step of training the machine learning algorithm (3) for each part of the training data comprises sequentially applying the first cost function and the second cost function to the corresponding part of the training data. A method for calibrating at least one optical sensor, the method comprising calibrating the at least one optical sensor based on a machine learning algorithm, and wherein the machine learning algorithm is characterized by a method for training a machine learning algorithm according to one of Claims 1 until 6 was trained. Control device for training a machine learning algorithm, which maps input data into output data, the input data being image data, and wherein the control device (12) has a provision unit (13) which is designed to provide labeled image data, the labeled image data come from at least one optical sensor, and wherein the labeled image data represents training data for training the machine learning algorithm, and a training unit (14) which is designed to train the machine learning algorithm based on the training data, wherein the training the machine learning algorithm includes applying a first cost function that compares predictions with ground truth information, and wherein training the machine learning algorithm includes applying a second cost function that, for each of the at least one optical sensor, has a symmetry of predictions based on image data supplied by the corresponding optical sensor regarding the same scene. control unit Claim 8 , wherein the control device (12) further has an augmentation unit (15), which is designed to generate additional image data by an augmentation method, and wherein the training data includes the labeled image data and the additional image data. control unit Claim 8 or 9 , wherein the second cost function has a weighting which is designed to prevent very frequently generated training data from being taken into account correspondingly frequently when training the machine learning algorithm. Control unit according to one of the Claims 8 until 10 , wherein the second cost function is designed to compare the predictions with an average value formed from at least some of the predictions. Control unit according to one of the Claims 8 until 11 , wherein the first cost function and the second cost function are combined into a common cost function, and wherein the training unit (14) is designed to train the machine learning algorithm by applying the common cost function. Control unit according to one of the Claims 8 until 12 , wherein the training data is divided into several parts, and wherein the training unit (14) is designed to successively apply the first cost function and the second cost function to the corresponding part of the training data for each part of the training data. System for training a machine learning algorithm, wherein the system (10) has at least one optical sensor (11) for generating image data and a control device for training a machine learning algorithm (12) according to one of Claims 8 until 13 and wherein the control device (12) is designed to train the machine learning algorithm based on image data generated by the at least one optical sensor (11). Control device for calibrating at least one optical sensor, wherein the control device is designed to calibrate the at least one optical sensor based on a machine learning algorithm, and wherein the machine learning algorithm is controlled by a control device for training a machine learning algorithm according to one of Claims 8 until 13 was trained.

Images