DE102020202973A1 - Method and apparatus for processing images - Google Patents

Abstract

A method, in particular a computer-implemented method, for processing, in particular digital, images, comprising the following steps: providing a first image and a second image, transforming the first image into a first descriptor image and the second image into a second descriptor image, determining a distance image based on on a comparison of the first descriptor image with the second descriptor image, forming a detection image based on the distance image.

Description

State of the art

The disclosure relates to a method, in particular a computer-implemented one, for processing, in particular digital, images.

The disclosure also relates to a device for processing, in particular digital, images.

Disclosure of the invention

Preferred embodiments relate to a method, in particular a computer-implemented method, for processing, in particular digital, images, comprising the following steps: providing a first image and a second image, transforming the first image into a first descriptor image and the second image into a second Descriptor image, determining a distance image based on a comparison of the first descriptor image with the second descriptor image, and, optionally, forming a detection image based on the distance image. This enables, for example, a particularly efficient detection of changes in relation to the images, e.g. detection of a movement of at least one object depicted on at least one of the images.

In the context of the present invention, an image can be understood to be an encoded data record which describes or represents a representation of the image or an image representation. Using a decoding rule, a control rule for a display device can be determined by means of a computer from the data record representing the image in such a way that the display device displays the coded image using the control rule. The display device can be, for example, a display unit or a projector unit. The display device can, for example, be arranged on or in a driver's cab of a vehicle.

In further preferred embodiments, a transformation is used for the transformation into the descriptor images, which transforms the environment of a picture element ("pixel") of the first or second image in the respective image into a descriptor that describes this environment, preferably in a compact manner, that is to say with a few bits, in particular with fewer bits than corresponds to the information content of the area around the pixel. In further preferred embodiments, the value of the descriptor is referred to as the signature. In further preferred embodiments, the signature has a fixed length, in particular word length, of e.g. B. 8 bit to e.g. 32 bit, but in further preferred embodiments it can also be longer than 32 bit or shorter than 8 bit.

In further preferred embodiments, the transforming includes carrying out the transformation for a respective environment of a plurality of pixels, for example each pixel in the (first or second) image, so that as a result a respective (first or second) "image of descriptors" , i.e. the descriptor image or images already mentioned, is created. In further preferred embodiments, a descriptor image can also be understood as a plurality of descriptor values or signatures, which are preferably organized in a matrix-like arrangement of rows and columns, corresponding to the position of the pixels evaluated for their formation or the respective surroundings of a pixel under consideration. In this respect, a descriptor image is e.g. comparable to the first or second image, but instead of the regular image information (such as brightness or intensity values of one or more gray scale or color channels), each image element of the descriptor image is assigned the respective signature.

In further preferred embodiments, pixels at the image edge of the first and / or second image can be disregarded for the transformation to the descriptor images, because there, for example, the frame for the transformation protrudes beyond the image, and the “pixel values” are therefore undefined. In further preferred embodiments, pixels at the image edge of the first and / or second image can be taken into account for the transformation to the descriptor images, with any missing or undefined pixel values at the edge being supplemented according to a prescribable rule, e.g. by copying the pixel values of existing pixels and / or setting the previously undefined values to a predefinable value.

In further preferred embodiments, at least one of the following methods can be used for the transformation: SIFT (scale-invariant feature transformation), SURF (Speeded Up Robust Features, for example according to Bay H., Tuytelaars T., Van Gool L. (2006) SURF: Speeded Up Robust Features . In: Leonardis A., Bischof H., Pinz A. (eds) Computer Vision - ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3951. Springer, Berlin, Heidelberg, https://doi.org/10.1007/11744023_32 ), ORB ( E. Rublee, V. Rabaud, K. Konolige and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," 2011 International Conference on Computer Vision, Barcelona, 2011, pp. 2564-2571; DOI: 10.1109 / ICCV.2011.6126544 ), BRISK ( Leutenegger, Stefan & Chli, Margarita & Siegwart, Roland. (2011). BRISK: Binary Robust invariant scalable keypoints. Proceedings of the IEEE International Conference on Computer Vision. 2548-2555. 10.1109 / ICCV.2011.6126542.), LETTER ( Binary Robust Independent Elementary Features, DOI: 10.1007 / 978-3-642-15561-1_56), DAISY ( E. Tola, V. Lepetit and P. Fua, "DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 815-830, May 2010. ), LATCH (https://arxiv.org/pdf/1501.03719.pdf).

In further preferred embodiments, for the descriptor selected to form the descriptor images, a distance measure is available or can be specified which, for example, allows a difference between two descriptor values to be determined or assessed.

In further preferred embodiments, a similarity measure can also be used as an alternative to the distance measure, it being possible in particular for both measures to be convertible into one another. Therefore, in the following, only the distance measure is considered as an example and without limiting the general validity, with all steps and embodiments described by way of example also when using a similarity measure - instead of a distance measure - to evaluate a difference between the descriptor images, e.g. to characterize a result of the comparison of the first descriptor image with the second descriptor image.

In further preferred embodiments it is provided that a descriptor is used for the transformation, for which a distance measure and / or a similarity measure is definable and / or defined, in particular the comparison of the first descriptor image with the second descriptor image based on the distance measure and / or the similarity measure is carried out.

In further preferred embodiments, it is provided that the Hamming distance or a distance measure based on the Hamming distance is used as the distance measure for the comparison, the Hamming distance in particular being compared with a predeterminable threshold value and, based on the comparison, a particularly binary one , Comparison value is determined.

In further preferred embodiments, using the Hamming distance as a distance measure, a bit-wise comparison of two descriptor values is carried out with one another, the Hamming distance indicating the number of different bits of the descriptor values compared with one another. If the descriptor has a word length of N bits, the Hamming distance D _{H has} a range of values D _H ∈ {0,1,2, ..., N}. The value D _H = 0 means that the two descriptor values compared with one another are identical, while with D _H = N they are maximally different. If, for example, one compares two randomly selected descriptor values with one another and all representable descriptor values 0, 1, ..., 2 ^{N-1 occur with} approximately the same probability, a Hamming distance of approximately N / 2 is to be expected, for example, as a statistical mean. For a change detection in accordance with further preferred embodiments, this means that distance values close to 0 are to be expected in the case of essentially matching image areas - and for image areas that do not match, e.g. B. due to moving objects, distance values significantly greater than 0, and then z. B. be around N / 2.

According to further preferred embodiments, the distance measure can optionally be further simplified, for example by binarization. For example, the Hamming distance for this can be compared with a threshold L and a binary distance D _B can be obtained according to ${\mathrm{D.}}_{\mathrm{B.}}=\{\begin{array}{c}0falls{\mathrm{D.}}_H<1\\ 1sOnst\end{array}.$

In further preferred embodiments, as an alternative or in addition to an at least temporary storage of descriptor values in the first and / or second descriptor image, the descriptor values can also be determined if necessary, e.g. "on the fly". In other words, in further preferred embodiments, the transformation of the first image into the first descriptor image and / or the transformation of the second image into the second descriptor image can also be dynamic, that is to say, for example, as required and / or in real time, in particular, for example, also in direct temporal connection with the Determining the distance image, are carried out.

In further preferred embodiments, the transformation of the first image into the first descriptor image and / or the transformation of the second image into the second descriptor image can also be parallelized at least temporarily, e.g. B. if several cores are available to carry out the respective transformation (s).

In further preferred embodiments, it is also conceivable to carry out the transformation to form the respective (first and / or second) descriptor image simultaneously or at least partially overlapping in time on a first part of the (first and / or second) image. In other words, in further preferred embodiments it is also conceivable to parallelize the transformation of the first image into the first descriptor image, with, for example, a first part of the first image being transformed into a first part of the first descriptor image, and at the same time, for example, a second part of the first image is transformed into a second part of the first descriptor image.

In further preferred embodiments, the first descriptor image belonging to a first point in time, for example the current point in time, is compared with a second descriptor image which belongs to a second, in particular earlier, point in time.

In further preferred embodiments, both the transformation to the descriptor images and the determination of the distance image can be carried out, for example, only on a predeterminable sub-area of the first and second images or the descriptor images that can be derived therefrom. In further preferred embodiments, partial areas can also be selected dynamically, i.e. at the runtime of the method or a device executing the method, e.g. based on a current content of the images and / or previously recognized changes and / or objects.

In further preferred embodiments, the comparison of the first descriptor image with the second descriptor image, i.e. the formation of the distance measure, takes place pixel by pixel, so a picture element of the first descriptor image is compared with a corresponding picture element of the second descriptor image and a corresponding value for the distance measure ("Distance value") received. In other words, in further preferred embodiments, a descriptor value at a specific coordinate of the first descriptor image is compared with the descriptor value at the corresponding (same) coordinate of the second descriptor image. In further preferred embodiments, the distance value determined in this way is entered in a distance image or in the distance image at the corresponding coordinate. It can be seen that the distance image has the same size (number of pixels, e.g. characterizable by width and height) as the descriptor images to be compared. In further preferred embodiments, the distance image (i.e. each picture element) can have values in the range 0 to N, for example, which characterize the comparison result between the first descriptor image and the second descriptor image.

In further preferred embodiments it is provided that the method further comprises: filtering the distance image, as a result of which a filtered distance image is obtained, wherein in particular the formation of the detection image takes place based on the filtered distance image.

In further preferred embodiments, the filtering is carried out in such a way that the distance image is converted into a filtered image which shows changes in a compact form, which is useful, for example, for a functional interface according to further preferred embodiments.

In further preferred embodiments, the detection image has one or more contiguous regions, which are also referred to as “blobs” according to further preferred embodiments, for those image areas in which, in particular significant, changes between the (first and second) images or the herewith corresponding descriptor images are available.

In further preferred embodiments, as an alternative or in addition to an at least temporary complete or partial storage of the distance image and / or the detection image, an (at least partial) determination of the distance image and / or the detection image can also take place if necessary, e.g. "on the fly". In other words, in further preferred embodiments it is conceivable to (also) not save the distance image and / or the detection image at least completely, but to process it or them, e.g., in particular immediately, e.g. to save memory or to obtain intermediate results (e.g. parts of the Distance image and / or detection image) to be able to provide faster.

In further preferred embodiments, the detection image can be converted into another form in an optional further or alternative step, e.g. B. to be able to transmit it more efficiently via an interface. For this purpose, in further preferred embodiments, for example, the contours of at least one blob are approximated, e.g. described as polygons.

Alternatively or in addition, in further preferred embodiments the detection image can also be compressed, e.g. with a run length coding or another entropy coding, e.g. with a common coding for segment images.

In further preferred embodiments it is provided that the filtering includes the application of a majority filter and / or a threshold value filter.

In further preferred embodiments it is provided that the method further comprises: further processing of the detection image, in particular formation of output information, based on at least one of the following elements: a) detection image, b) first image, c) second image.

In further preferred embodiments it is provided that the output information has at least one of the following elements: a) acoustic signal, b) haptic signal, c) optical signal, d) image, in particular digital image, with at least one graphically highlighted Image area, in particular an image area graphically highlighted based on the detection image. As a result, a change in the first and second images detected by means of the method can be brought to the attention of a user, for example a driver of a forklift, efficiently. In this way, the driver can be informed reliably and easily perceptibly, for example, about changes in the surroundings of his vehicle, whereby, for example, accidents with people approaching the vehicle can be avoided.

In further preferred embodiments it is provided that the method further comprises: assigning an evaluation measure to at least one descriptor of the first descriptor image and / or the second descriptor image, and, optionally, taking the evaluation measure into account when determining the distance image, wherein in particular the evaluation measure is noise or characterizes a signal-to-noise ratio, in particular a region of the first image and / or of the second image associated with the respective descriptor.

In further preferred embodiments, the noise can thus also be taken into account when forming the descriptors or the descriptor images, for example in accordance with the method shown in FIG DE 10 2017 212 339 described procedure.

In further preferred embodiments, at least some, in particular each, formed descriptor (s) can be given an evaluation that is dependent on the strength of the noise or the signal-to-noise ratio (SNR): in some preferred embodiments, for example, in the form of a binary one Additional information (for example coded as an additional bit) which, in further preferred embodiments, can also be interpreted as the suitability of the descriptor for further processing.

In further preferred embodiments, the additional information allows, for example, to identify suitable descriptors less well (e.g. for subsequent further processing) due to the noise, e.g. B. as "unsuitable", e.g. to treat them differently in the event of a change detection, than descriptors that are better suited for further processing, e.g. due to lower noise or higher SNR.

In further preferred embodiments, this additional information can be taken into account when determining the distance image (“distance calculation”), for example according to the following rule: If at least one of the descriptors to be compared is marked as “unsuitable”, the distance is not calculated according to the usual rule, but rather one Another rule is applied, according to which, for example, the actual distance is replaced by a specifiable, in particular fixed, value. In further preferred embodiments, in the example of the Hamming distance as a distance measure, the value can then be set to "0", for example, which means, for example, that image regions dominated by noise are treated as if they were motionless.

In further preferred embodiments, however, the Hamming distance can also be set to a different value, e.g. B. "1" or "2", so that, for example, areas excluded due to noise are not treated differently (e.g. better placed) than non-excluded, unmoved areas. In further preferred embodiments, this consideration can be particularly relevant due to an optional subsequent filtering of the distance image, in which, for example, a sliding window for the filtering can contain both excluded and non-excluded pixels at the same time.

• Stufe 0: Aufgrund von Rauschen sollte der Deskriptor nicht verwendet werden.
• Stufe 1: Aufgrund von Rauschen ist der Deskriptor zwar für „Anwendung A“ geeignet, nicht jedoch für „Anwendung B“.
• Stufe 2: Der Deskriptor ist für die „Anwendungen A“ und „B“ geeignet, das Rauschen ist somit z.B. nicht signifikant.

In further preferred embodiments, the evaluation of the noise is not passed on as binary information, but rather more than two levels are provided, for example three or four (or more) levels. If, for example, three stages are provided according to further preferred embodiments, these could have the following meanings in further preferred embodiments:

• Level 0: The descriptor should not be used due to noise.
• Level 1: Due to noise, the descriptor is suitable for "Application A", but not for "Application B".
• Level 2: The descriptor is suitable for "Applications A" and "B", so the noise is, for example, not significant.

In further preferred embodiments, "Applications A" and "B" can stand for two applications from, for example: optical flow, change detection, correspondence formation, disparity estimation, tracking, etc.

In the case of further preferred embodiments, further configurations are also conceivable. For example, the local signal-to-noise ratio of a viewed image region (e.g. associated with a descriptor, i.e. taken into account in the formation of the descriptor) can be attached to the respective descriptor as additional information (e.g. in the sense of a concatenation), e.g. B. as a number. In further preferred embodiments, a distance calculation can evaluate and pass on this additional information, for example as a minimum or maximum or mean value of the two numbers of the descriptors involved in the distance calculation. This additional information can be used for further Preferred embodiments, for example, can also be further taken into account in a subsequent optional filtering step and possibly also passed on, for example as a confidence of a decision made about the presence of an object in the first and / or second image.

In further preferred embodiments it is provided that the method further comprises: at least temporarily storing the first descriptor image and / or the second descriptor image, e.g. for subsequent use. For example, when two distance images are determined consecutively, a participating descriptor image can be used twice for the distance calculation, namely once in the sense of the first descriptor image and a second time in the sense of the second descriptor image.

In further preferred embodiments it is provided that the method further comprises: Compensating for a movement associated with the first image and / or the second image, in particular an intrinsic movement of a camera providing the first image and / or the second image, for at least one area, in particular Surface.

In further preferred embodiments it is provided that the first image and the second image are each part of the same video data stream of at least one camera.

In further preferred embodiments it is provided that at least one further image is present in the video data stream between the first image and the second image. In other words, in further preferred embodiments, a time interval between the first image and the second image can be changed, for example, by not using directly consecutive images, e.g. of the video data stream, as the first and second image, but rather that e.g. one or more images, e.g. of the Video data stream between the first image and the second image can be omitted.

In further preferred embodiments it is provided that the method (in particular the steps of providing and / or transforming into the descriptor images and / or determining the distance image and, optionally, determining the detection image) is applied to a plurality of image pairs each having a first and a second image is applied, the first image of a first image pair being a first image of the video data stream, the second image of the first image pair being an n-th image, with n> 2 (e.g. n = 5), of the video data stream, the first image being one second image pair is a second image of the video data stream, the second image of the second image pair being an (n + 1) th image of the video data stream, etc.

As a result, despite “omitting” images (with regard to the respective pair of images being viewed), a quick response can be made and, in particular, detections (eg in the form of the detection image) can also be output in quick succession, which is important for a graphical display for a user, for example can be (so that it does not jerk).

In further preferred embodiments, it is provided that the method is carried out, in particular in parallel, on different pairs of images (first image, second image) of the same or the same video data stream, with a respective first image and a respective second image each having a different time interval have to each other.

Further preferred embodiments relate to a device for processing, in particular digital, images, the device being designed to carry out the method according to the embodiments.

In further preferred embodiments it is provided that the device has: a computing device (“computer”) having at least one computing core, a memory device assigned to the computing device for at least temporary storage of at least one of the following elements: a) data, b) computer program, in particular for execution of the method according to the embodiments.

In further preferred embodiments, the data DAT can at least temporarily and / or partially contain the at least one video data stream and / or the first image and / or the second image and / or data that can be derived therefrom, e.g. the first descriptor image and / or the second descriptor image or the Have distance image or the detection image or at least parts thereof.

In further preferred embodiments, the memory device has a volatile memory (e.g. working memory (RAM)) and / or a non-volatile memory (e.g. flash EEPROM).

In further preferred embodiments, the computing device can also have at least one of the following elements: microprocessor (µP), microcontroller (µC), application-specific integrated circuit (ASIC), system on chip (SoC), programmable logic module (e.g. FPGA, field programmable gate array) , Hardware circuit, graphics processing unit (GPU), or any combination thereof. Further Preferred embodiments relate to a computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method according to the embodiments.

Further preferred embodiments relate to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to execute the method according to the embodiments.

Further preferred embodiments relate to a data carrier signal that characterizes and / or transmits the computer program according to the embodiments. The data carrier signal can be received, for example, via an optional data interface of the device.
Further preferred embodiments relate to a use of the method according to the embodiments and / or the device according to the embodiments and / or the computer-readable storage medium according to the embodiments and / or the computer program according to the embodiments and / or the data carrier signal according to the embodiments for at least one of the the following elements: a) Detection a1) an environment of a system, in particular a vehicle, in particular an industrial truck such as forklift and / or forklift, and / or a2) a scene, b) determining changes in relation to the first image and the second image, in particular robust change detection with regard to the first image and the second image, c) determining obstacles, in particular moving objects, d) avoiding accidents, in particular in the area of a system or in the environment, in particular at comparatively low speeds of the system, in particular little equ I a walking pace of a person, further in particular during a transition between a standstill and a movement or vice versa, e) application both when the system is stationary and when the system is moving, f) visualization of changes in relation to the first image and the second image, In particular, displaying information or additional information characterizing the changes or highlighting at least one area of the first image and / or the second image that is associated with the changes, in particular in real time (“real time visualization”).

Further features, possible applications and advantages of the invention emerge from the following description of exemplary embodiments of the invention, which are shown in the figures of the drawing. All of the features described or shown form the subject matter of the invention individually or in any combination, regardless of how they are summarized in the claims or their reference and regardless of their formulation or representation in the description or in the drawing.

• 1 schematisch ein vereinfachtes Blockdiagramm gemäß bevorzugten Ausführungsformen,
• 2A schematisch ein vereinfachtes Flussdiagramm von Verfahren gemäß weiteren bevorzugten Ausführungsformen,
• 2B, 2C, 2D, 2E, 2F jeweils schematisch ein vereinfachtes Flussdiagramm von Verfahren gemäß weiteren bevorzugten Ausführungsformen,
• 3A schematisch ein beispielhaftes Bild gemäß weiteren bevorzugten Ausführungsformen,
• 3B schematisch ein mit dem Bild gemäß 3A assoziiertes Deskriptorbild gemäß weiteren bevorzugten Ausführungsformen,
• 4A schematisch ein beispielhaftes Distanzbild gemäß weiteren bevorzugten Ausführungsformen,
• 4B schematisch ein mit dem Distanzbild gemäß 4A assoziiertes Detektionsbild gemäß weiteren bevorzugten Ausführungsformen,
• 5 schematisch eine Anwendung eines Majoritätsfilters gemäß weiteren bevorzugten Ausführungsformen,
• 6 schematisch beispielhafte Gewichtungen für eine Filterung gemäß weiteren bevorzugten Ausführungsformen,
• 7 schematisch ein beispielhaftes Bild gemäß weiteren bevorzugten Ausführungsformen,
• 8, 9, 10 jeweils schematisch ein vereinfachtes Blockdiagramm gemäß weiteren bevorzugten Ausführungsformen,
• 11 schematisch ein vereinfachtes Blockdiagramm einer Vorrichtung gemäß weiteren bevorzugten Ausführungsformen,
• 12 schematisch Aspekte einer Verwendung gemäß weiteren bevorzugten Ausführungsformen,
• 13 schematisch beispielhafte Bildfolgen gemäß weiteren bevorzugten Ausführungsformen,
• 14A, 14B, 14C, 14D, 14E, 14F jeweils schematisch ein Bild gemäß weiteren bevorzugten Ausführungsformen,
• 15A, 15B, 15C, 15D jeweils schematisch ein Bild gemäß weiteren bevorzugten Ausführungsformen,
• 16A, 16B, 16C, 16D jeweils schematisch ein Bild gemäß weiteren bevorzugten Ausführungsformen, und
• 17A, 17B, 17C, 17D jeweils schematisch ein Bild gemäß weiteren bevorzugten Ausführungsformen.

In the drawing shows:

• 1 schematically a simplified block diagram according to preferred embodiments,
• 2A schematically a simplified flow diagram of methods according to further preferred embodiments,
• 2 B , 2C , 2D , 2E , 2F each schematically a simplified flowchart of methods according to further preferred embodiments,
• 3A schematically an exemplary image according to further preferred embodiments,
• 3B schematically a with the picture according to 3A associated descriptor image according to further preferred embodiments,
• 4A schematically an exemplary distance image according to further preferred embodiments,
• 4B schematically a with the distance image according to 4A associated detection image according to further preferred embodiments,
• 5 schematically an application of a majority filter according to further preferred embodiments,
• 6th schematically exemplary weightings for filtering according to further preferred embodiments,
• 7th schematically an exemplary image according to further preferred embodiments,
• 8th , 9 , 10 each schematically a simplified block diagram according to further preferred embodiments,
• 11 schematically a simplified block diagram of a device according to further preferred embodiments,
• 12th schematically aspects of a use according to further preferred embodiments,
• 13th schematically exemplary image sequences according to further preferred embodiments,
• 14A , 14B , 14C , 14D , 14E , 14F each schematically an image according to further preferred embodiments,
• 15A , 15B , 15C , 15D each schematically an image according to further preferred embodiments,
• 16A , 16B , 16C , 16D each schematically an image according to further preferred embodiments, and
• 17A , 17B , 17C , 17D each schematically an image according to further preferred embodiments.

1 shows schematically a simplified block diagram of a system 10 for use with a method of processing images in accordance with preferred embodiments. The system 10 is designed, for example, as a vehicle, in particular an industrial truck (eg forklift and / or forklift) and has at least one camera 12th on that multiple images B1 , B2 , especially in the form of a video data stream VDS or can be derived from at least one video data stream VDS , provides. Optionally, the system 10 also at least one more camera 12 ' have, in turn, z. B. can provide one or more images or a corresponding video data stream (not shown). The system 10 can itself eg in an environment U , for example on a reference surface RF such as a floor surface (e.g. a manufacturing facility). Furthermore, one or more objects OBJ which in particular also obstacles H for the system 10 can represent, at least temporarily in the area U to be available.

In further preferred embodiments, the system provides 10 does not represent a vehicle or has the system 10 no vehicle on, but e.g. the camera 12th as described above, for example the video data stream VDS or the pictures B1 , B2 delivers or one the camera 12th containing stationary facility. For example, the camera 12th intended to be a scene SZ in the neighborhood U to be observed, e.g. used as a surveillance camera.

Further preferred embodiments relate to a method, in particular a computer-implemented method, cf. 2A , for processing, in particular digital, images B1 , B2 ( 1 ), comprising the following steps: Deploy 100 of a first picture B1 and a second image B2 (especially in each case as a digital image), transform 102 of the first picture B1 into a first descriptor image DB1 and the second image B2 into a second descriptor image DB2 , Determine 104 a distance image DISTB based on a comparison of the first descriptor image DB1 with the second descriptor image DB1 , and, optionally, make up 106 of a detection image DETB based on the distance image DISTB . This enables, for example, a particularly efficient detection of changes with regard to the images B1 , B2 possible, for example a detection of a movement of at least one on at least one of the images B1 , B2 pictured object. In particular, the distance image characterizes DISTB or the one in the distance image DISTB information contained at least partially changes to the images B1 , B2 or the descriptor images that can be derived from them DB1 , DB2 to each other.

In further preferred embodiments, the distance image DISTB as a detection image DETB can be used, in particular the optional step 106 not applicable. In further preferred embodiments, the detection image DETB based on the distance image DISTB be formed, for example in the optional step 106 can be done.

the 14A until 14F show, by way of example, different detection images described in more detail below, such as those according to preferred embodiments, for example based on the exemplary sequence according to FIG 2A , can be obtained.

In further preferred embodiments, the transforming 102 in the descriptor images DB1 , DB2 at least one transformation is used that defines the surroundings of a picture element (“pixel”) of the first or second image B1 , B2 in the respective image is converted into a descriptor which describes this environment, preferably in a compact manner, that is to say with a few bits, in particular with fewer bits than the information content of the environment of the pixel in the image B1 , B2 itself corresponds. In further preferred embodiments, the value of the descriptor is referred to as the signature. In further preferred embodiments, the signature has a fixed length, in particular word length, of e.g. B. 8 bits to, for example, 32 bits, but can also be longer than 32 bits or shorter than 8 bits in further preferred embodiments.

In further preferred embodiments, the transforming comprises 102 performing the transformation for a respective neighborhood on a plurality of pixels, for example each pixel in the first image B1 or in the second picture B2 so that the result is a respective first or second “image of descriptors”, that is, the descriptor image or images already mentioned DB1 , DB2 , arises. In further preferred embodiments, a descriptor image DB1 , DB2 can also be understood as a multiplicity of descriptor values or signatures, which are preferably organized in a matrix-like arrangement of rows and columns, corresponding to the position of the pixels evaluated for their formation or the respective surroundings of a pixel under consideration. To that extent is a descriptor image DB1 , DB2 eg with the first or second picture B1 , B2 comparable, but instead of the regular image information (such as brightness or intensity values of one or more gray scale or color channels) each The respective signature is assigned to the image element of the descriptor image, and consequently information about the surroundings of the original image B1 , B2 which were included in the formation of the descriptor or the signature.

In this context shows 3A an exemplary image or initial image B1a , and 3A shows one of it by means of the transformation 102 ( 2A ) derivable associated descriptor image DB1a . 3A, B exemplifies how the environment U1 around the base of the delineator post of the original image B1a into a descriptor D. is changed, see the arrow A1 . The associated descriptor image DB1a ( 3B ), which is no longer like the original image B1a looks (and is usually not intended to be looked at), has information about the individual pixels of the original image B1a and their respective environment (“context”). In the descriptor image DB1a Descriptor values are stored pixel by pixel, which in the present case correspond, for example, to intensity values (black / white or gray levels). The descriptor values, eg the designated descriptor D. , describe a respective environment U of the original image B1a at the appropriate place in (compared to the environment U of the original image B1a ) more compact form.

Instead of the descriptor values as exemplified in 3B shown in the descriptor image DB1a To save them, in further preferred embodiments they can also be formed “on the fly”, that is to say, for example, only when required.

In further preferred embodiments, pixels at the image edge of the first and / or second image B1 , B2 for the transformation 102 ( 2A ) to the descriptor images DB1 , DB2 remain unconsidered because there z. B. the framework for the transformation 102 about the picture B1 , B2 protrudes, the "pixel values" are therefore undefined.

In further preferred embodiments, pixels at the image edge of the first and / or second image B1 , B2 for the transformation 102 to the descriptor images DB1 , DB2 are taken into account, with missing or undefined pixel values at the edge being supplemented according to a prescribable rule, for example by copying the pixel values (eg neighboring) existing pixels and / or setting the previously undefined values to a prescribable value or values.

In further preferred embodiments, for transforming 102 at least one of the methods listed below or parts thereof or combinations thereof can be used: SIFT (scale-invariant feature transformation, see e.g. U.S. 6,711,293 ), SURF ( Speeded Up Robust Features, eg according to Bay H., Tuytelaars T., Van Gool L. (2006) SURF: Speeded Up Robust Features . In: Leonardis A., Bischof H., Pinz A. (eds) Computer Vision - ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3951. Springer, Berlin, Heidelberg, https://doi.org/10.1007/11744023_32 ), ORB ( E. Rublee, V. Rabaud, K. Konolige and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," 2011 International Conference on Computer Vision, Barcelona, 2011, pp. 2564-2571; DOI: 10.1109 / ICCV.2011.6126544 ), BRISK ( Leutenegger, Stefan & Chli, Margarita & Siegwart, Roland. (2011). BRISK: Binary Robust invariant scalable keypoints. Proceedings of the IEEE International Conference on Computer Vision. 2548-2555. 10.1109 / ICCV.2011.6126542. ), LETTER ( Binary Robust Independent Elementary Features, DOI: 10.1007 / 978-3-642-15561-1_56 ), DAISY ( E. Tola, V. Lepetit and P. Fua, "DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 815-830, May 2010. ), LATCH (https://arxiv.org/pdf/1501.03719.pdf). In further preferred embodiments, for transforming 102 another method not mentioned above can also be used.

In further preferred embodiments, is for the formation of the descriptor images DB1 , DB2 ( 2A ) the selected descriptor, a distance measure is present or can be specified which, for example, allows a difference between two descriptor values to be determined or evaluated.

In further preferred embodiments, a similarity measure can also be used as an alternative to the distance measure, it being possible in particular for both measures to be convertible into one another. Therefore, in the following, only the distance measure is considered by way of example and without restricting the general validity, with all steps and embodiments described by way of example correspondingly also when using a similarity measure - instead of a distance measure - for evaluating a difference in the descriptor image DB1 , DB2 , for example to characterize a result of the comparison 104 of the first descriptor image DB1 with the second descriptor image DB2 , are usable.

In further preferred embodiments it is provided that a descriptor is used for the transformation, for which a distance measure and / or a similarity measure can be defined and / or defined, with the comparison in particular 104 of the first descriptor image DB1 with the second descriptor image DB2 is carried out based on the distance measure and / or the similarity measure.

In further preferred embodiments it is provided that as a distance measure for the comparison 104 the hamming distance or on The distance measure based on the Hamming distance is used, the Hamming distance in particular being compared with a predeterminable threshold value and a, in particular binary, comparison value being determined on the basis of the comparison.

In further preferred embodiments, using the Hamming distance as a distance measure, a bit-wise comparison of two descriptor values is carried out with one another, the Hamming distance indicating the number of different bits of the descriptor values compared with one another. If the descriptor has a word length of N bits, the Hamming distance D _{H has} a range of values D _H ∈ {0,1,2, ..., N}. The value D _H = 0 means that the two descriptor values compared with one another are identical, while with D _H = N they are maximally different. If, for example, one compares two randomly selected descriptor values with one another and all representable descriptor values 0, 1, ..., 2 ^{N-1 occur with} approximately the same probability, a Hamming distance of approximately N / 2 is to be expected, for example, as a statistical mean. For a change detection in accordance with further preferred embodiments, this means that distance values close to 0 are to be expected in the case of essentially matching image areas - and for image areas that do not match, e.g. B. due to moving objects, distance values significantly greater than 0, and then z. B. be around N / 2.

According to further preferred embodiments, the distance measure can optionally be further simplified, for example by binarization. For example, the Hamming distance for this can be compared with a threshold L and a binary distance D _B can be obtained according to ${\mathrm{D.}}_{\mathrm{B.}}=\{\begin{array}{c}0falls{\mathrm{D.}}_H<\mathrm{L.}\\ 1sOnst\end{array}.$

4A shows an example of a distance image DISTB1 according to further preferred embodiments, which has been obtained, for example, using the Hamming distance with subsequent binarization.

the 15A until 15D show exemplary images from the camera 12th ( 1 ), in which an industrial truck FFF and a person (not designated) have been successfully detected according to further preferred embodiments, and in which corresponding areas are graphically highlighted according to further preferred embodiments, for example a change in the images B1 , B2 has been recognized. Also show 15A until 15D an effect of a variation of the above-described threshold L in the optional binarization of the distance image DISTB ( 2A ) to a detection image that can be derived from it DETB , see the extent of the blobs or highlighting H1 , H2 , H3 (Area of the person), H4 (Area of the FFF forklift truck). In other words, the images correspond to 15A , 15B , 15C , 15D in each case a detection image that can be determined according to further preferred embodiments DETB which is essentially eg on the first picture B1 and / or the second image B2 based, and in which the mentioned emphasis H1 , H2 , H3 , H4 based on a binarized according to the threshold L distance image are included. To determine the detection image DETBa according to 15A a threshold L = 0 has been used for the binarization for the detection image DETBb according to 15B a threshold L = 1, for the detection image DETBc according to 15C a threshold L = 2, and for the detection image DETBd according to 15D a threshold L = 4.

In the pictures of 15th are among other things in the form of the highlighting H1 , H2 For example, to see light projections that the truck FFF also shown on the ground RF projected, e.g. to draw the attention of people and / or other drivers to yourself or to the possible danger associated with you. These projections H1 , H2 are preferably clearly bordered and move with the vehicle FFF, for example. In further preferred embodiments, they are (likewise) detected by a change detection, which is also desired in further preferred embodiments.

Using the highlighting, for example H1 , H2 in 15th it can be seen how the variation of the threshold L affects the detection image DETB ( 2A ) can affect. It can be observed that L should not be selected too small in further preferred embodiments (see 15A, L = 0), because otherwise camera noise and / or artifacts, for example due to image compression (“compression artifacts”), could lead to incorrect detections. On the other hand, in other preferred embodiments, L should not be too large (see 15D, L = 4), otherwise the detection becomes too insensitive, so that small objects H2 or slowly moving objects could be overlooked. In further preferred embodiments, however, it is possible to find a sensible compromise for the choice of the threshold L. In further preferred embodiments, this depends in particular on the quality of the camera image or the image sensor and on the word length N of the descriptor. In further preferred embodiments, the threshold is, for example, L = 2.

In further preferred embodiments, as an alternative or in addition to an at least temporary storage of descriptor values in the first and / or second descriptor image DB1 , DB2 ( 2A ) the descriptor values can also be determined if necessary, eg "on the fly". In other words, in further preferred embodiments, the transforming 102 of the first Image B1 in the first descriptor image DB1 and / or transforming 102 of the second picture B2 in the second descriptor image DB2 also dynamically, that is to say, for example, when required and / or in real time, in particular, for example, also in a direct temporal connection with the determination 104 of the distance image DISTB to be executed.

In further preferred embodiments, the transforming can 102 of the first picture B1 in the first descriptor image DB1 and / or transforming 102 of the second picture B2 in the second descriptor image DB2 can also be parallelized at least temporarily, for example if several computing cores 202a a device 200 (see below to 11 ) to carry out the respective transformation (s) 102 Are available.

In further preferred embodiments, the transformation 102 to form the respective (first and / or second) descriptor image DB1 , DB2 at the same time or at least partially overlapping in time on a first part of the (first and / or second) image B1 , B2 to execute. In other words, the transformation is also conceivable in further preferred embodiments 102 of the first picture B1 in the first descriptor image DB1 to parallelize, for example a first part of the first image B1 in a first part of the first descriptor image DB1 is transformed, and at the same time, for example, a second part of the first image B1 into a second part of the first descriptor image DB1 is transformed.

In further preferred embodiments, the first descriptor image belonging to a first point in time, for example the current point in time, becomes DB1 with a second descriptor image DB2 compared that belongs to a second, in particular earlier, point in time.

In further preferred embodiments, both the transforming 102 to the descriptor images DB1 , DB2 as well as determining 104 of the distance image DISTB in each case, for example, only on a predeterminable sub-area of the first and second images B1 , B2 or the descriptor images that can be derived from them DB1 , DB2 are executed. In further preferred embodiments, the selection of partial areas can also be dynamic, that is to say at the runtime of the method or a device executing the method, for example 200 ( 11 ), for example based on the current content of the images B1 , B2 and / or (previously) recognized changes and / or objects and / or a state of the system 10 , e.g. its current direction of travel.

In further preferred embodiments, the comparison takes place 104 ( 2A ) of the first descriptor image DB1 with the second descriptor image DB2 , that is, the formation of the distance measure, pixel by pixel, so there is always a picture element of the first descriptor image DB1 with a corresponding picture element of the second descriptor picture DB2 compared and get a corresponding value for the distance measure ("distance value"). In other words, in further preferred embodiments, a descriptor value is in each case at a specific coordinate of the first descriptor image DB1 with the descriptor value at the corresponding (same) coordinate of the second descriptor image DB2 compared. In further preferred embodiments, the distance value determined in this way is converted into a or the distance image DISTB entered at the corresponding coordinate. It can be seen that the distance image DISTB the same size (number of pixels, for example characterizable by width and height) as the descriptor images to be compared DB1 , DB2 . In further preferred embodiments, the distance image DISTB (i.e. each picture element) have, for example, values in the range 0 to N, which are the result of the comparison between the first descriptor picture DB1 and second descriptor image DB2 characterize.

In further preferred embodiments, cf. 2 B , it is provided that the method further comprises: filtering 105 of the distance image DISTB , creating a filtered distance image DISTB ' is obtained, in particular forming 106 of the detection image DETB based on the filtered distance image DISTB ' he follows. Otherwise, the process can be carried out according to 2 B eg according to the procedure 2A correspond.

In further preferred embodiments, the filtering is used 105 ( 2 B ) executed so that the distance image DISTB is converted into a filtered image DISTB ' , which displays changes in a compact form, which is useful, for example, for a functional interface according to further preferred embodiments. 4B shows an example of a filtered distance image DISTB1 'as it is by means of filtering 105 based on the distance image DISTB1 according to 4A has been received.

For example, based on the filtered distance image DISTB1 ' according to 4B and one of the formation of the filtered distance image DISTB1 ' underlying image (not shown) is a detection image similar to the example in 14th images shown are obtained, with the information of the filtered distance image DISTB1 ' For example, they can be used to graphically highlight corresponding regions of the underlying image.

In this way, according to the detection image DETB_1 14A for example a person P1 with a goods carrier P1a through the highlighting HP1 marked. Similarly, in 14B Two people P1 , P2 by means of appropriate emphasis HP1 , HP2 for example, based on information from a filtered distance image similar to the filtered distance image DISTB1 ' according to 4B based, identified. In a comparable way, the following are listed individually 14C until 14F the objects or people named in key words are identified: 14C : a person P1 , 14D : a vehicle F1 , 14E : a cyclist R1 , 14F : a person P1 and a forklift G1 ,

In further preferred embodiments, the detection image has one or more contiguous regions, which are also referred to as “blobs” according to further preferred embodiments, for those image areas in which, in particular significant, changes between the (first and second) images or the herewith corresponding descriptor images are available. In further preferred embodiments, the blobs can, for example, based on the filtered distance image DISTB1 'as it eg in step 105 according to 2 B is obtained.

In further preferred embodiments, as an alternative or in addition to an at least temporary complete or partial storage of the (filtered) distance image DISTB , DISTB ' and / or the detection image DETB also an (at least partial) determination of the (filtered) distance image DISTB , DISTB ' and / or the detection image DETB if necessary, eg "on-the-fly". In other words, in further preferred embodiments it is conceivable to (also) not save the distance image and / or the detection image at least completely, for example in the context of step 104 respectively. 105 respectively. 106 but rather to process it further, for example, in particular immediately, for example to save memory or to be able to provide intermediate results (for example parts of the distance image and / or detection image) more quickly.

In further preferred embodiments, the detection image DETB ( 2A , 2 B ) be converted into another form in an optional further or alternative step, e.g. B. to be able to transmit it more efficiently via an interface. For this purpose, in further preferred embodiments, for example, the contours of at least one blob are approximated, for example described as polygons.

Alternatively or in addition, in further preferred embodiments the detection image can also be compressed, e.g. with a run length coding or another entropy coding, e.g. with a common coding for segment images.

In further preferred embodiments, cf. 2 B , that filtering is provided 105 comprises applying a majority filter and / or a threshold filter.

In further preferred embodiments, the distance image DISTB , see also reference numerals DISTB1 according to 4A , have a high level of detail per se and are therefore not well suited, for example, to be passed on to a functional interface and / or transmitted in any other way.

Therefore, in further preferred embodiments, the optional step already described above is 105 ( 2 B ) the filtering is provided, in further preferred embodiments that in the distance image DISTB , DISTB1 The information contained is “condensed”, that is, compressed, and thus the distance image, for example for forwarding, for example, to an optional subsequent function (e.g. determination of the detection image DETB ) suitably prepared.

As mentioned earlier, in 4A an example of a distance image DISTB1 shown as an input image for optional filtering, and in 4B the filtered distance image DISTB1 ' as the initial image of the filtering step 105 ( 2 B ). In this exemplary embodiment, both the input and the output data are binary images DISTB1 , DISTB1 ', which are shown in black and white by way of example. Black stands for “changed” or “moved”, white for the opposite.

In further preferred embodiments, the filtering 105 a majority filter is used, the mode of operation of which is based on the exemplary illustration according to FIG 5 is easy to understand. Reference symbol BA1 5 denotes an exemplary part of the distance image DISTB1 e.g. according to 4A , and reference numerals BA2 the end 5 denotes an exemplary part of the filtered distance image DISTB1 ' according to 4B . By means of the majority filter, for example, in the sliding window GF , the present 3 x 3 pixels of the distance image DISTB1 covers, determines whether the black or the white pixels are in the in the window GF are in the majority (majority). Since in the present case (due to the odd edge lengths assumed by way of example with the value “3” - in other examples, other, in particular even-numbered edge lengths are also possible) there is an odd number of pixels in the window GF the result of the majority filter is unambiguous. The output pixel AP that with the window GF at the current position in the distance image DISTB1 corresponds, receives the color of the majority, here for example black, because in the window GF the result is “5: 4” in favor of black.

For the other eight result pixels of the area BA2 the evaluation of the majority filter can be reproduced in a comparable way.

In further preferred embodiments, instead of checking the majority, which in the present example equates to a comparison with a threshold of 9/2 = 4.5, a comparison can also be made with another threshold.

the end 4B It can be clearly seen that due to the optional filtering according to preferred embodiments (in the present example with a majority filter and a filter radius of R = 15, deviating from the simplified representation according to FIG 5 ) the filtered distance image DISTB1 ', has far less noise (in the sense of a variation of the binary decision from pixel to pixel), which, for example, enables reliable detection of self-moving objects OBJ1 , OBJ2 enables.

In further preferred embodiments is a (filtered) result image DISTB1 ' ( 4B ) with an edge length of the filter of 2R + 1 by 2R many pixels shorter (narrower or lower) than the input image DISTB1 ( 4A ).

In further preferred embodiments, the filter works for filtering 105 ( 2 B ), which is a two-dimensional filter, on a square window GF with edge length 3 x 3 ( 5 ).

In further preferred embodiments, the filter window GF but also be rectangular and not square, or have a different shape (e.g. circle or polygon). In further preferred embodiments, the filter radius R is larger than in 5 shown for better clarity, e.g. with values between 2 and 30.

In further preferred embodiments, all the pixels involved in the filter window are correct GF , 5 , with the same voting weight: everyone has one vote.

In further preferred embodiments, the voice weight SGW in the middle of the window GF have a maximum and e.g. to the edge of the window GF take off what is in 6th is shown by way of example for one dimension along the coordinate axis x, see curve K1 . In contrast, curve corresponds to K2 eg filtering with constant voice weight. Curve K1 In further preferred embodiments, it can also be referred to as a “triangular filter” with the filter radius R, which effects center-weighted weighting, and a curve K2 also as a "rectangular filter".

In further preferred embodiments, the distribution of the voice weights along the other, not in 5 dimension shown (e.g. perpendicular to the plane of the drawing 6th , because of the two-dimensional filter / filter window GF ) be identical to the distribution along the dimension or axis x. The resulting voice weight can then result, for example, as a product or as the sum of the voice weights of the first dimension and the second dimension.

In further preferred embodiments, the filtering can be performed 105 ( 2 B ) eg integral filter or integral images or a representation of the triangular filter K1 can be used as a convolution of two rectangular filters or a suitable series connection of several integral filters.

In further preferred embodiments it is provided, in particular in the case of center-weighted weighting (curve K1 according to 6th ) determine a suitable decision threshold. Here one can again be guided by the idea of the majority filter according to further preferred embodiments: Imagine that in the window with an odd edge length either the black or the white elements are just one element in the majority, and that the colors are evenly distributed are (e.g. alternating in a checkerboard pattern pixel by pixel). In further preferred embodiments, the threshold is then to be selected in such a way that this simple majority just overturns the decision.
7th shows in the left half LH an example of a binary-valued pattern (here a chessboard as an arbitrarily chosen example), which in 7th from top to bottom is increasingly noisy with binary noise (so-called salt and pepper noise). The signal-to-noise ratio varies, for example, from infinity to 1/3. This noisy “input image” LH is filtered with a majority filter, here as a triangular filter with a radius of R = 7. The result of the filtering is shown in the right half RH.

For example, the left half of the image LH according to 7th the distance image DISTB according to 2 B and in the right half of the image BH the result of the filtering is the majority filter with center-weighted weighting (triangular filter K1 ) and radius R = 7. The right half of the image RH then corresponds, for example, to the filtered distance image DISTB ' ( 2 B ), which in further preferred embodiments, for example, directly as a detection image DETB is usable, or on the basis of which the detection image DETB can be determined, see step 106 the end 2 B .

How to exemplify 7th sees the step leads 105 ( 2 B ) the filtering to a preferred for many applications according to further Embodiments adequate level of (not too high) detailing.

17A until 17D represents the influence of the choice of the radius R for the optional filtering 105 ( 2 B ) shows the level of detail obtained for a real example. Two people are walking as an example P1 , P2 through the image of the camera of a forklift. In further preferred embodiments, it is important for a desired function that the people P1 , P2 to be detected, for example so that the driver of the forklift can be made aware of this, or an autonomous or remote-controlled transport system can appropriately target the people P1 , P2 can react. In 17A , 17B the results of the filtering according to further preferred embodiments are too detailed ( 17A : Radius R = 0, 17B : R = 5). In 17C if R = 15 there are two detections for the two people P1 , P2 correspond, see the two assigned emphases H1 , H2 . This level of detail is well suited, for example, for change detection in accordance with further preferred embodiments. If you increase the filter radius further, e.g. B. on R = 50, cf. 17D so the highlights merge H1 , H2 of the two people P1 , P2 to a single blob H12 and the details of the people P1 , P2 (e.g. feet, heads) are reproduced less precisely. Depending on the application, according to further preferred embodiments, this coarser level of detail can also be sufficient.

In further preferred embodiments, cf. 2C , it is provided that the method further comprises: further processing 108 of the detection image, in particular the formation of output information AI, based on at least one of the following elements: a) detection image, b) first image, c) second image.

In further preferred embodiments it is provided that the output information AI has at least one of the following elements: a) acoustic signal, b) haptic signal, c) optical signal, d) image, in particular digital image, with at least one graphically highlighted image area, in particular one based on the detection image (or the filtered) distance image DISTB , DISTB ' ) graphically highlighted image area, see, for example, the highlighting HP1 , HP2 , F1 , R1 , P1 , G1 according to 14th and / or the highlights H1 , H2 according to 15th and / or the highlights H1 , H2 , H12 according to 17th . As a result, one can use the method (cf. 2A ) detected change with respect to the first and second images B1 , B2 efficiently brought to the attention of a user, for example a driver of a forklift truck. In this way, the driver can be informed reliably and easily perceptibly, for example, about changes in the surroundings of his vehicle, whereby, for example, accidents with people approaching the vehicle can be avoided.

In further preferred embodiments, cf. 2D , it is provided that the method further comprises: Assign 110 an evaluation measure BM for at least one descriptor of the first descriptor image DB1 ( 2A , 2 B ) and / or the second descriptor image DB2 , and, optionally, consider 112 of the assessment measure BM when determining the distance image DISTB , wherein in particular the evaluation measure BM is a noise or a signal-to-noise ratio, in particular a region of the first image associated with the respective descriptor B1 and / or the second image B2 , characterized.

In further preferred embodiments, for example, during the formation of the descriptors or the descriptor images DB1 , DB2 the noise can also be taken into account, for example according to the in DE 10 2017 212 339 described procedure.

In further preferred embodiments, at least some, in particular each, formed descriptor (s) can be given an evaluation that is dependent on the strength of the noise or the signal-to-noise ratio (SNR): in some preferred embodiments, for example, in the form of a binary one Additional information (for example coded as an additional bit) which, in further preferred embodiments, can also be interpreted as the suitability of the descriptor for further processing.

In further preferred embodiments, the additional information allows, for example, to identify suitable descriptors less well (e.g. for subsequent further processing) due to the noise, e.g. B. as "unsuitable", e.g. to treat them differently in the event of a change detection, than descriptors that are better suited for further processing, e.g. due to lower noise or higher SNR.

In further preferred embodiments, this additional information can be used when determining the distance image ("distance calculation") DISTB ( 2A ) for example according to the following rule: If at least one of the descriptors to be compared is marked as "unsuitable", the distance is not formed according to the usual rule, but a different rule is used, according to which, for example, the actual distance is determined by a specifiable, in particular fixed, Value is replaced. In further preferred embodiments, in the example of the Hamming distance as a distance measure, the value can then be set to "0", for example, which means, for example, that image regions dominated by noise are treated as if they were motionless.

In further preferred embodiments, however, the Hamming distance can also be set to a different value, e.g. B. "1" or "2", so that, for example, areas excluded due to noise are not treated differently (e.g. better placed) than non-excluded, unmoved areas. In further preferred embodiments, this consideration can be particularly relevant due to an optional subsequent filtering of the distance image, in which, for example, a sliding window for the filtering can contain both excluded and non-excluded pixels at the same time.

• Stufe 0: Aufgrund von Rauschen sollte der Deskriptor nicht verwendet werden.
• Stufe 1: Aufgrund von Rauschen ist der Deskriptor zwar für „Anwendung A“ geeignet, nicht jedoch für „Anwendung B“.
• Stufe 2: Der Deskriptor ist für die „Anwendungen A“ und „B“ geeignet, das Rauschen ist somit z.B. nicht signifikant.

In further preferred embodiments, the evaluation of the noise is not passed on as binary information, but rather more than two levels are provided, for example three or four (or more) levels. If, for example, three stages are provided according to further preferred embodiments, these could have the following meanings in further preferred embodiments:

• Level 0: The descriptor should not be used due to noise.
• Level 1: Due to noise, the descriptor is suitable for "Application A", but not for "Application B".
• Level 2: The descriptor is suitable for "Applications A" and "B", so the noise is, for example, not significant.

In further preferred embodiments, "Applications A" and "B" can stand for two applications from, for example: optical flow, change detection, correspondence formation, disparity estimation, tracking, etc.

In the case of further preferred embodiments, further refinements that deviate therefrom are also conceivable. For example, the local signal-to-noise ratio of a viewed image region (e.g. associated with a descriptor, i.e. taken into account in the formation of the descriptor) can be attached to the respective descriptor as additional information (e.g. in the sense of a concatenation), e.g. B. as a number. In further preferred embodiments, a distance calculation can evaluate and pass on this additional information, for example as a minimum or maximum or mean value of the two numbers of the descriptors involved in the distance calculation. In further preferred embodiments, this additional information can also be used, for example, in a subsequent optional filtering step 105 ( 2 B ) can be further taken into account and, if necessary, passed on, e.g. as the confidence of a decision made about the existence of an object OBJ1 , OBJ2 ( 4B ) in the first and / or second image.
In further preferred embodiments, cf. 2E , it is provided that the method further comprises: at least temporary storage 114 of the first descriptor image DB1 and / or the second descriptor image DB2 , e.g. for subsequent use (see e.g. step 104 according to 2A ). For example, when two distance images are determined consecutively, a participating descriptor image can be used twice for the distance calculation, namely once in the sense of the first descriptor image and a second time in the sense of the second descriptor image.

In further preferred embodiments it is provided that the method further comprises: compensation 116 one with the first picture B1 ( 1 ) and / or the second picture B2 associated movement, in particular a proper movement of the first image B1 and / or the second image B2 deploying camera 12th , for at least one area RF , in particular surface, for example a homography compensation with respect to the ground plane RF .

• B105: zweites Kamerabild, vgl. auch Bild B2 gemäß 2A, wobei Bild B105 zu einem früheren Zeitpunkt erfasst wurde als das erste Kamerabild B101,
• B200: Transformation, die ein Kamerabild in ein Deskriptorbild wandelt, s. z.B. auch Schritt 102 gemäß 2A, 2B
• B201: erstes Deskriptorbild, vgl. auch Bezugszeichen DB1 gemäß 2A, 2B,
• B205: zweites Deskriptorbild, vgl. auch Bezugszeichen DB2 gemäß 2A, 2B,
• B300: Schritt des Vergleichens (vgl. auch Bezugszeichen 104 gemäß 2A, 2B) der beiden Deskriptorbilder, dabei entsteht ein Distanzbild B303,
• B400: optionaler Filterungsschritt (vgl. auch Schritt 105 gem. 2B), z.B. um im Falle bewegter Objekte OBJ (1) wenige(r) zusammenhängende Regionen zu erhalten,
• B403: Detektionsbild, das z.B. gemäß weiteren bevorzugten Ausführungsformen direkt in Form des mittels Schritt B400 gefilterten Distanzbilds erhalten wird, B500: optionaler Schritt der Weiterverarbeitung des Detektionsbilds B403,
• B503 weiterverarbeitete Form des Detektionsbilds, die z.B. geeignet ist für eine Übertragung über eine Schnittstelle, z. B. an ein Warnsystem oder eine Aktuatorik (nicht gezeigt), oder zur Ausgabe auf einer Anzeigevorrichtung (nicht gezeigt), z.B. für den Fahrer des Fahrzeugs 10 (1).

8th schematically shows a simplified block diagram according to further preferred embodiments. The identifier B101 until B503 have the following meaning according to further preferred embodiments: B101 : first camera image, see also image B1 according to 2A ,

• B105: second camera image, see also image B2 according to 2A , taking picture B105 was captured at an earlier point in time than the first camera image B101 ,
• B200: Transformation that converts a camera image into a descriptor image, see also step 102 according to 2A , 2 B
• B201: first descriptor image, see also reference symbols DB1 according to 2A , 2 B ,
• B205: second descriptor image, see also reference symbols DB2 according to 2A , 2 B ,
• B300: step of comparing (see also reference numbers 104 according to 2A , 2 B ) of the two descriptor images, creating a distance image B303 ,
• B400: optional filtering step (see also step 105 according to 2 B ), e.g. in the case of moving objects OBJ ( 1 ) to preserve a few contiguous regions,
• B403: Detection image, which, for example, according to further preferred embodiments directly in the form of the means of step B400 filtered distance image is obtained, B500: optional step of further processing the detection image B403 ,
• B503 further processed form of the detection image which is suitable, for example, for transmission via an interface, e.g. B. to a warning system or actuators (not shown), or for output on a display device (not shown), for example for the driver of the vehicle 10 ( 1 ).

9 schematically shows a simplified block diagram according to further preferred embodiments. Compared to 8th the following elements are added:

B250: Buffer for descriptor images. This provides, for example, older descriptor images B205 ready and keeps them until they are no longer needed. This saves recalculations. It can be per new input image B101 for example a transformation B200 be sufficient, as opposed to twice as many at 8th , when configuring the optional buffer B250 is not provided. With the optional memory B250 according to 9 In further preferred embodiments, a memory (not shown here) for storing camera images can also be used B101 , e.g. until they are needed, B105 , omitted.

• B150 optionaler Schritt der Kompensation einer Eigenbewegung in einem Bild, für eine Oberfläche RF (1). Zum Beispiel eine Homographie-Kompensation bezüglich der Bodenebene RF,
• B151: Parameter zur Ansteuerung der Kompensation B150 für das erste Bild B101,
• B155 Parameter zur Ansteuerung der Kompensation B150 für das zweite Bild B105,
• B111 optional kompensiertes, erstes Kamerabild,
• B115 optional kompensiertes, zweites Kamerabild,

10 schematically shows a simplified block diagram according to further preferred embodiments. Compared to 8th the following elements are added:

• B150 optional step of compensating a proper movement in an image for a surface RF ( 1 ). For example homography compensation with respect to the ground plane RF ,
• B151: Parameter for controlling the compensation B150 for the first picture B101 ,
• B155 Parameter for controlling the compensation B150 for the second picture B105 ,
• B111 optionally compensated, first camera image,
• B115 optionally compensated, second camera image,

In the case of further preferred embodiments, a compensation is already sufficient B150 only in the upper branch Z1 or in the lower branch Z2 to be provided. You in both branches Z1 , Z2 providing and dividing the work in half, for example, can have numerical advantages in further preferred embodiments.

When the compensation B150 according to further preferred embodiments only, for example, in the lower branch Z2 takes place, i.e. for the second camera image B105 , this is for example compensated in such a way that it (with regard to the selected surface RF ) to the first and thus newest camera image B101 fits. In further preferred embodiments, this variant is usually the preferred one, because then the resulting detection image B403 also in the coordinates of the latest camera image B101 is present. This is particularly advantageous for real-time visualization in accordance with further preferred embodiments.

Further preferred embodiments, cf. 11 , refer to a device 200 for processing, especially digital, images B1 , B2 , the device 200 to carry out the method according to the embodiments (cf. 2 ) is trained.

In further preferred embodiments it is provided that the device 200 has: at least one computation kernel 202a having computing device 202 ("Computer"), one of the computing devices 202 associated storage device 204 for at least temporary storage of at least one of the following elements: a) data DAT, b) computer program PRG, in particular for executing the method according to the embodiments.

In further preferred embodiments, the data DAT can at least temporarily and / or partially comprise the at least one video data stream VDS (or part of it) and / or the first picture B1 and / or the second image B2 and / or data that can be derived therefrom, for example the first descriptor image DB1 and / or the second descriptor image DB2 or the distance image DISTB (respectively. DISTB ' ) or the detection image DETB or at least have parts thereof.

In further preferred embodiments, the memory device 204 volatile memory 204a (e.g. main memory (RAM)) and / or a non-volatile memory 204b (e.g. Flash EEPROM).

In further preferred embodiments, the computing device 202 also have at least one of the following elements: microprocessor (µP), microcontroller (µC), application-specific integrated circuit (ASIC), system on chip (SoC), programmable logic module (e.g. FPGA, field programmable gate array), hardware circuit, graphics processor (GPU, graphics processing unit), or any combination thereof.

Further preferred embodiments relate to a computer-readable storage medium SM, comprising instructions PRG which, when executed by a computer 202 cause this, carry out the method according to the embodiments.

Further preferred embodiments relate to a computer program PRG, comprising instructions that are used when the program PRG is executed by a computer 202 cause them to carry out the method according to the embodiments.

Further preferred embodiments relate to a data carrier signal DCS that characterizes and / or transmits the computer program PRG according to the embodiments. The data carrier signal DCS is for example via an optional data interface 206 , 208 the device 200 receivable. In further preferred embodiments, there are also the video data stream VDS or the pictures B1 , B2 via an optional data interface 206 receivable.

The device 200 or components 202 , PRG thereof can, in further preferred embodiments, also be designed, for example, to at least temporarily conform to the configuration 8th and / or 9 and / or 10 to be implemented.

Further preferred embodiments, cf. 12th , refer to a use 300 of the method according to the embodiments and / or the device 200 according to the embodiments and / or the computer-readable storage medium SM according to the embodiments and / or the computer program PRG according to the embodiments and / or the data carrier signal DCS according to the embodiments for at least one of the following elements: a) detection 302 a1) an environment U ( 1 ) of a system 10 , in particular a vehicle, in particular a floor conveyor such as for example forklifts and / or forklifts, and / or a2) a scene SZ , b) Determine 304 of changes to the first image B1 and the second image B2 , in particular robust change detection with respect to the first image and the second image, c) determining 306 of obstacles OBJ , H ( 1 ), especially moving objects, d) avoidance 308 of accidents, especially in the area of one or the system 10 or in the environment U , especially at comparatively low system speeds 10 , in particular less than or equal to a walking speed of a person, further in particular in the case of a transition between a standstill and a movement or vice versa, e) application 309 both when the system is at a standstill and when the system is in motion, f) visualizing (not shown) changes in relation to the first image and the second image, in particular displaying information or additional information characterizing the changes or highlighting at least one area of the first Image and / or the second image associated with the changes, see the highlighting HP1 , HP2 according to 14B .

Robust change detection 304 According to further preferred embodiments means, for example, no number of false detections (false alarms) or a number below a predeterminable threshold value, e.g. B. because the lighting changes suddenly (e.g. by flashing lights, warning lights, light cones passing by, etc.), but at the same time reliable detection of relevant cases, especially all relevant cases.

13th shows schematically exemplary image sequences BF1, BF2 according to further preferred embodiments, such as for example by means of the camera 12th ( 1 ) or based on the video data stream VDS can be obtained. In the case of the image sequence BF1, an image becomes periodically with the period duration ΔT − 1τ B1 , B2 , B3 , .. provided, where τ characterizes a time interval between two consecutive images, for example 1/30 second. For example, after 1τ the first image B1 and the second picture that is already available B2 according to 2A , 2 B etc. are evaluated, for example a first detection image DETB to investigate. For example, after 2τ the image B2 and the picture that is already available B3 according to 2A , 2 B (in the sense of the pictures B1 , B2 ) etc. can be evaluated, for example by a second detection image DETB to determine, etc.

In further preferred embodiments, see the sequence of images BF2 according to 13th , it is provided that in the video data stream temporally between the first image B1 and the second picture B2 at least one (in the present case three) further image (s) B 'is or are present. In other words, in further preferred embodiments, a time interval ΔT between the first image can be used B1 and the second picture B2 For example, they can be changed by not using directly consecutive images e.g. of the video data stream between the first and second image, but rather that e.g. one or more images B 'e.g. of the video data stream between the first image and the second image are omitted, see the arrow B12 , for which applies: ΔT - 4τ.

In further preferred embodiments it is provided that the method (in particular the steps of providing 100 and / or transforming 102 into the descriptor images and / or the determination 104 the distance image and, optionally, the determination 106 of the detection image) to several first and second images B1 , B2 having image pairs is applied, the first image B1 a first image of the video data stream of a first image pair VDS is where the second picture B2 of first image pair an n-th image, with n> 2 (eg n = 5), of the video data stream VDS wherein the first image of a second image pair is a second image of the video data stream, wherein the second image of the second image pair is an (n + 1) th image of the video data stream VDS is.

In further preferred embodiments, an m-th (m = 1, 2, 3, ...) image pair can have the m-th image of a video data stream as the first image, and an (m + k) -th image (with k > 1) of the video data stream. This is in 13th for the sequence of images BF2 through the (with the exception of arrow B12 ) symbolizes unmarked arrows, an origin of an arrow with the respective first image B1 of a pair of images viewed corresponds, and a tip of the same arrow points to the respective second image of the pair of images viewed. The first pair of images thus corresponds, for example, to the arrow B12 , and the second and all further pairs of images the in 13th in the sequence of images BF2 arrows that are further to the right and are not individually labeled.

As a result, in spite of “omitting” images (with regard to the respective image pair being viewed, corresponding to the value of the parameter k), it is possible to react quickly and, in particular, detections (for example in the form of the detection image) can also be carried out DETB ) are output in quick chronological order, which can be important for a graphical display for a user (for example, so that it does not jerk).

16 shows different detection images for this, which have been obtained based on camera images each with a different time interval ΔT, where for 16A ΔT - 1τ holds for 16B ΔT - 2τ, for 16C ΔT - 4τ, and for 16D ΔT - 8τ, see also 13th . It can be seen that the different time intervals between the camera images lead to different detection results. In particular, the time interval ΔT between the two camera images can be B1 , B1 (see also B101 , B105 the end 8th ) have a significant influence on the detection result. If the time interval is small ( 16A ), only comparatively fast movements are detected, cf. the feet PF of person P. If he is tall, cf. 16D the detection may be too sensitive.

In further preferred embodiments, it is therefore useful to provide a suitable time interval ΔT, which in further preferred embodiments can also be dynamically adapted if necessary. In further preferred embodiments, this also includes, for example, the possibility of a triggered camera recording, in which the camera does not record or provide the images in a fixed time grid, but in a controllable variable time interval.

In further preferred embodiments, the time interval ΔT can be specified as follows, for example: a) Selection of the time interval τ between successive images B1 , B2 ; B2 , B3 , ..., b) skipping pictures. When skipping by omitting images, the case could arise, for example, that the method or system reacts too slowly to changes.

Therefore, based on the sequence of images BF2 according to 13th a solution for skipping without skipping images according to further preferred embodiments is shown. Every newest picture is exemplified here B101 used and the change detection, for example, with the fourth from last image B105 carried out. The possible disadvantage of a sluggish reaction is thus eliminated. Nevertheless, the time interval ΔT is sufficiently large (here, for example, 4/30 s) to be able to reliably detect even slowly moving objects.

In further preferred embodiments it is also possible to work in parallel with different time intervals ΔT in order to detect both (e.g. with a small ΔT) fast moving or large objects and (e.g. with a large ΔT) slowly moving or small objects. In further preferred embodiments, the detection results can be combined or fused.

In other words, in further preferred embodiments it is provided that the method is carried out, in particular in parallel in time, on respectively different images of the same or the same video data stream, see also step 118 according to 2F , wherein a respective first image and a respective second image each have a different time interval from one another.

The principle according to preferred embodiments enables, among other things, a particularly robust change detection in camera images B1 , B2 with comparatively little computing effort. Robust means in particular that it does not provide any false detections (false alarms), e.g. B. because the lighting suddenly changes (through flashing lights, warning lights, light cones passing by, etc.), but at the same time reliably detects relevant cases.

According to preferred embodiments, the detections can be supplied in a compactness that is appropriate for the subsequent function. This means, for example, when a person moves through the image, according to further preferred embodiments, if possible, an object should be connected to an interface (for example a system for visualization, that is to say for example outputting the images B1 , B2 , if necessary with highlighting based on the detected objects OBJ , OBJ1 , OBJ2 ) to be delivered.

If a group of people moves through the image together and closely spaced from one another, according to further preferred embodiments it is also sufficient to deliver a coherent object to the interface that includes the group, since according to further preferred embodiments (at least initially) the group needs to respond - not to each person individually.

The absence of false detections and appropriate (not unnecessarily annoying) reactions to relevant cases according to further preferred embodiments are of importance for user acceptance of the system, which should not be underestimated.

These requirements can at least temporarily meet at least some preferred embodiments. At the same time, the (additional) computational effort for realizing the change detection according to further preferred embodiments is very low.

QUOTES INCLUDED IN THE DESCRIPTION

This list of the documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102017212339 [0031, 0113]
• US 6711293 [0066]

Non-patent literature cited

• Bay H., Tuytelaars T., Van Gool L. (2006) SURF: Speeded Up Robust Features [0008]
• Leonardis A., Bischof H., Pinz A. (eds) Computer Vision - ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3951. Springer, Berlin, Heidelberg, https://doi.org/10.1007/11744023_32 [ 0008, 0066]
• E. Rublee, V. Rabaud, K. Konolige and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," 2011 International Conference on Computer Vision, Barcelona, 2011, pp. 2564-2571; DOI: 10.1109 / ICCV.2011.6126544 [0008, 0066]
• Leutenegger, Stefan & Chli, Margarita & Siegwart, Roland. (2011). BRISK: Binary Robust invariant scalable keypoints. Proceedings of the IEEE International Conference on Computer Vision. 2548-2555. 10.1109 / ICCV.2011.6126542.), [0008]
• E. Tola, V. Lepetit and P. Fua, "DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 815-830, May 2010. [0008, 0066]
• Speeded Up Robust Features, e.g. according to Bay H., Tuytelaars T., Van Gool L. (2006) SURF: Speeded Up Robust Features [0066]
• Leutenegger, Stefan & Chli, Margarita & Siegwart, Roland. (2011). BRISK: Binary Robust invariant scalable keypoints. Proceedings of the IEEE International Conference on Computer Vision. 2548-2555. 10.1109 / ICCV.2011.6126542. [0066]
• Binary Robust Independent Elementary Features, DOI: 10.1007 / 978-3-642-15561-1_56 [0066]

Claims (18)

A method, in particular a computer-implemented method, for processing, in particular digital, images (B1, B2), comprising the following steps: providing (100) a first image (B1) and a second image (B2), transforming (102) the first image (B1) into a first descriptor image (DB1) and the second image (B2) into a second descriptor image (DB2), determining (104) a distance image (DISTB) based on a comparison of the first descriptor image (DB1) with the second descriptor image (DB2 ), and, optionally, forming (106) a detection image (DETB) based on the distance image (DISTB). Procedure according to Claim 1 , further comprising: filtering (105) the distance image (DISTB), whereby a filtered distance image (DISTB ') is obtained, in particular the formation (106) of the detection image (DETB) taking place based on the filtered distance image (DISTB'). Procedure according to Claim 2 wherein the filtering (105) comprises applying a majority filter and / or a threshold filter. Method according to at least one of the preceding claims, further comprising: further processing (108) of the detection image (DETB), in particular formation of output information (AI), based on at least one of the following elements: a) detection image (DETB), b) first image (B1) ), c) second image (B2). Procedure according to Claim 4 , the output information (AI) having at least one of the following elements: a) acoustic signal, b) haptic signal, c) optical signal, d) image, in particular digital image, with at least one graphically highlighted image area, in particular one based on the detection image ( DETB) graphically highlighted image area. Method according to at least one of the preceding claims, wherein a descriptor is used for transforming (102) for which a distance measure and / or a similarity measure is definable and / or defined, in particular comparing (104) the first descriptor image (DB1) with the second descriptor image (DB2) is executed based on the distance measure and / or the similarity measure. Method according to at least one of the preceding claims, wherein the Hamming distance or a distance measure based on the Hamming distance is used as the distance measure for the comparison (104), the Hamming distance in particular being compared with a predeterminable threshold value and based on the comparison a comparison value, in particular a binary one, is determined. Method according to at least one of the preceding claims, further comprising: assigning (110) an evaluation measure (BM) to at least one descriptor of the first descriptor image (DB1) and / or the second descriptor image (DB2), and, optionally, taking into account (112) the evaluation measure (BM) in the determination (104) of the distance image (DISTB), in particular the evaluation measure (BM) being a noise or a signal-to-noise ratio, in particular an area of the first image (B1) and associated with the respective descriptor / or of the second image (B2). Method according to at least one of the preceding claims, further comprising: at least temporarily storing (114) the first descriptor image (DB1) and / or the second descriptor image (DB2). The method according to at least one of the preceding claims, further comprising: compensating (116) a movement associated with the first image (B1) and / or the second image (B2), in particular an intrinsic movement of the first image (B1) and / or the second Image (B2) providing camera (12) for at least one surface (RF), in particular surface. Method according to at least one of the preceding claims, wherein the first image (B1) and the second image (B2) are each part of the same video data stream (VDS) of at least one camera (12). Procedure according to Claim 11 , wherein at least one further image (B ') is present in the video data stream (VDS) temporally between the first image (B1) and the second image (B2), and / or wherein in particular the method is based on a plurality of first and second images Image pairs is applied, the first image of a first image pair being a first image of the video data stream (VDS), the second image of the first image pair being an n-th image, with n> 2, of the video data stream (VDS), the first image being of a second image pair is a second image of the video data stream (VDS), the second image of the second image pair being an (n + 1) th image of the video data stream (VDS). Method according to at least one of the preceding claims, wherein the method is carried out (118), in particular temporally parallel, on different images of the same or the same video data stream (VDS), a respective first image and a respective second image each having a different time interval have to each other. Device (200) for processing, in particular digital, images (B1, B2), the device (200) being designed to carry out the method according to at least one of the preceding claims. Computer-readable storage medium (SM), comprising instructions (PRG) which, when executed by a computer (202), cause the computer (202) to perform the method according to at least one of Claims 1 until 13th to execute. Computer program (PRG), comprising instructions which, when the program (PRG) is executed by a computer (202), cause the computer (202) to execute the method according to at least one of the Claims 1 until 13th to execute. Data carrier signal (DCS) that the computer program after Claim 16 transmits and / or characterizes. Use (300) of the method according to at least one of Claims 1 until 13th and / or the device (200) Claim 14 and / or the computer-readable storage medium (SM) according to Claim 15 and / or the computer program (PRG) Claim 16 and / or the data carrier signal (DCS) Claim 17 for at least one of the following elements: a) detection (302) a1) an environment (U) of a system (10), in particular a vehicle, in particular an industrial truck such as a forklift and / or forklift, and / or a2) a scene (SZ) , b) determining (304) changes in relation to the first image (B1) and the second image (B2), in particular robust change detection in relation to the first image (B1) and the second image (B2), c) determining (306) obstacles (H), in particular moving objects, d) avoidance (308) of accidents, in particular in the area of a system (10) or in the environment (U), in particular at comparatively low speeds of the system (10), in particular at lower speeds equals a walking pace of a person, further in particular when there is a transition between a standstill and a movement or vice versa, e) application (309) both when the system (10) is standing still and when the system (10) is moving.

Images