DE102020215415A1 - Method and device for detecting a sturgeon plant image in a camera raw image and image processing device - Google Patents

Abstract

The approach presented here relates to a method for detecting an image of a sturgeon plant (107) in a raw camera image (110). At least one R color signal (140) representing a sensed red value, which represents a signal provided by a red pixel (R) of a picture element (150) of an image sensor (115), and an NIR color signal (155) representing a sensed infrared gray value , which represents a signal provided by a NIR pixel (NIR) of the picture element (150). The sensed infrared gray level is compared to a defined minimum infrared limit (160) to obtain a first comparison result (162), the sensed infrared gray level is compared to a defined maximum infrared limit (164) to obtain a second comparison result (166), and the sensed Red value compared with a defined maximum red limit value (168) to obtain a third comparison result (170). The pixel (150) is recognized as a sturgeon plant image (107) or plant image (175) using the first comparison result (162), second comparison result (166) and third comparison result (170).

Description

State of the art

The approach is based on a device or a method according to the species of the independent claims.

The normalized differentiated vegetation index, "NDVI" for short (Normalized Difference Vegetation Index), is an index that shows the biomass, i.e. the proportion of plants, in one image. Plants reflect relatively little radiation in the red spectral range and relatively much radiation in the near infrared spectral range. Infrared cameras or cameras with Bayer sensors can be used to determine the NDVI when observing vegetation.

Disclosure of Invention

Against this background, with the approach presented here, a method for detecting an image of a sturgeon plant in a raw camera image, a device that uses this method, and finally an image processing device according to the main claims are presented. Advantageous developments and improvements of the device specified in the independent claim are possible as a result of the measures listed in the dependent claims.

The advantages that can be achieved with the approach presented are that a particularly precise NDVI image can be generated, which only depicts relevant plants.

A method for detecting a sturgeon plant image in a raw camera image is presented. The method has a reading step, a comparing step and a recognition step. In the reading step, at least one R color signal representing a sensed red value, which represents a signal provided by a red pixel of a pixel of an image sensor, and an NIR color signal representing a sensed infrared gray value, which represents a signal provided by an NIR pixel of the pixel represented, read. In the comparison step, the sensed infrared gray value is compared with a defined minimum infrared limit value in order to obtain a first comparison result, the sensed infrared gray value is compared with a defined maximum infrared limit value in order to obtain a second comparison result, and the sensed red value is compared with a defined maximum red limit value in order to to obtain a third comparison result. In the recognition step, the pixel is recognized as a disturbing plant image or plant image using the first comparison result, second comparison result and third comparison result.

This method can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control unit.

Red and infrared light is used for plant detection. Plants reflect relatively little radiation in the red spectral range and relatively much radiation in the near infrared spectral range. An NDVI image is created from the normalized difference between the two spectral ranges, which depicts the plants. However, it is possible that, for example due to unfavorable lighting conditions such as overexposure or underexposure, image areas in the conventional NDVI image incorrectly identify plants where there are no plants at all, these are referred to here as "interfering plants" and their depiction as "interfering plant illustration" or "interfering object". designated. Thanks to the method presented here, these sturgeon plant images can be recognized using a comparison with limit values.

The image sensor can be an infrared sensor with a pure red channel and a pure near infrared channel. However, the image sensor can also be a Bayer sensor with at least one red pixel with an R color filter element for transmitting red light and/or infrared light and with at least one NIR pixel designed for transmitting near infrared light. The NIR pixel can be a blue pixel with a B color filter element for transmitting blue light and near infrared light and/or one of at least two green pixels with a G color filter element each for transmitting green light and infrared light of the image sensor. The Bayer sensor can also have a filter device covering the pixels of the image sensor for transmitting red light and near-infrared light and/or for blocking green light and/or blue light. In such a Bayer sensor with a filter device, the blue pixel and/or green pixel would only sense near-infrared light as the NIR pixel, and the red pixel would sense a mixed light of red light and near-infrared light.

According to one embodiment, the method can also have a removal step, in which the pixel recognized as the image of a sturgeon plant is removed from the raw camera image. This allows a comparison with a plant segmentation in an NDVI image to be carried out. If the pixel is recognized as a nuisance plant, it will no longer appear in the NDVI image as a plant segmented. In this way, an image of sturgeon plants can advantageously be avoided in the NDVI image.

In the detection step, the pixel can be recognized as a nuisance plant image if the first comparison result indicates that the sensed infrared gray value falls below the defined minimum infrared limit value and/or the second comparison result indicates that the infrared gray value exceeds the defined maximum infrared limit value and/or the third comparison result displays that the sensed red value exceeds the defined maximum red limit. If the sensed infrared gray level falls below the defined minimum infrared limit, this may indicate underexposure, if the infrared gray level exceeds the defined maximum infrared limit, this may indicate overexposure, if the sensed red level exceeds the defined maximum red limit, this may indicate overexposure.

It is also advantageous if, according to one embodiment, the pixel is recognized as a plant image in the recognition step if the first comparison result indicates that the sensed infrared gray value exceeds the defined minimum infrared limit value and/or the second comparison result indicates that the infrared gray value exceeds the defined falls below the maximum infrared limit and/or the third comparison result indicates that the sensed red value falls below the defined maximum red limit. In this way, plant images can be confirmed as such based on limit values. According to this embodiment, a pixel whose gray value has the minimum limit value remains as a plant image in exactly the same way as a pixel whose gray value reaches the maximum limit value. Correspondingly, a different assignment of the gray values lying on a threshold value is also possible if the threshold values are selected correspondingly differently.

The method can also have a plausibility check step, in which the image point recognized as a plant image is imaged in order to check a region of an NDVI image for plausibility. Plants are therefore not recognized in the NDVI image, since the NDVI image is then already generated. However, areas in the image can be checked for plausibility using the limit values in the infrared, red and IR+red channels. That means, as an example, there must not be a plant at one point (image point or pixel) in the image because the plausibility check has recognized it as such, and vice versa. It could be plausible that there could be a plant at one point, then it also depends on the result of the NDVI segmentation whether there is a plant at this point in the NDVI image at all. In this way, plants can be reliably displayed in the NDVI image in order to subsequently enable selective application of herbicides to the plants, for example.

According to one embodiment, the method can also have a step of applying a dilation in which further pixels adjacent to the pixel are also recognized as a sturgeon plant image if the pixel was recognized as a sturgeon plant image in order to recognize an extended sturgeon plant image. Such a dilatation is advantageous, since edge areas of the interfering plant image that still belong to the interfering plant image can have a color value progression, which can have color values that can lie outside of the defined limit values. For example, according to one embodiment, the expanded sturgeon plant image can then be removed from the raw camera image in the removal step. In this way it can be ensured that the entire sturgeon plant image is not imaged in the NDVI image.

In the application step, a filter size for the dilation can be set. The filter size can be parameterized independently of the number of sturgeon plant images.

In the reading step, the NIR color signal can be read from the NIR pixel, which is a blue pixel and/or a green pixel of the image sensor. The blue pixel may have a B color filter element for transmitting blue light and near infrared light and/or the green pixel may have a G color filter element for transmitting green light and infrared light. A filter device for letting through red light and near-infrared light and/or for blocking green light and/or blue light can be arranged on the blue pixel and/or green pixel. The image sensor can be a Bayer sensor, which has 50% green pixels, 25% red pixels and 25% blue pixels, with the green pixels being arranged diagonally opposite one another. Such a Bayer sensor is advantageously inexpensive to purchase.

In the reading step, the R color signal can be read, which represents a mixed value of red light and near-infrared light, wherein in the comparing step the mixed value is compared to a defined maximum mixing limit value in order to obtain a fourth comparison result and/or compared to a defined minimum mixing limit value in order to obtain a fifth comparison result, wherein in the recognition step the pixel is recognized as a sturgeon plant image or plant image using the fourth comparison result and fifth comparison results. This can be done using the preceding described Bayer sensor with color filter device.

Alternatively, in the reading step, a mixed light signal representing a sensed mixed value of red light and near-infrared light can also be read in, which represents a signal provided by the red pixel, wherein in the comparing step the sensed mixed value is compared with a defined maximum mixed limit value in order to obtain a fourth to obtain a comparison result and/or is compared with a defined minimum mixing limit value in order to obtain a fifth comparison result, wherein in the recognition step the pixel is recognized as a sturgeon plant image or plant image using the fourth comparison result and fifth comparison results.

In the recognition step, the pixel can be recognized as a nuisance plant image if the fourth comparison result indicates that the mixed value has reached or fallen below the defined minimum mixed limit and/or reached or exceeded the defined maximum mixed limit. If the blend value equals or falls below the defined minimum blend limit, this can indicate underexposure, if the blend value equals or exceeds the defined maximum blend limit, it can indicate overexposure.

The approach presented here also creates a device that is designed to carry out, control or implement the steps of a variant of a method presented here in corresponding devices. The task on which the approach is based can also be solved quickly and efficiently by this embodiment variant of the approach in the form of a device.

For this purpose, the device can have at least one computing unit for processing signals or data, at least one memory unit for storing signals or data, at least one interface to a sensor or an actuator for reading in sensor signals from the sensor or for outputting data or control signals to the Have actuator and / or at least one communication interface for reading or outputting data that are embedded in a communication protocol. The arithmetic unit can be, for example, a signal processor, a microcontroller or the like, with the memory unit being able to be a flash memory, an EEPROM or a magnetic memory unit. The communication interface can be designed to read in or output data wirelessly and/or by wire, wherein a communication interface that can read in or output wire-bound data can, for example, read this data electrically or optically from a corresponding data transmission line or can output it to a corresponding data transmission line.

In the present case, a device can be understood to mean an electrical device that processes sensor signals and, depending thereon, outputs control and/or data signals. The device can have an interface that can be configured as hardware and/or software. In the case of a hardware design, the interfaces can be part of a so-called system ASIC, for example, which contains a wide variety of functions of the device. However, it is also possible for the interfaces to be separate integrated circuits or to consist at least in part of discrete components. In the case of a software design, the interfaces can be software modules which are present, for example, on a microcontroller alongside other software modules.

Furthermore, an image processing device with the device described above and the image sensor in one of the variants described above is presented. Such an image processing device is suitable for plant detection, images of sturgeon plants advantageously being able to be reliably recognized as such.

• 1 eine schematische Darstellung einer Bildverarbeitungseinrichtung mit einer Vorrichtung zum Detektieren einer Störpflanzenabbildung in einem Kamerarohbild und einem Bildsensor gemäß einem Ausführungsbeispiel;
• 2 eine schematische Darstellung eines unter Verwendung einer Vorrichtung erzeugten vorläufigen NDVI-Bilds gemäß einem Ausführungsbeispiel;
• 3 eine schematische Darstellung eines unter Verwendung einer Vorrichtung erzeugten NDVI-Bilds gemäß einem Ausführungsbeispiel;
• 4 ein Ablaufdiagramm eines Verfahrens zum Detektieren einer Störpflanzenabbildung in einem Kamerarohbild gemäß einem Ausführungsbeispiel; und
• 5 ein Blockschaltbild einer Vorrichtung gemäß einem Ausführungsbeispiel.

Exemplary embodiments of the approach presented here are shown in the drawings and explained in more detail in the following description. It shows:

• 1 a schematic representation of an image processing device with a device for detecting a sturgeon plant image in a camera raw image and an image sensor according to an embodiment;
• 2 12 is a schematic representation of a preliminary NDVI image generated using an apparatus according to an embodiment;
• 3 12 is a schematic representation of an NDVI image generated using an apparatus according to an embodiment;
• 4 a flow chart of a method for detecting a sturgeon plant image in a camera raw image according to an embodiment; and
• 5 a block diagram of a device according to an embodiment.

In the following description of favorable exemplary embodiments of the present approach Identical or similar reference symbols are used for the elements shown in the various figures and which act similarly, with a repeated description of these elements being dispensed with.

1 shows a schematic representation of an image processing device 100 with a device 105 for detecting a sturgeon plant image 107 in a camera raw image 110 and an image sensor 115 according to an embodiment.

The device 105 according to this exemplary embodiment is accommodated in or on the image processing device 100 purely by way of example. The device 105 has a reading device 120 , a comparison device 125 and a detection device 130 . According to this exemplary embodiment, the device 105 optionally also has a plausibility check device 135 .

Reading device 120 is designed to receive at least one R color signal 140 representing a sensed red value, which represents a signal provided by a red pixel R of a pixel 150 of image sensor 115, and at least one NIR color signal 155 representing a sensed infrared gray value, which represents a signal provided by a NIR pixel NIR of the pixel 150 to be read. The comparison device 125 is designed to compare the sensed infrared gray value with a defined minimum infrared limit value 160 in order to obtain a first comparison result 162, to compare the sensed infrared gray value with a defined maximum infrared limit value 164 in order to obtain a second comparison result 166, and to sensed red value is compared with a defined maximum red limit value 168 in order to obtain a third comparison result 170. The recognition device 130 is designed to recognize the pixel as a sturgeon plant image 107 or plant image 175 using the first comparison result 162 , second comparison result 166 and third comparison result 170 .

In this way, an NDVI image can be created based on the raw camera image 110 .

According to this exemplary embodiment, the image sensor 115 is an infrared sensor with a pure red channel realized by the red pixel R and a pure near-infrared channel realized by the NIR pixel NIR.

In addition to the pixel 150, the image sensor 115 includes a plurality of further pixels, for example in a matrix-like arrangement made up of a plurality of columns and rows. According to one embodiment, each pixel of the image sensor 115 includes at least two pixels. According to one embodiment, the pixels of the image sensor 115 are identical.

The plausibility check device 135 is designed to check plant images in the NDVI image for plausibility. For example, the plausibility checking device 135 is designed to remove the pixel 150 recognized as a sturgeon plant image 107 from the raw camera image 110 or to remove a segmentation of the pixel 150 as a plant. Additionally or alternatively, the plausibility checking device 135 is designed according to one exemplary embodiment to image the pixel 150 recognized as a plant image 175 or to maintain a segmentation of the pixel 150 as a plant. If the pixel 150 is recognized as a disturbing plant, it is therefore no longer segmented as a plant in the NDVI image that has already been generated. On the other hand, if the pixel 150 is recognized as a plant, this pixel is segmented as a plant in the NDVI image that has already been generated.

According to this exemplary embodiment, the recognition device 130 is designed to recognize the pixel 150 as a disturbing plant image 107 if the first comparison result 162 indicates that the sensed infrared gray value falls below the defined minimum infrared limit value 160 and/or the second comparison result 166 indicates that the infrared gray value exceeds the defined maximum infrared limit 164 and/or the third comparison result 170 indicates that the sensed red value exceeds the defined maximum red limit 168 . According to this exemplary embodiment, the recognition device 130 is also designed to recognize the pixel 150 as a plant image 175 if the first comparison result 162 indicates that the sensed infrared gray value exceeds the defined minimum infrared limit value 160 and/or the second comparison result 166 indicates that the infrared gray value falls below the defined maximum infrared limit value 164 and/or the third comparison result 170 indicates that the sensed red value falls below the defined maximum red limit value 168. According to this exemplary embodiment, the limit values 160, 164, 168 are chosen, merely by way of example, in such a way that pixels 150 with gray values lying on the limit values 160, 164, 168 are recognized as plant images 175.

According to an alternative exemplary embodiment, the image sensor 115 is in the form of a Bayer sensor with at least two green pixels, one blue pixel and the red pixel R, which together form the pixel 150 . The red pixel R of the Bayer sensor has an R color filter element for transmitting red light and/or infrared light and at least least the NIR pixel NIR designed to transmit near-infrared light. According to one embodiment, the NIR pixel NIR is the blue pixel of the Bayer sensor, which has a B color filter element for transmitting blue light and infrared light and/or the green pixel of the Bayer sensor, which has a G color filter element for transmitting green light and Infrared light of the image sensor 115 has. In the form of a Bayer sensor, the image sensor 115 according to the alternative embodiment also has a filter device covering the pixels R, NIR of the image sensor 115 for transmitting red light and near-infrared light and/or for blocking green light and/or blue light. Thus, according to one embodiment, the green pixels and/or blue pixels of the Bayer sensor serve as pure near-infrared channels and the red pixel R of the Bayer sensor serves as a mixed channel for sensing a mixed value of red light and near-infrared light. According to this exemplary embodiment, reading device 120 is designed to read in R color signal 140, which represents such a mixed value of red light and near-infrared light, comparison device 125 being designed to compare the mixed value with a defined maximum mixing limit value in order to obtain a fourth comparison result obtained and/or compared with a defined minimum mixing limit value in order to obtain a fifth comparison result, wherein the recognition device 130 is designed to recognize the pixel 150 as a sturgeon plant image 107 or plant image 175 using the fourth comparison result and fifth comparison result.

According to one exemplary embodiment, reading device 120 is alternatively designed to read in a mixed light signal representing a sensed mixed value of red light and near-infrared light, which represents a signal provided by the red pixel R, wherein comparison device 125 is designed to compare the sensed mixed value with the defined maximum mixing limit value compare in order to obtain the fourth comparison result and/or compare it with the defined minimum mixing limit value in order to obtain the fifth comparison result, with the recognition device 130 being designed to identify the pixel 150 as a nuisance plant image 107 or plant image 175 using the fourth comparison result and fifth comparison results to recognize. According to one exemplary embodiment, the recognition device 130 is then designed to recognize the pixel 150 as a disturbing plant image 107 if the fourth comparison result indicates that the mixed value falls below the defined minimum mixed limit and/or exceeds the defined maximum mixed limit.

According to this exemplary embodiment, values for the defined minimum infrared limit value 160, defined maximum infrared limit value 164, defined maximum red limit value 168, defined maximum mixing limit value and/or defined minimum mixing limit value are stored in a memory device of device 105 or can be read in by device 105.

According to this exemplary embodiment, the recognition device 130 is also designed to apply a dilatation, in which further pixels adjacent to the pixel 150 are also recognized as a sturgeon plant image 107 if the pixel 150 was recognized as a sturgeon plant image 107 in order to recognize an extended sturgeon plant image. According to one exemplary embodiment, the plausibility check device 135135 is designed to remove the expanded sturgeon plant image from the raw camera image 110 . According to one exemplary embodiment, the recognition device 130 is designed to set a filter size for the dilation.

Aus der normierten Differenz aus beiden Spektralbereichen entsteht das NDVI-Bild. Wobei der Index von -1 bis 1 reicht. Ein Index von -1 ist sichere keine Pflanze, wenn der Index nahezu 1 wird, handelt es sich relativ sicher um eine Pflanze. Wobei der Grauwert jedes Bildpixels des NDVI-Bilds identisch ist zu seinem NDVI-Index.Plants reflect relatively little radiation in the red spectral range, the so-called "red channel", and relatively much radiation in the near infrared spectral range, the so-called "NIR channel". $$NDVI=\frac{NIR-ROt}{NIR+ROt}$$

The NDVI image is created from the normalized difference between the two spectral ranges. The index ranges from -1 to 1. An index of -1 is definitely not a plant, if the index is close to 1 it is almost certainly a plant. The gray value of each image pixel of the NDVI image is identical to its NDVI index.

In the image, by setting a threshold such as 0.3, the content in the NDVI image can be separated into "No Plants" and "Plants". This gives a classification of the plant portion in the image. In reality, there are many sources of interference that negatively affect the segmentation/detection of plants using NDVI. I.e. there are non-plant objects, but they are visible as plants in the NDVI image. Certain lighting conditions (under- and overexposure) also lead to areas in the NDVI image being incorrectly segmented as plants. A major disruptive factor is overexposure. That is, the gray values in the image reach their maximum value. The signal is cut off, so to speak. If the ratio of two image channels is now calculated in NDVI, the index is incorrect. In the worst case, the index ends up in exactly the same order of magnitude as the index of plants tended to is target. Thus, this leads to incorrect segmentation using NDVI.

The device 100 presented here improves the quality of a segmentation of plant parts using an NDVI image by detecting the described interference in the raw image data of the NDVI, red + NIR image channel, and, according to one embodiment, removing it from the image area from the outset, so that only real plant material is segmented. The device 105 thus advantageously enables an extension of a conventional NDVI segmentation, or in other words an extended NDVI algorithm.

After areas in the raw images, ie the red and NIR image, which lead to such disturbances in the NDVI have been identified, these are filtered out or removed by an NDVI plant segmentation mask according to one embodiment. This improves, so to speak, the quality of the detected plant material. Non-plant objects that were previously segmented and classified as a plant are now correctly classified by the device 105 as "non-plant" as well. For example, when herbicides are applied to plants at certain points, image data of fields are recorded with a very high optical resolution and plant parts are classified using the NDVI. The following disturbances can occur: noise, wet or damp soil, cracks in the soil, certain reflective properties of soil, rocks, straw and other dead plant debris, under- or overexposed scenes due to changing lighting conditions. Even segmentation errors due to imaging errors caused by the optics when the image is created, such as chromatic aberration, can be easily filtered out. After a further classification of the segmented plant part into crop and weeds, these areas are treated with herbicides depending on the degree of weed coverage. However, if there are many non-plant objects in the segmentation of plant parts, it would skew the decision to use herbicides.

In order to achieve a result that is as optimal as possible, the extended NDVI algorithm presented here is parameterized by a large number of image data according to one exemplary embodiment. The parameterization depends on the given image quality. This is essentially determined by the camera, lighting, sensor and optics.

• Im Bildverarbeitungsalgorithmus wird folgendermaßen vorgegangen. Es wird mithilfe von Grauwertschwellen 160, 164 entschieden, ob ein Pixel aufgrund seiner Grauwerte im Rot- und NIR-Kanal schon pauschal gar keine Pflanze sein kann oder sein sollte. Beispielsweise hat ein 12bit-Rohkamerabild 110 Grauwerte von 0 bis 4095. Die Grauwerte der Pixel von Pflanzen im NIR-Kanal sind immer höher als die im Rot-Kanal.

The following is an example of a procedure using device 105:

• The procedure in the image processing algorithm is as follows. Gray value thresholds 160, 164 are used to decide whether a pixel cannot or should not be a plant at all based on its gray values in the red and NIR channel. For example, a 12-bit raw camera image has 110 gray values from 0 to 4095. The gray values of plant pixels in the NIR channel are always higher than those in the red channel.

The first threshold, the defined minimum infrared limit 160, is the "underexposure in the NIR channel". That is, one decides that not all pixels in the raw camera image 110 can be plants, but instead reduces the possible plant pixels by all pixels in the NIR channel < "MinNIR" not being plants. As a result, image areas such as underexposure and shadows or shadow transitions at edges and wet/damp soil are generally assigned to "No Plant".

The second threshold, the defined maximum infrared limit 164, is the "overexposure in the NIR channel". That is, one decides that not all pixels in the raw camera image 110 can be plants, but instead reduces the possible plant pixels by all pixels in the NIR channel > "MaxNIR" not being plants. As a result, image areas such as overexposure and very brightly reflecting objects such as straw, stones, lens errors, which are noticeable in sharp edge transitions, are generally assigned to "No plant".

The third threshold, the defined maximum red limit 168, is "overexposure in the red channel". That is, one decides that not all pixels in the raw camera image 110 can be plants, but instead reduces the possible plant pixels by all pixels in the red channel > "MaxRot" not being plants. Because plants absorb red light, i. i.e. reflecting less red light, plants in the red channel appear very "dark", the idea is that all pixels in the red channel that have a higher gray value than "MaxRot" cannot be plants. As a result, image areas such as overexposure and very brightly reflecting objects such as straw, stones, lens errors, which are noticeable in sharp edge transitions, are generally assigned to "No plant".

A threshold for "underexposure in the red channel" does not make sense because, as already mentioned, plants in the red channel have very low gray values anyway. A lower gray value threshold for removing interference areas makes no sense at this point, because otherwise there is a great danger of removing real plant areas during segmentation in the image. The parameters for the "MinNIR", "MaxNIR" and "MaxRot" thresholds can be adapted to the image quality in an application-specific manner. Because depending on the quality, depending on the camera and lens and lighting, the interference is sometimes greater and sometimes less.

If, according to an alternative exemplary embodiment, a camera setting with a Bayer sensor is used, two additional thresholds are required, the previously described defined maximum mixing limit and defined minimum mixing limit. Since the raw image channel for infrared of a Bayer sensor contains a red and infrared signal as a mixed signal, there is also a threshold for underexposure and overexposure for this channel (red+NIR). The functionality is identical to the first and second threshold for underexposure and overexposure in the NIR channel. The parameters for the thresholds "MinRotNIR" and "MaxRotNIR" can, like the parameters above, be adapted to the image quality and camera setting specific to the application.

In addition, according to an exemplary embodiment, the device 105 draws a safety area around the interfering plant images 107 in the form of interference areas that have been found. This extended image area is also by definition "not a plant". Due to the defined thresholds for under- and overexposure, it is sometimes not possible to remove all of the interfering objects as a whole. The approach is improved to the effect that many of these disturbances in the image locally extend over several pixels and have a gray value progression that does not always match the defined thresholds from under- and overexposure. In the image processing algorithm, according to one exemplary embodiment, a dilatation in the form of a morphological operation is added after each threshold for the underexposure and overexposure. This leads to the behavior described above. The dilation is parameterized per threshold and desired behavior according to one embodiment. i.e. if the image has already been relatively well segmented "only plants", under certain circumstances no dilatation is required or only with a small filter size. If, on the other hand, there are many interfering objects even after removal due to underexposure and overexposure, the filter size of the dilation per threshold is increased according to one exemplary embodiment.

An image of a piece of straw, for example, would be segmented as a plant in the NDVI without the device 105 presented here. The straw, on the other hand, is recognized as "not a plant" by the extended NDVI algorithm presented here in the R+NIR channel in the case of "overexposure". However, since the straw has a gray value progression from very light to less light, overexposure without dilatation would not segment the entire disruptive object, but only a very light inner core of the object. Therefore, in such an embodiment, in the algorithm for this camera setting, a dilation with a filter of an exemplary size 5 pixels in diameter is applied to this segmentation. The result is an expanded object that also includes a surrounding less bright area. This ensures that the entire straw is found as a disruptive object over the entire extent and is removed from the possible plant area from the outset.

According to one exemplary embodiment, all recognized disturbing plant images 107, for example all straw that can potentially negatively influence plant segmentation, are defined as “no plant” from the outset by means of a mask. After all interference areas found from all image channels have been segmented and combined, according to one exemplary embodiment, plant segmentation is no longer carried out in these image areas in the NDVI image, since these areas were previously defined as "no plants" by the segmentations (interfering objects).

As a boundary condition, it is important in the device 105 to use the NDVI with the raw image channels in the form of the red and NIR channels in the image processing algorithm in order to be able to use the described extension of the NDVI algorithm. The described method that can be carried out by the device 105 can be used to detect plants, for example for the selective dispensing of herbicides.

2 FIG. 2 shows a schematic representation of a preliminary NDVI image 200 generated using an apparatus according to an embodiment. It can be based on 1 act described device. According to this exemplary embodiment, both plant images 175 and sturgeon plant images 107 or extended sturgeon plant images are imaged in the provisional NDVI image 200 . According to this exemplary embodiment, four recognized sturgeon plant images 107/expanded sturgeon plant images and eight plant images 175 are shown in the preliminary NDVI image 200. The preliminary NDVI image 200 shown here represents a common NDVI image.

3 17 shows a schematic representation of an NDVI image 177 generated using an apparatus according to an embodiment. It can be based on 1 act described device. Opposite the inside 2 In the preliminary NDVI image described above, the sturgeon plants images/extended sturgeon plants images have been removed to improve the NDVI image 177 according to this embodiment.

4 FIG. 4 shows a flowchart of a method 400 for detecting a sturgeon plant image in a raw camera image according to an embodiment. This can be a method 400 that can be controlled or executed using the device presented in one of the preceding figures.

The method 400 has a step 405 of reading in, a step 410 of comparing and a step 415 of recognition. In step 405 of reading in, at least one R color signal representing a sensed red value, which represents a signal provided by a red pixel of a pixel of an image sensor, and one NIR color signal representing a sensed infrared gray value, which represents a signal provided by an NIR pixel of the pixel Signal represented, read. In step 410 of comparison, the sensed infrared gray value is compared with a defined minimum infrared limit value in order to obtain a first comparison result, the sensed infrared gray value is compared with a defined maximum infrared limit value in order to obtain a second comparison result, and the sensed red value is compared with a defined maximum red limit value, to obtain a third comparison result. In step 415 of recognition, the pixel is recognized as a disturbing plant image or plant image using the first comparison result, second comparison result and third comparison result. In this way, according to one embodiment, an NDVI image is generated.

Optionally, the method 400 according to this exemplary embodiment also has a step 420 of applying a dilatation, a step 425 of removal and/or a step 430 of plausibility checking. As a result, the generated NDVI image can be improved, i.e. it can be cleaned of images of sturgeon plants, for example.

In step 420 of applying a dilation, a dilation is applied in which further pixels adjacent to the pixel are also recognized as a sturgeon plant image if the pixel was recognized as a sturgeon plant image in order to recognize an extended sturgeon plant image. In step 425 of removal, the pixel recognized as a sturgeon plant image is removed from the raw camera image in order to improve the already existing NDVI image. In step 430 of the plausibility check, the pixel identified as a plant image is used to check the NDVI image for plausibility.

The method steps presented here can be repeated and carried out in a different order than the one described.

5 10 shows a block diagram of a device 105 according to an embodiment. This can be the in 1 act described device 105, which is designed to implement the NDVI algorithm. According to this exemplary embodiment, a first block 500 represents an input image in the NIR channel, which is checked with regard to the defined minimum infrared limit value 160 in the form of the “underexposure NIR channel” threshold and with regard to the defined maximum infrared limit value 164 in the form of the “overexposure NIR channel” threshold. A dilatation then optionally takes place in a second block 505 after the “SizeMinNIR” parameter and in a third block 510 after the “SizeMaxNIR” parameter. According to this exemplary embodiment, a fourth block 515 represents an input image in the red channel, which is checked with regard to the defined maximum red limit value 168 in the form of the “overexposure red channel” threshold. Optionally, a dilatation then takes place in a fifth block 520 after the “SizeMaxRot” parameter. According to this exemplary embodiment, an optional sixth block 525 represents an input image in the red+NIR channel which, with regard to the defined minimum mixing limit value 530 in the form of the “underexposure red+NIR channel” threshold and the defined maximum mixing limit value 535 in the form of the “overexposure red+NIR channel” is checked. A dilatation then optionally takes place in a seventh block 540 after the “SizeMinRotNIR” parameter and in an eighth block 545 after the “SizeMaxRotNIR” parameter. A ninth block 550 indicates improved plant segmentation using NDVI.

Claims (12)

Method (400) for detecting a sturgeon plant image (107) in a raw camera image (110), the method (400) having the following steps: reading in (405) at least one R color signal (140) representing a sensed red value, which is one of a signal provided by a red pixel (R) of a pixel (150) of an image sensor (115), and a NIR color signal (155) representing a sensed infrared gray value representing a signal provided by a NIR pixel (NIR) of the pixel; Comparing (410) the sensed infrared gray value with a defined minimum infrared limit value (160) in order to obtain a first comparison result (162), the sensed infrared gray value with a defined maximum infrared limit value (164) in order to obtain a second comparison result (166), and the sensed red value with a defined maximum red limit value (168) to obtain a third comparison result (170); and identifying (415) the pixel (150) as a disturbance plant zen image (107) or plant image (175), using the first comparison result (162), second comparison result (166) and third comparison result (170). Method (400) according to claim 1 , with a step (425) of removal, in which the image point (150) recognized as a sturgeon plant image (107) is removed from the raw camera image (110). Method (400) according to one of the preceding claims, in which in the step (415) of recognition the pixel (150) is recognized as a sturgeon plant image (107) if the first comparison result (162) indicates that the sensed infrared gray value exceeds the defined minimum infrared limit value ( 160) and/or the second comparison result (166) indicates that the infrared gray value exceeds the defined maximum infrared limit value (164) and/or the third comparison result (170) indicates that the sensed red value exceeds the defined maximum red limit value (168). Method (400) according to one of the preceding claims, in which in the step (415) of recognizing the pixel (150) is recognized as a plant image (175) if the first comparison result (162) indicates that the sensed infrared gray value exceeds the defined minimum infrared limit value ( 160) and/or the second comparison result (166) indicates that the infrared gray value falls below the defined maximum infrared limit value (164) and/or the third comparison result (170) indicates that the sensed red value falls below the defined maximum red limit value (168). Method (400) according to one of the preceding claims, with a plausibility check step (430), in which the pixel (150) recognized as a plant image (175) is used to check a region of an NDVI image (177) for plausibility. Method (400) according to one of the preceding claims, with a step (420) of applying a dilation in which further pixels adjacent to the pixel (150) are also recognized as a sturgeon plant image (107) if the pixel (150) as a sturgeon plant image ( 107) was detected to detect extended sturgeon plant imaging. Method (400) according to claim 6 , in which in the applying step (420) a filter size is set for the . Method (400) according to one of the preceding claims, in which in step (405) of reading in the NIR color signal (155) is read in from the NIR pixel (NIR), which is a blue pixel and/or green pixel of the image sensor (115). . Method (400) according to one of the preceding claims, in which in step (405) of reading in the R color signal (140) is read in, which represents a mixed value of red light and near-infrared light, wherein in step (410) of comparing the mixed value with is compared to a defined maximum mixing limit value (535) in order to obtain a fourth comparison result and/or is compared to a defined minimum mixing limit value (530) in order to obtain a fifth comparison result, the pixel (150) as a sturgeon plant image in step (415) of recognizing (107) or plant image (175) is recognized using the fourth comparison result and fifth comparison result. Method (400) according to any one of Claims 1 until 8th , in which, in the step (405) of reading in, a mixed light signal representing a sensed mixed value of red light and near-infrared light is also read in, which signal represents a signal provided by the red pixel (R), wherein in step (410) of comparing the sensed mixed value with a is compared with a defined maximum mixing limit value (535) in order to obtain a fourth comparison result and/or is compared with a defined minimum mixing limit value (530) in order to obtain a fifth comparison result, with the step (415) of recognizing the pixel (150) as a disturbing plant image ( 107) or plant image (175) is recognized using the fourth comparison result and fifth comparison result. Device (105) set up to carry out the steps (405, 410, 415, 420, 425, 430) of the method (400) according to one of the preceding claims in corresponding units (120, 125, 130, 135) and/ or to control. Image processing device (100) with a device (105) according to claim 11 and the image sensor (115).

Images