WO2023072464A1 - Procédé de détection de salissure sur une unité de lentille d'une caméra d'une machine de travail agricole - Google Patents

Procédé de détection de salissure sur une unité de lentille d'une caméra d'une machine de travail agricole Download PDF

Info

Publication number
WO2023072464A1
WO2023072464A1 PCT/EP2022/074795 EP2022074795W WO2023072464A1 WO 2023072464 A1 WO2023072464 A1 WO 2023072464A1 EP 2022074795 W EP2022074795 W EP 2022074795W WO 2023072464 A1 WO2023072464 A1 WO 2023072464A1
Authority
WO
WIPO (PCT)
Prior art keywords
cells
contamination
unit
images
image
Prior art date
Application number
PCT/EP2022/074795
Other languages
German (de)
English (en)
Inventor
Farid Khani
Andreas Weimer
Original Assignee
Robert Bosch Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch Gmbh filed Critical Robert Bosch Gmbh
Publication of WO2023072464A1 publication Critical patent/WO2023072464A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30181—Earth observation
    • G06T2207/30188—Vegetation; Agriculture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the invention is based on a method for identifying soiling on a lens unit of an optical detection unit, in particular a camera, preferably an agricultural working machine and a corresponding computing unit, an optical detection unit and an agricultural working machine according to the species of the independent claims.
  • the subject matter of the present invention is also a computer program and a machine-readable storage medium
  • Agricultural machines are in a harsh environment with a lot of dirt. Dirt on the camera lens or on the cover glass of the camera lens blocks the incoming light. This is imaged as a fuzzy point on the sensor, resulting in a dark spot in the image. However, the scene is always adequately illuminated by the sun and the LEDs. If a camera lens becomes soiled by clods of earth, the camera must detect and report this, since a dirty camera lens is partially or fully blind and therefore cannot see objects.
  • DE 10 2016 118 335 A1 discloses a method for detecting dirt on a lens of a camera of a motor vehicle.
  • DE 10 2018 209 020 A1 describes a device which is designed to detect soiling of a light-permeable cover of at least one transmission window and/or one reception window of an optical sensor.
  • the subject matter of the present invention is a method for identifying soiling on a lens unit of an optical detection unit, in particular a camera, preferably an agricultural working machine, with the steps:
  • the subject matter of the present invention is also a computing unit which is set up to carry out and/or control the steps of a previously described method.
  • the subject matter of the present invention is also an optical detection unit, in particular a camera for detecting a field section of an agricultural area in order to capture images of field sections, with a lens unit and a computing unit described above.
  • the subject matter of the present invention is also an agricultural working machine, in particular a field sprayer, with an agricultural working tool, in particular a spraying device, and an optical detection unit as described above.
  • the present invention also relates to a computer program that is set up to carry out and/or control the steps of a method described above when the computer program is executed on a computer, and a machine-readable storage medium on which the computer program is stored.
  • the method according to the invention now makes it possible to carry out soiling on a lens unit of an optical detection unit or camera in a very simple manner with very little computing power. Furthermore, the method can be carried out using series of images, so that no still image is necessary.
  • the optical detection unit is preferably a camera.
  • the lens unit comprises the lens and, if necessary, the cover glass or protective glass.
  • the received images are preferably a defined selection of an image series captured directly one after the other by the optical capture unit.
  • the defined selection can include all images of the image series.
  • the defined selection can also only include a defined number of images of the image series captured immediately one after the other. Depending on how quickly you have to react to the contamination and, if necessary, in order to save computing time, the defined number can also include only every xth image, for example every 5th image of the image series recorded.
  • the images can be images of different field sections or of the same field section (when the optical detection unit is stationary).
  • the image cells are preferably picture elements or pixels. However, the image cells can also have a higher resolution and thus comprise two or more pixels.
  • the image cells or pixels that fall below a defined gray threshold value are segmented in order to identify potential contamination cells in the received images.
  • the defined gray threshold value can, for example, be less than or equal to 100 out of 12 bits, i.e. out of 4096 gray values.
  • the segmented image cells are themselves the potential contamination cells.
  • the images are overlaid with a grid whose grid cells comprise at least two, in particular a large number of, image cells, the segmented image cells being linked to the grid and grid cells with a degree of coverage greater than or equal to a defined coverage threshold value as the pollution cells are identified.
  • the defined coverage threshold can be 50%, for example, so that grid cells that are at least 50% covered with segmented image cells are identified as the contamination cells.
  • the steps of receiving the images and segmenting the image cells are preferably carried out successively and repeatedly.
  • an image is received and the image cells in it are segmented and the steps are then repeated one after the other for all other received images.
  • the matching contamination cells which are the same in the received images, are determined using the processing unit in order to identify them (using the processing unit) as contamination on the lens unit.
  • the identified potential contamination cells are compared with each other across the received images, and the potential contamination cells identified in all images are identified as actual contamination on the lens unit.
  • the segmented image cells which have not changed or have remained segmented in the received images or the image series are identified as soiling.
  • the matching contamination cells are preferably identified as contamination only above a defined contamination cell threshold value (by means of the computing unit).
  • the defined pollution cell threshold may be, for example, 5000 pollution cells.
  • An agricultural area can be understood to mean an area used for agriculture, an area under cultivation for plants or also a parcel of such an area or area under cultivation.
  • the agricultural area can thus be arable land, grassland or pasture.
  • the field section can be a detection section or a detected image section of the optical detection unit.
  • the method can include a step of capturing the field sections of the agricultural area by means of the optical capturing unit.
  • the step of detecting can be carried out during a crossing or a flight of the agricultural working machine.
  • At least one further step of the method, in particular all steps of the method, can be carried out during a crossing or a flight of the agricultural working machine.
  • the agricultural machine can include a mobile unit or be arranged on a mobile unit, the mobile Unit can be designed in particular as a land vehicle and / or aircraft and / or trailer.
  • the mobile unit can also be a self-propelled or autonomous robot.
  • the agricultural working machine is preferably a weed regulating machine, in particular a field sprayer.
  • the agricultural working tool is preferably a spray device, but can also be a mechanical tool for weed control.
  • the method includes a step of outputting a signal, in particular a control signal for activating a cleaning device, in particular the optical detection unit, and/or an information signal, depending on the identified contamination on the lens unit.
  • the information signal can in particular be an optical and/or acoustic signal, for example to signal the contamination to a driver or operator of the agricultural working machine.
  • the arithmetic unit is or the arithmetic units are designed or set up for image processing, so that it can carry out calculation steps or image processing steps for carrying out the method according to the invention. Accordingly, each computing unit has corresponding image processing software.
  • the arithmetic unit can be a signal processor, a microcontroller or the like, for example, while the storage unit can be a flash memory, an EPROM or a magnetic storage unit.
  • the communication interface can be designed to read in or output data wirelessly and/or by wire, with a communication interface that can read in or output wire-bound data reading this data, for example electrically or optically, from a corresponding data transmission line or outputting it into a corresponding data transmission line.
  • the method according to the invention can be implemented, for example, in software or hardware or in a mixed form of software and hardware in the processing unit or a control unit.
  • the processing unit can be arranged completely or partially on the optical detection unit or the agricultural working machine or be integrated into them.
  • the computing unit can also be completely or partially external, for example integrated in a cloud.
  • the arithmetic unit can thus also be divided among different units, for example mobile and stationary units.
  • a computer program product or computer program with program code which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and for carrying out, implementing and/or controlling the steps of the method according to one of the embodiments described above, is also advantageous used, especially when the program product or program is run on a computer or device.
  • Fig. 1 is designed as a dirty camera optical
  • FIG. 2 shows an image captured by the dirty camera from FIG. 1;
  • 3a-c shows an image series of received images with identified potential contamination cells according to an embodiment
  • FIG. 5 shows the image from FIG. 2 with a superimposed grid
  • FIG. 8 shows a flow chart of a method according to a
  • the optical detection unit 10 is designed as a camera 10 .
  • the optical detection unit 10 has a lens unit 12 . Dirt 14 adheres to or on the lens unit 12.
  • FIG. 2 shows an image 16 of a field section 18 of an agricultural area captured by means of the optical capture unit 10 or camera 10 from FIG. 1 .
  • the image 16 has a dark section 20 or spot 20 due to the dirt 14 on the lens unit 12 .
  • Fig. 3a-c an image series of received images 16a-c is shown.
  • image cells 22a-c or image pixels 22a-c which fall below a defined gray threshold value are segmented in the received images 16a-c by means of a computing unit (not shown).
  • the segmented image cells 22a-c are each identified as potential contamination cells 24a-c in the received images 16a-c of the image series.
  • Fig. 4 now shows the corresponding contamination cells 26, which are the same in all images 16a-c.
  • the dirt cells 26 are then identified as dirt 14 on the lens unit 12 by means of the computing unit.
  • the matching contamination cells 26 can also only be identified as contamination 14 from a defined contamination cell threshold value, for example from 5000 contamination cells 26, which is the case here.
  • FIGS. 5 to 7 The sequence of a further embodiment of the method is shown in FIGS. 5 to 7 .
  • FIG. 5 shows the captured image 16 from FIG. 2
  • FIGS. 6a-c show the series of images 16a-c from FIGS 28 are superimposed, whose grid cells 30 each comprise 100 image cells or image pixels.
  • the grid 28 is linked to the respective segmented image cells 22a-c, grid cells 30 with a degree of coverage of greater than or equal to 50% of segmented image cells 22a-c being identified as potential contamination cells 24a-c. This leads to a lower resolution, which, however, can save computing time.
  • FIG. 7 now shows the corresponding contamination cells 26, which are the same in all images 16a-c.
  • the dirt cells 26 are then identified as dirt 14 on the lens unit 12 by means of the computing unit.
  • the matching contamination cells 26 can also only be identified as contamination 14 from a defined contamination cell threshold value, for example from 5 contamination cells 26, which is the case here.
  • FIG. 8 shows a flowchart of an exemplary embodiment of the approach presented here as a method 100 for identifying soiling 14 on a lens unit 12 of an optical detection unit 10, in particular a camera 10, preferably an agricultural working machine.
  • the method 100 comprises a step of receiving 102 images 16, 16a-c, in particular of field sections 18 of an agricultural area, which were recorded by means of the optical detection unit 10, in particular camera 10, preferably the agricultural working machine.
  • the method 100 also includes a step of segmenting 104 image cells 22a-c, which fall below a defined gray threshold value, of the received images 16, 16a-c by means of a computing unit in order to identify potential contamination cells 24a-c in the received images 16, 16a-c to identify.
  • the method 100 includes also a step of determining 106 the matching contamination cells 26, which are the same in the received images 16, 16a-c, by means of the computing unit in order to identify them as contamination 14 on the lens unit 12.
  • the method 100 comprises an optional step of outputting 108 a signal, in particular a control signal for controlling a cleaning device, in particular the optical detection unit 10, and/or an information signal, depending on the identified contamination 14 on the lens unit 12.
  • a signal in particular a control signal for controlling a cleaning device, in particular the optical detection unit 10, and/or an information signal, depending on the identified contamination 14 on the lens unit 12.

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

L'invention concerne un procédé d'identification de salissure sur une unité de lentille d'une unité de détection optique, en particulier d'une caméra, de préférence d'une machine de travail agricole, comprenant les étapes consistant à : - recevoir des images (16a), en particulier de sections de champ (18) d'une zone agricole qui ont été capturées par l'unité de détection optique, en particulier une caméra de la machine de travail agricole ; - segmenter des cellules d'image (22a) des images capturées (16a) qui sont au-dessous d'un seuil de gris défini au moyen d'une unité de calcul afin d'identifier les cellules souillées potentielles (24a) dans les images capturées (16a) ; et - déterminer les cellules souillées correspondantes qui sont les mêmes dans les images capturées (16) au moyen de l'unité de calcul afin d'identifier lesdites cellules en tant que salissures sur l'unité de lentille.
PCT/EP2022/074795 2021-10-26 2022-09-07 Procédé de détection de salissure sur une unité de lentille d'une caméra d'une machine de travail agricole WO2023072464A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102021212020.2A DE102021212020A1 (de) 2021-10-26 2021-10-26 Verfahren zum Erkennen einer Verschmutzung an einer Linseneinheit einer Kamera einer landwirtschaftlichen Arbeitsmaschine
DE102021212020.2 2021-10-26

Publications (1)

Publication Number Publication Date
WO2023072464A1 true WO2023072464A1 (fr) 2023-05-04

Family

ID=83400593

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2022/074795 WO2023072464A1 (fr) 2021-10-26 2022-09-07 Procédé de détection de salissure sur une unité de lentille d'une caméra d'une machine de travail agricole

Country Status (2)

Country Link
DE (1) DE102021212020A1 (fr)
WO (1) WO2023072464A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117788464A (zh) * 2024-02-26 2024-03-29 卡松科技股份有限公司 一种工业齿轮油杂质视觉检测方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4245452B2 (ja) * 2003-10-06 2009-03-25 富士通株式会社 レンズの汚れ判定方法及び装置
US20100302398A1 (en) * 2009-03-03 2010-12-02 Samsung Digital Imaging Co., Ltd. Digital camera dust error photographed image correction
DE102016118335A1 (de) 2016-09-28 2018-03-29 Connaught Electronics Ltd. Verfahren zum Erkennen einer Verschmutzung auf einer Linse einer Kamera eines Kraftfahrzeugs anhand von Helligkeitswerten in Bildern, Kamera sowie Kraftfahrzeug
DE102018209020A1 (de) 2018-06-07 2019-12-12 Robert Bosch Gmbh Vorrichtung, ausgebildet zur Erkennung einer Verschmutzung wenigstens eines Sendefensters und/oder eines Empfangsfensters eines Sensors
CN108629886B (zh) * 2017-03-17 2020-06-09 深圳怡化电脑股份有限公司 一种纸币污损等级的检测方法及装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010038223A1 (fr) 2008-10-01 2010-04-08 Hi-Key Limited Procédé et système de détection de la présence sur une lentille d’un dispositif de capture d’images d’une obstruction au passage de la lumière à travers la lentille du dispositif de capture d’images
DE102012015282B4 (de) 2012-08-01 2023-03-16 Application Solutions (Electronics and Vision) Ltd. Verfahren zur Detektion eines verdeckten Zustands einer Bilderfassungseinrichtung eines Kraftfahrzeugs, Kamerasystem und Kraftfahrzeug

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4245452B2 (ja) * 2003-10-06 2009-03-25 富士通株式会社 レンズの汚れ判定方法及び装置
US20100302398A1 (en) * 2009-03-03 2010-12-02 Samsung Digital Imaging Co., Ltd. Digital camera dust error photographed image correction
DE102016118335A1 (de) 2016-09-28 2018-03-29 Connaught Electronics Ltd. Verfahren zum Erkennen einer Verschmutzung auf einer Linse einer Kamera eines Kraftfahrzeugs anhand von Helligkeitswerten in Bildern, Kamera sowie Kraftfahrzeug
CN108629886B (zh) * 2017-03-17 2020-06-09 深圳怡化电脑股份有限公司 一种纸币污损等级的检测方法及装置
DE102018209020A1 (de) 2018-06-07 2019-12-12 Robert Bosch Gmbh Vorrichtung, ausgebildet zur Erkennung einer Verschmutzung wenigstens eines Sendefensters und/oder eines Empfangsfensters eines Sensors

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117788464A (zh) * 2024-02-26 2024-03-29 卡松科技股份有限公司 一种工业齿轮油杂质视觉检测方法
CN117788464B (zh) * 2024-02-26 2024-04-30 卡松科技股份有限公司 一种工业齿轮油杂质视觉检测方法

Also Published As

Publication number Publication date
DE102021212020A1 (de) 2023-04-27

Similar Documents

Publication Publication Date Title
EP2286653B1 (fr) Plateforme mobile autonome destinée au traitement de surface
EP3291172B1 (fr) Procédé de détermination de données d'image de résultat
DE102016118227A1 (de) Bildanlysesystem für landwirtschaftliche Arbeitsmaschinen
DE102005050302B4 (de) Verfahren und Vorrichtung zum berührungslosen Ermitteln des aktuellen Ernährungszustandes eines Pflanzenbestandes und zum Verarbeiten dieser Informationen zu Düngeempfehlungen
DE102016118237A1 (de) Landwirtschaftliche Arbeitsmaschine mit Bildanalysesystem
DE102011106072A1 (de) Schattenentfernung in einem durch eine fahrzeugbasierte kamera erfassten bild unter verwendung einer optimierten ausgerichteten linearen achse
DE102008060141A1 (de) Verfahren zur Vermessung des Wachstums von Blattscheiben sowie eine dazu geeignete Vorrichtung
EP2034461A2 (fr) Procédé de détection et/ou de suivi d'objets mobiles dans une scène de surveillance présentant des perturbateurs, dispositif et programme informatique
WO2023072464A1 (fr) Procédé de détection de salissure sur une unité de lentille d'une caméra d'une machine de travail agricole
DE102019218192A1 (de) Verfahren zum Bearbeiten von Pflanzen auf einem Feld
EP2854506A1 (fr) Procédé de détermination sans contact de l'état nutritionnel d'un peuplement végétal et de traitement de cette information
WO2021105019A1 (fr) Procédé de génération d'une pluralité d'images annotées
WO2021105017A1 (fr) Procédé de traitement de plantes dans un champ
DE102017217258A1 (de) Verfahren zum Klassifizieren von Pflanzen
DE102010006146A1 (de) System und Verfahren zum Durchführen von optischer Navigation unter Verwendung von verbesserten Frames von Bilddaten
DE102012200930A1 (de) Vorrichtung und Verfahren zum Erfassen einer Pflanze vor einem Hintergrund
DE102019219734B4 (de) Verfahren zum Betreiben eines Auswertungssystems für Messdaten aus mehreren Domänen
WO2021105015A1 (fr) Procédé de traitement de plantes dans un champ, de génération de données d'apprentissage et d'entraînement d'un réseau de neurones
WO2022122240A1 (fr) Procédé et dispositif de détection d'une fausse image de plante dans une photo de caméra brute, et appareil de traitement de photo
EP4064818B1 (fr) Procédé de traitement de plantes dans un champ
DE102019102672A1 (de) Intersensorisches lernen
DE10230200A1 (de) Verfahren und Vorrichtung zur Detektion von auf einer Oberfläche eines transparenten Elements sich befindenden Objekten
DE102019133515B3 (de) Verfahren und Vorrichtung zur Parallaxenbestimmung von Aufnahmen eines Multilinsen-Kamerasystems
DE102020121857A1 (de) Landwirtschaftliche Arbeitsmaschine
WO2021180547A1 (fr) Procédé et dispositif de traitement d'images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22773656

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22773656

Country of ref document: EP

Kind code of ref document: A1