DE102019207245A1 - Agricultural land classification - Google Patents

Abstract

The invention describes a method and a system (100) for providing a classifier (K) for an agricultural area (2). Area information (6, 8) relating to the agricultural area (2) is recorded in an air-supported manner by area detection sensors (22) attached to an unmanned missile (20). A classifier (K) for assigning area information (6, 8) is defined and trained with the area information (6, 8) recorded by air.

Description

The invention relates to a method and a system for providing a classifier for an agricultural area.

It is known to arrange a positioning sensor on an agricultural vehicle, with which a current position of the agricultural vehicle on the agricultural area can be detected. It is also known that an agricultural vehicle can be navigated at least partially automatically on a usable agricultural area based on such a position detection. In such a navigation, a temporary detection gap can occur in the position detection, in which the agricultural vehicle can no longer be navigated based on the position detection.

In one aspect, the invention relates to a method for providing a classifier for an agricultural area. The classifier can be provided for localizing an agricultural vehicle on the agricultural area. The method can thus also be carried out to localize an agricultural vehicle on the agricultural area.

The classifier can have at least one of an assignment logic, an assignment function and an assignment rule for assigning acquired area information to at least one area information class. The classifier can therefore function to classify or classify acquired area information into at least one area information class. The classifier can be a generic or universal area information classifier for assigning agricultural area information. In this way, the classifier can be used to assign area information across the usable area. The classifier can thus also function to assign area information from several or different agricultural areas.

The method has, as one step, an air-supported acquisition of area information on the agricultural area using an area acquisition sensor system mounted on an unmanned missile. The area information can thus be area information acquired airborne.

Acquisition of area information on the agricultural area can include acquisition of measurement data on at least one of a terrain surface of the agricultural area, vegetation on the agricultural area, a structure on the agricultural area, an infrastructure object on the agricultural area or a vehicle on the agricultural area exhibit. Area information on the agricultural area can also be recorded with the aid of a vehicle within the scope of the invention. Area information on the agricultural area can therefore also be recorded in the context of the invention with the aid of air and vehicle.

The unmanned missile can be an unmanned aerial vehicle (UAV). The unmanned missile can be remotely controlled by an operator of the unmanned missile for the purpose of airborne acquisition of area information. The unmanned missile can fly autonomously over the agricultural area in order to record area information on the agricultural area with the aid of the air. The unmanned missile can also be a so-called “drone”. The unmanned missile can be a multicopter, which can be a quadrocopter, for example.

The airborne acquisition of area information can be carried out starting from the unmanned missile. For this purpose, the area detection sensor system can be arranged on the unmanned missile or integrated therein. For this purpose, the area detection sensor system can also be carried by the unmanned missile. The air-assisted acquisition of area information can therefore include acquisition of area information from the air. For this purpose, the unmanned missile can function as a flying acquisition platform for airborne acquisition of area information. The air-based acquisition is therefore not a ground-based or satellite-based acquisition of area information.

The air-supported detection of area information can include georeferencing of the air-supported area information. For this purpose, the area information recorded in the air can be transformed into a predetermined coordinate system. The coordinate system can be a global or superordinate coordinate system. The coordinate system can also be a space-related or vehicle-related coordinate system.

The unmanned missile can have a position detection sensor system for detecting a position of the unmanned missile during Have airborne detection of area information. The position detection sensor system can have a GNSS antenna with which a position of the unmanned missile can be detected. The air-supported area information or corresponding raw measurement data can be transformed into a coordinate system based on such a position detection. The air-supported area information or corresponding raw measurement data can also be transformed into the coordinate system based on an orientation detection or based on extrinsic parameters of the unmanned missile. Furthermore, the area information captured by air can be transformed into a geographic information system (GIS) or into a map. Georeferencing or transforming can be carried out on the unmanned missile (online) or in a data center (offline).

The area detection sensor system can have a distance-measuring sensor. Alternatively or additionally, the sensor system can have an image-capturing sensor. In the context of the invention, the area detection sensor system can be an air-supported or vehicle-supported sensor system.

The distance-measuring sensor can also be designed as a scanning sensor. The distance-measuring sensor can be, for example, a laser scanner, a radar measuring device or an ultrasonic measuring device. The laser scanner, the radar measuring device or the ultrasonic measuring device can be designed as a two-dimensional or three-dimensional scanning sensor. The distance-measuring sensor can also be a distance-measuring 3D camera. If the sensor system has a distance-measuring sensor, the agricultural area can be recorded selectively or discretely scanned. With the distance measuring sensor, a point cloud with measuring points can be recorded on the agricultural area. The point cloud can be a two-dimensional or three-dimensional point cloud.

The image capturing sensor can be designed as a camera. The camera can be an RGB camera or a multispectral camera. The RGB camera and the multispectral camera can each be a camera which can capture light in several spectral channels. The multispectral camera can also be used to determine a normalized differentiated vegetation index (NDVI index) of an area of the agricultural area, based on which an area object or a usable area boundary can be recognized. The spectral channels of the multispectral camera can have at least one spectral channel in which light can be detected in a spectral band that is invisible to humans. The spectral band, which is invisible to humans, can be light outside a wavelength range from 400 nm to 750 nm. A spectral band that is not visible to humans can at least partially comprise infrared radiation, for example near infrared light, medium infrared light and / or thermal infrared light. If the sensor system has an image-capturing sensor as an alternative or in addition to a distance-measuring sensor, an image of the agricultural area can be captured or recorded. The image can have image information in at least one visible spectral range. Alternatively or additionally, the image can have image information in at least one invisible spectral range, for example in an infrared spectral range.

As a further step, the method has a definition of a classifier for assigning area information to at least one area information class. The classifier can be a neural network. The classifier can also be a binary or a linear classifier. The classifier can function for the binary or linear assignment of area information to at least one area information class. The classifier can thus be a corresponding agricultural land classifier.

The assignment of area information can include a classification of area information into the at least one area information class, wherein the classification can include establishing class boundaries for the at least one area information class. The assignment of area information can be or include a classification of area information into the at least one area information class, wherein the classification can include sorting or dividing the at least one area information class into the at least one area information class. The assigning, the classifying, the classifying, the sorting or the dividing can be carried out in an unambiguous way or in an ambiguous way or optionally carried out.

According to a further-developing embodiment of the method, area information relating to the agricultural area is formed in an area information class. The area information relating to the agricultural area can have area information relating to an interior of the agricultural area. The interior can be an area within the have agricultural area or within an agricultural area boundary. The area information relating to the agricultural area can have area information relating to an agricultural area boundary.

According to a further developing embodiment of the method, an area information class that does not relate to the agricultural area is formed. The area information that does not relate to the agricultural area can form an area information class as an alternative or in addition to the area information related to the agricultural area. The area information that does not relate to the agricultural area can have area information related to an exterior of the agricultural area. The exterior can have an area outside the agricultural area or outside an agricultural area boundary. The area information that does not relate to the agricultural area can have area information related to an agricultural area boundary.

According to a further developing embodiment of the method, area information relating to an agricultural usable area boundary is formed in an area information class. The area information relating to the agricultural area boundary can form an area information class as an alternative or in addition to the area information classes described further. A recognized area object or at least one feature of this can, as described, also be assigned to the area information relating to the agricultural area boundary. The agricultural vehicle can thus be located in an advantageous manner inside or outside of an agricultural area boundary in order to prevent the agricultural vehicle from leaving the agricultural area, for example when it is navigating automatically.

The area information relating to the agricultural area boundary can have a height difference, for example formed by a ditch, a wall, or a fence on the agricultural area. The area information relating to the agricultural area boundary can also have a vegetation boundary or different vegetation. The area information relating to the agricultural area boundary can also have vegetation, for example a hedge, or at least one tree. The agricultural land boundary can therefore be a natural or actual boundary of use and therefore not an official parcel boundary. The usable area boundary can have a straight boundary section. As an alternative or in addition to this, the usable area boundary can have a curved boundary section.

The method has, as a further step, training of the classifier with the area information recorded in the air. The training of the classifier can be based on the air-supported acquisition of area information on the agricultural area or it can depend on this. The training step can be carried out at least partially automatically. Alternatively or additionally, the training step can be carried out at least partially manually. Automatic training of the classifier can also be checked manually or automatically. In addition to this, the training of the classifier can also be based on or depend on vehicle-supported acquisition of area information on the agricultural area. As a further step, the method can include training of the classifier with area information acquired with the aid of a vehicle. The training can also be based on merged airborne and vehicle-based measurement data.

The step of training the classifier can have, as a substep, the step of air-assisted acquisition of area information on the agricultural area using the area detection sensor system mounted on the unmanned missile. The method can therefore be implemented as a method for training a classifier for an agricultural area. The step of training the classifier can have, as a further substep, the step of defining the classifier for assigning area information to at least one area information class. At least one of the substeps can be performed automatically. Alternatively or additionally, at least one of the substeps can be carried out manually.

The step of training the classifier can have, as a substep, a recognition of a surface object in the surface information recorded airborne. The surface object can be recognized automatically. Alternatively or additionally, the area object can be recognized manually. A recognized area object can be assigned to at least one area information class. A recognized area object can, for example, also be assigned area information which relates to an agricultural area boundary.

A surface object can be recognized using methods of pattern recognition. Alternatively or additionally, a surface object can be recognized by means of template matching methods. As an alternative or in addition, a surface object can be recognized using methods of semantic segmentation. The methods can be corresponding two-dimensional or three-dimensional methods. For automatic recognition of a surface object, automatic methods of pattern recognition or automatic methods of template matching can be involved. It can also be a question of manual recognition through manual marking or labeling of an area object. The training of the classifier or the recognition of the surface object can include matching or labeling of a surface object in the surface information recorded with the aid of the air. The step of training or at least one of its manually executable sub-steps can be performed by an operator of the unmanned missile or an operator. The training of the classifier can also have, as a further sub-step, manual verification of an automatically recognized surface object by an operator or surgeon.

The training of the classifier can have as a further substep an assignment or linking of at least one feature of a recognized area object to at least one area information class. The assignment or linking can be an automatic assignment or an automatic linking. As an alternative or in addition, the assignment or linking can be a manual assignment or a manual linking.

An area object can be, for example, a land surface area, a vegetation area, a building, an infrastructure object or a vehicle on the agricultural area. The terrain surface area can have at least one of a topology, a waviness, a terrain contour and a surface quality of the agricultural area. The terrain contour can be a ditch, for example. The vegetation area can, for example, have a usable area planting. The structure can be, for example, a road, a path, a wall or a fence. The road, the path, the wall or the fence can limit the agricultural area. The building can therefore represent a detectable or recognizable usable area boundary. The infrastructure object can be, for example, a mast or a traffic sign.

A feature of a recognized surface object can be derived manually or automatically from the surface information recorded with the aid of the air. Here, a subset of the recorded area information can be assigned to the recognized area object. The feature can have a geometry, a pattern, a cluster, a color, a reflectivity or an intensity in measurement data acquired with the area acquisition sensor system. The feature can be a corresponding two-dimensional or three-dimensional feature. A recognized area object can thus be assigned to an area information class if a feature is present or if a recognized feature corresponds to a predefined feature. To train the classifier, a predefined feature can be modified or supplemented by a recognized feature. A classifier trained in this way can make an assignment of recorded area information to at least one area information class more reliable for a user of the classifier.

The training of the classifier can also have, as a further sub-step, an assignment or linking of at least two features of a recognized area object to at least one area information class. For this purpose, a predefined feature can be modified or supplemented by a recognized feature. The at least two features can be grouped in a feature vector or in a feature matrix. The at least two features can be linked to one another or correlated in the feature vector or in the feature matrix. A recognized area object can thus be assigned to an area information class if at least two features correlated with one another are present or if at least two recognized and interrelated features correspond to a predefined feature vector or a predefined feature matrix. For this purpose, a recognized feature vector or a recognized feature matrix can be compared with a predefined feature vector or a predefined feature matrix.

Within the scope of the invention, training data for training a classifier for an agricultural area can be recorded with the aid of the air. From the air-assisted acquired training data, which can have measurement data or aerial images of the agricultural area, area information can be obtained for area information classes based on one in the Area information are assigned to recognized area objects. The training data can include georeferenced or transformed air-supported area information. The assignment of the recorded area information or the area objects recognized in it can then be carried out based on object features. The training data can also have ground truth data that are recorded manually. The ground truth data can be a polygon that can represent a measured usable area boundary.

The classifier can be trained in such a way as to automatically assign acquired area information to at least one area information class. The detection of objects in the air-supported area information can be automated. Furthermore, the assignment of features to the recognized objects can be automated. By assigning objects or their features to an area information class, areas of the area information can be automatically assigned to at least one area information class.

In the context of the invention, the classifier can also be trained by an end-to-end training process or an end-to-end learning process. “End-to-end” can refer to a correspondingly automated process. “End-to-end” can also refer to adaptive training or provision of the classifier, with the method being able to be carried out in a manner adapted to a current surface area or useful area boundary of the agricultural area. “End-to-end” can also refer to cyclical training or provision of the classifier, with the method being able to be carried out as a loop. The classifier can thus be trained and made available with the method, for example, on a yearly or seasonal basis or in work steps. “End-to-end” can also refer to a usable area-specific or customer-specific training or provision of the classifier, with the method being able to be carried out for an individual usable area or for an individual farmer.

The area information recorded in the air can be used for at least one other agricultural application. The further agricultural application can be a smart farming application or an e-farming application.

In a further-developing embodiment of the method, the step of air-assisted acquisition has at least one of air-assisted acquisition of topographical information about the agricultural area and air-assisted acquisition of image information about the agricultural area. The air-based acquisition of topographic information and the air-based acquisition of image information can be carried out with the area acquisition sensors mounted on the unmanned missile.

The air-assisted acquisition of topography information can include an air-assisted acquisition of height information on the agricultural area. The air-supported acquisition of topography information can be carried out, for example, with a laser scanner. The air-assisted acquisition of image information can include an air-assisted acquisition of texture information on the agricultural area. The air-supported acquisition of image information can be carried out, for example, with a camera.

Topography information captured by air can be merged with image information captured by air. Fused information can be combined information. As an alternative or in addition, topography information acquired airborne can be merged with image information acquired non-airborne, satellite-based, earth-based or vehicle-based. For example, an air-supported point cloud relating to the agricultural area can be merged with a current image of the agricultural area to form a colored point cloud relating to the agricultural area. The step of air-assisted acquisition can furthermore also include air-assisted acquisition of a 3D terrain model of at least one surface area of the agricultural land. The 3D terrain model can be a textured 3D terrain model, which can have a topography of the agricultural area textured with an image.

In a further-developing embodiment of the method, the method has as a further step a transformation of the air-supported area information acquired into a vehicle-related coordinate system of an agricultural vehicle. The step of transforming can be performed prior to the step of training the classifier. The classifier can be trained on the basis of surface information that has been transformed in this way and recorded in the air. In a further step, area information recorded in the vehicle-related coordinate system relating to the agricultural area can be assigned to at least one area information class with the trained classifier. Acquisition and assignment of area information acquired with the aid of a vehicle can thus advantageously be carried out without a further (coordinate transformation directly in the vehicle-related coordinate system or in relation to a detection perspective or a detection position of a vehicle-based area detection can be carried out. As an alternative or in addition, the method can have, as a further step, a transformation of vehicle-supported area information and air-supported area information detected into a usable area-related coordinate system of the agricultural usable area.

In a further-developing embodiment of the method, the step of training the classifier is carried out based on at least one of training data stored in a training database and a pre-trained classifier. The training data stored in the training database can be air-supported area information on the agricultural area. The area information recorded air-supported can be recorded with the area detection sensors attached to the unmanned missile. The area detection sensor system can be an air-supported area detection sensor system. As an alternative or in addition, the training data stored in the training database can be vehicle-based, satellite-based or ground-based area information on the agricultural area. Area information acquired with the aid of a vehicle can be acquired with an area acquisition sensor system mounted on an agricultural vehicle. The area detection sensor system can be a vehicle-supported area detection sensor system. Area information acquired by satellite can be acquired with an area acquisition sensor system installed on a satellite. Surface information acquired from the ground can be acquired using surface acquisition sensors carried by a surgeon. The stored training data can also be supplemented or merged with the area information recorded in the air.

The pre-trained classifier can be designed like the described classifier. The pre-trained classifier can be pre-trained with air-supported area information according to the steps of the method. Alternatively or additionally, the pre-trained classifier can be pre-trained with air-supported or satellite-supported area information. Alternatively or in addition, the pre-trained classifier can be pre-trained with vehicle-supported or ground-based area information.

With the method, training data can be supplemented or modified in a training database. The method can also be used to supplement or modify a pre-trained classifier. In this way, a pre-trained classifier can advantageously be adapted to area information which can be recorded for an agricultural area and which is not yet available in the stored training data. With the method, the classifier can thus be adapted to an agricultural area that has not yet been recorded. The classifier can therefore be an adaptive classifier which can be adapted to area information that can currently be recorded for an agricultural area.

The pre-trained classifier can be pre-trained based on the training data stored in the training database. For this purpose, the stored training data can have the pre-recorded area information on the agricultural area. The training database can therefore be an agricultural training database with area information from various agricultural areas. The pre-trained classifier can therefore also be a classifier for assigning area information from different agricultural areas. The classifier can therefore function as a classifier that extends across all areas. The training database can be in a computer center. The training data can thus be available offline.

In a further-developing embodiment of the method, the method has, as a further step, the transmission of the trained classifier as an over-the-air update (OTA update) to an agricultural vehicle. The transmission can include sending and receiving of the over-the-air update. The over-the-air update can be a radio-based update for the agricultural vehicle. The over-the-air update can be transmitted to the agricultural vehicle via a cellular network or a WLAN network. The unmanned missile can have a corresponding interface for sending the over-the-air update. The agricultural vehicle can have a corresponding interface for receiving the over-the-air update. With the classifier transmitted to the agricultural vehicle and trained at least on the basis of the area information recorded in the air, area information recorded by the agricultural vehicle can later be assigned to at least one area information class with the transmitted classifier. Operating times of the agricultural vehicle can thus be extended and vehicle service can be improved.

In a further developing embodiment of the method, the method has, as a further step, a vehicle-supported acquisition of area information on the agricultural area with an on an agricultural vehicle attached area detection sensors. The area information can thus be area information acquired with the aid of a vehicle.

The agricultural vehicle can be an agricultural or forestry machine, an agricultural or forestry device or an agricultural or forestry utility vehicle. The agricultural vehicle can be an agricultural tractor, for example a tractor. The agricultural vehicle can also be an agricultural work vehicle, for example a harvesting machine. The usable agricultural area can be an agricultural or forestry area to be cultivated. For example, the agricultural area can have at least one of a field, a field with planted or harvested useful plants, a meadow and a forest. The agricultural vehicle can be designed as an automatically operated vehicle. The agricultural vehicle can be an autonomously driving or working agricultural vehicle.

The agricultural vehicle can be at least partially guided manually or remotely controlled by an operator of the agricultural vehicle for vehicle-assisted acquisition of area information. The agricultural vehicle can thus drive semi-automatically on the agricultural area. The agricultural vehicle can also drive (fully) automatically on the agricultural area in order to record area information on the agricultural area with the aid of a vehicle.

The vehicle-supported acquisition of area information can be carried out starting from the agricultural vehicle. For this purpose, the area detection sensor system can be arranged on the agricultural vehicle or integrated therein. The area detection sensor system can drive along on the agricultural vehicle for this purpose. The vehicle-supported acquisition of area information can therefore include acquisition of area information from the journey. For this purpose, the agricultural vehicle can act as a moving acquisition platform for vehicle-supported acquisition of area information. The vehicle-based acquisition is therefore not an air-based, ground-based or satellite-based acquisition of area information. Georeferencing or transforming can be carried out on the agricultural vehicle (online) or in a data center (offline).

The vehicle-assisted acquisition of area information can include georeferencing of the vehicle-assisted acquired area information. For this purpose, the area information recorded with the aid of a vehicle can be transformed into the predetermined coordinate system.

The agricultural vehicle can have a position detection sensor system for detecting a position of the agricultural vehicle on the agricultural area. The position detection sensor system can have a GNSS antenna with which the position of the agricultural vehicle can be detected. The area information acquired with the aid of a vehicle or corresponding raw measurement data can be transformed into the coordinate system based on such a position acquisition. The vehicle-supported area information or corresponding raw measurement data can also be transformed into the coordinate system based on an orientation detection or based on extrinsic parameters of the agricultural vehicle. Furthermore, the area information recorded with the aid of a vehicle can be transformed into the geographic information system (GIS) or into the map. The method can also be carried out if the position detection sensor system fails.

The area detection sensor system mounted on the agricultural vehicle can be designed like the area detection sensor system mounted on the unmanned missile. The area detection sensor system mounted on the agricultural vehicle can also have the distance-measuring sensor described. As an alternative or in addition, the area detection sensor system mounted on the agricultural vehicle can also have the image-capturing sensor described. The area detection sensor system attached to the unmanned missile and the area detection sensor system attached to the agricultural vehicle can have the same sensor design, can have the same sensors or can generate the same measurement data for the agricultural area.

According to this further developing embodiment, the method can include, as a further step, an assignment of the vehicle-supported area information with the classifier trained with the air-supported area information to at least one area information class.

The step of assigning the area information recorded with the aid of a vehicle can be a sub-step of recognizing an area object in the area information recorded with the aid of a vehicle exhibit. The surface object can be recognized as described for the step of training the classifier. The assignment of the area information detected with the aid of a vehicle can have as a further sub-step an assignment or linking of at least one feature of a recognized area object to at least one area information class. The assigning or linking can be an assignment or linking as described for the step of training the classifier. The area object can be one of the area objects described for the step of training the classifier on the agricultural area. A feature of a recognized surface object can be derived manually or automatically from the vehicle-assisted recorded surface information, as described for the step of training the classifier.

According to this further developing embodiment, the method can have, as a further step, a localization of the agricultural vehicle on the agricultural area based on the area information that is associated with the classifier and is detected with vehicle support. Locating the agricultural vehicle on the agricultural area can include positioning the agricultural vehicle on the agricultural area. The localization of the agricultural vehicle on the agricultural area can include determining a position of the agricultural vehicle on the agricultural area. A specific location can have a position of the agricultural vehicle on the agricultural area. As an alternative or in addition, a certain position can have an orientation of the agricultural vehicle on the agricultural area. Locating the agricultural vehicle on the agricultural area can also include determining a pose of the agricultural vehicle on the agricultural area. The location, the position, the orientation or the pose can be a relative location, a relative position, a relative orientation or a relative pose of the agricultural vehicle to an area or an area delimitation of the agricultural area.

As a further step, the method can include navigating the agricultural vehicle on the agricultural area based on the step of localizing the agricultural vehicle on the agricultural area. Navigating the agricultural vehicle can include following a predetermined trajectory with the agricultural vehicle, further based on the step of assigning the vehicle-supported area information with the classifier trained with the air-supported area information. With the trained classifier it is therefore advantageously not necessary to artificially delimit or mark an agricultural area in order to localize and navigate the agricultural vehicle based on the trained classifier of the agricultural area in the event of a failure of a positioning system. The reliability and safety of an automated operation of an agricultural utility vehicle on an agricultural area can thus be improved.

With the invention, the agricultural vehicle can advantageously continue to be automatically guided along the predetermined trajectory even if a positioning system fails, for example a component of a GNSS positioning system. The component of the GNSS positioning system can be a transmitting-side GNSS component or a receiving-side GNSS component. The agricultural vehicle can thus also be steered in an advantageous manner based on the area information associated with the classifier. It is therefore not necessary for the agricultural vehicle to have to be actively steered by a driver of the agricultural vehicle in the event of a failure of a positioning system. In the event of such a failure, automated steering does not necessarily have to be handed over to the driver.

To localize or navigate the agricultural vehicle on the agricultural area, the area detection sensor system can also be used to determine a distance between the agricultural vehicle and a recognized area object. For this purpose, an orientation of the agricultural vehicle in relation to the recognized area object can also be determined. The identified area object can also be area information relating to a usable area boundary.

According to a further-developing embodiment of the method, the step of vehicle-assisted acquisition has at least one of vehicle-assisted acquisition of topographical information on the agricultural area and vehicle-assisted acquisition of image information regarding the agricultural area. The vehicle-assisted acquisition of topography information and the vehicle-assisted acquisition of image information can be carried out with the area acquisition sensors installed on the agricultural vehicle.

The vehicle-based acquisition of topography information can include a vehicle-based acquisition of height information on the agricultural area. Height information can be recorded in a profile-like, fan-like or planar manner. The vehicle-supported acquisition of topography information can be carried out, for example, with a laser scanner, a radar device or an ultrasound device. The vehicle-assisted acquisition of image information can include a vehicle-assisted acquisition of texture information on the agricultural area. The vehicle-supported acquisition of image information can be carried out, for example, with at least one camera arranged on the agricultural vehicle. In a further example, the vehicle-supported acquisition of image information can be carried out with a camera system which is calibrated on the agricultural vehicle and which can have at least one camera arranged on the agricultural vehicle.

Topography information recorded with the aid of a vehicle can be merged with image information recorded with the aid of a vehicle. As an alternative or in addition, topography information acquired with the aid of a vehicle can be merged with image information that is acquired with a non-vehicle, satellite, earth or air support. For example, a point cloud recorded with the aid of a vehicle for the agricultural area can be merged with a current image of the agricultural area to form a colored point cloud for the agricultural area. The step of vehicle-assisted acquisition can also include vehicle-assisted acquisition of a 3D terrain model of at least one area of the agricultural area.

In a further-developing embodiment of the method, the method has, as a further step, filtering of area information recorded on the agricultural usable area with a filter for preprocessing recorded area information. The area information captured can be the area information captured air-supported. As an alternative or in addition, the recorded area information can be the area information recorded with the aid of a vehicle. The filtering can be, for example, carrying out an occupancy grid mapping. By clustering or collecting area information with an occupancy grid, the method can advantageously be carried out more robustly. The filtering can preprocess raw measurement data acquired with an area acquisition sensor system. For example, Cartesian coordinates can be generated from polar measurement data from a distance-measuring sensor. Furthermore, ground points of a point cloud can be filtered by means of a first-pulse method so that these points have vegetation points. Alternatively or additionally, vegetation points of a point cloud can be filtered by means of a last pulse method so that it has ground points.

According to a further developing embodiment of the method, the classifier has a neural network for assigning area information to at least one area information class. In one example, the neural network can be a single-layer or two-layer network. The neural network can also be based on deep learning and can have at least one intermediate layer or an inner network structure between an output layer and an input layer. The neural network can have at least one input that can be based on at least one feature of a recognized surface object. The at least one input can have a respective weighting. The neural network can have at least one output which can represent at least one area information class. The neural network can have at least one connection between the at least one input and the at least one output, which connection can function for automated assignment. For this purpose, the neural network can also have at least one node between the at least one input and the at least one output.

According to this embodiment, the training of the classifier can include training of the neural network. The training of the neural network can include adding, modifying or removing from the at least one connection based on acquired area information. Alternatively or additionally, the training of the neural network can include adding, modifying or removing the at least one weighting based on the recorded area information. As an alternative or in addition, the training of the neural network can include adding, modifying or removing the at least one input or output based on acquired area information. Furthermore, the further described training of the classifier can also be carried out in this way for training the neural network.

As a further step, the method can have a vehicle-assisted detection of an acceleration acting on the agricultural vehicle as a result of driving on the agricultural area. The vehicle-aided detection of the acceleration can be carried out with an acceleration sensor or an accelerometer which is mounted on the agricultural vehicle. For example, an inertial measuring unit (IMU), a MEMS sensor or a gyroscope can be attached to the agricultural vehicle to detect the acceleration. In addition to the localization of the agricultural vehicle based on an assignment of vehicle-supported area information detected, a location of the agricultural vehicle on the agricultural area can be detected. For example, the crossing of a trench by vertical acceleration can be detected in order to conclude that the agricultural area has been left.

In a further development aspect, the invention relates to a system for providing a classifier for an agricultural area. The system can also function to locate an agricultural vehicle on the agricultural area. The system can be designed and set up to carry out the method according to the preceding aspect.

The system has an area detection sensor system for air-supported detection of area information on the agricultural area. The area detection sensor system can be designed, set up and suitable as described for the preceding aspect.

The system has a system component for training a classifier for assigning area information to at least one area information class. The at least one area information class can be at least one of the area information classes described in relation to the preceding aspect. This and every further system component described can be a corresponding component of a control device. The classifier is trained with the area information recorded in the air. The classifier is stored in the system component. The system component has an input interface for reading in surface information recorded airborne. This and every other system component described can be a corresponding computer-implemented system component.

According to a further-developing embodiment of the system, it has an area detection sensor system for vehicle-supported detection of area information on the agricultural area. The area detection sensor system can be designed, set up and suitable as described for the preceding aspect.

According to this further-developing embodiment, the system can have a system component for assigning vehicle-supported area information with the trained classifier to at least one area information class. The trained classifier can be stored in the system component. The trained classifier can be stored in the system component based on an update, which can be an over-the-air update. The system component can have an input interface for reading in area information recorded with the aid of a vehicle. The system component can have a storage medium for holding the trained classifier on the agricultural vehicle.

According to this further developing embodiment, the system can have a system component for localizing the agricultural vehicle on the agricultural area based on vehicle-supported area information associated with the classifier. The system component can have an input interface for reading in vehicle-supported area information associated with the classifier. The system component can be a component of a navigation system on the agricultural vehicle.

• 1 zeigt schematisch Systeme zum Bereitstellen eines Klassifikators und zum Lokalisieren eines landwirtschaftlichen Fahrzeugs in Ausführungsformen der Erfindung.
• 2 zeigt ein Ablaufdiagramm zur Erläuterung von Verfahren zum Bereitstellen eines Klassifikators und zum Lokalisieren eines landwirtschaftlichen Fahrzeugs in einer zeitlichen Abfolge nach Ausführungsformen der Erfindung.

According to a further-developing embodiment of the system, the system component for assigning area information detected with the aid of a vehicle has an update input interface for receiving an over-the-air update (OTA update) which has the trained classifier. The over-the-air update can be designed and communicated as described for the previous aspect.

• 1 shows schematically systems for providing a classifier and for locating an agricultural vehicle in embodiments of the invention.
• 2 FIG. 3 shows a flowchart for explaining methods for providing a classifier and for localizing an agricultural vehicle in a time sequence according to embodiments of the invention.

1 shows schematically an area of an agricultural area 2 with an agricultural land border 4th . The agricultural area boundary 4th forms an edge of the agricultural area 2 . The agricultural area boundary 4th borders the area of the agricultural area shown schematically 2 at a corner area of the agricultural area 2 . The two in 1 Agricultural land shown above and below 2 it is the same agricultural area 2 .

Above the in 1 agricultural area shown above 2 flies an unmanned missile 20th . On an underside of the unmanned missile 20th is an air-supported area detection sensor system 22nd appropriate. The unmanned missile 20th thus supports the air-supported area detection sensors 22nd in the air. The air-supported area detection sensors 22nd has an airborne detection area 23 in which the air-supported area detection sensors 22nd Area information 6th , 8th to the agricultural area 2 detected.

With air-supported area detection sensors 22nd In one embodiment, it is an air-supported area detection system which comprises a laser scanner (not shown) and a camera (not shown). The unmanned missile 20th and the airborne area detection sensors 22nd are located in one embodiment during the acquisition of the area information 6th , 8th at rest above the agricultural area 2 . In another embodiment, the unmanned missile is located 20th and the airborne area detection sensors 22nd above the agricultural area 2 during the acquisition of the area information 6th , 8th moving. Area information 6th , 8th is thus static or dynamic by the air-supported area detection sensors 22nd detected. The captured area information 6th , 8th indicates the agricultural area 2 related area information 6th and does not affect the agricultural area 2 related area information 8th on.

In 1 is a system component 110 shown, which with the air-supported area detection sensors 22nd of the unmanned missile 20th connected is. In one embodiment, the system component is 110 on the unmanned missile 20th arranged. According to this embodiment is the system component 110 with the air-supported area detection sensors 22nd wired. In a further embodiment, the system component 110 not on the unmanned missile 20th arranged. According to this embodiment is the system component 110 a component of a data center. According to this embodiment, the air-supported area detection sensor system 22nd with the system component 110 connected via a wireless connection. The wireless connection is a radio-based connection. In one embodiment, the data center is a mobile data center which is located on or next to the agricultural area 2 during a flight of the unmanned missile 20th installed. In a further embodiment, the data center is a stationary data center, which is located in a control center for the remote-controlled flying of the unmanned missile 20th installed.

The system component 110 assigns a classifier K on. The classifier K is via one of the connections described with the unmanned aerial vehicle 20th and airborne area detection sensors 22nd connected.

The air-supported area detection sensors 22nd of the unmanned missile 20th , the system component 110 and the classifier K form a system 100 to provide the classifier K for the agricultural area 2 . The system component 110 is a component of the system 100 . One in 1 not shown control device of the unmanned missile 20th assigns the system component 110 on.

The system component 110 is in 1 above and to the right of the displayed agricultural area 2 shown in more detail. The system component 110 has an input interface 112 with which the airborne area detection sensors 22nd recorded area information 6th , 8th to the agricultural area 2 into the system component 110 is read. One of the described connections of the airborne area detection sensors 22nd and the classifier K assigns the input interface 112 on. The input interface 112 is in the system component 110 with a filter F. connected. The read-in air-supported area information 6th , 8th becomes the filter F. for filtering or preprocessing the air-supported area information 6th , 8th fed. The filtered air-supported area information 6th , 8th becomes the classifier K to train the classifier K fed. The classifier K is trained automatically or manually. The automatic or manual training of the classifier K is based on a training database T and a pre-trained classifier K ' . The training database T and the pre-trained classifier K ' are each with the classifier K in the system component 110 connected. The classifier K is trained in such a way that the airborne recorded area information 6th , 8th can be assigned to at least one area information class.

An area information class is formed on the agricultural area 2 related area information 6th . Another area information class is not based on the agricultural area 2 related area information 8th . Yet another area information class is the agricultural area boundary 4th which is a natural boundary between the two previous area information classes. The trained classifier K is in the system component 110 with an update output interface 114 the system component 110 connected.

With the system component 110 in one embodiment it is a computer-implemented component. The system component 110 is a component of a data processing system for providing the trained classifier K and to control the unmanned missile 20th . The filter F. , the classifier K , the training database T and the pre-trained classifier K ' form corresponding computer-implemented elements of the system component 110 . The connections in the system component 110 are set up as computer-implemented connections according to this embodiment.

Via the update outbound interface 114 of the system 100 is an over-the-air update in the embodiment shown OTA provided. The over-the-air update OTA is a radio-based update. The over-the-air update OTA is a link between the in 1 system shown above 100 and the other in 1 system shown below 200 . The over-the-air update OTA is thus a link between the system 100 to provide the classifier K and the system 200 for locating and navigating an agricultural vehicle 10 . In one embodiment, the in 1 system shown above 100 this in 1 system shown below 200 on.

The agricultural vehicle 10 is on the agricultural land 2 operated in the area in which the area information 6th , 8th previously with the air-supported area detection sensors 22nd has been captured. On the agricultural vehicle 10 is a vehicle-based area detection sensor system 12 appropriate. In the embodiment shown, the vehicle-based area detection sensor system 12 in a front report of the agricultural vehicle 10 arranged. The vehicle-based area detection sensors 12 has a vehicle-based detection area 13 on. The vehicle-based area detection sensors 12 captures area information 6th , 8th to the agricultural area 2 . In one embodiment, the vehicle-based area detection sensor system 12 an area detection system which comprises a laser scanner and a camera. The airborne detection area 23 the air-supported area detection sensors 22nd and the vehicle-based detection area 13 the vehicle-based area detection sensors 12 are each conical in one embodiment. A cone axis of the airborne detection area 23 is oriented vertically. A cone axis of the vehicle-based detection area 13 is oriented horizontally.

This in 1 system shown below 200 shows in addition to the vehicle-based area detection sensors 12 two system components 210 , 220 on. A first system component 210 assigns the trained classifier K on. With the first system component 210 is the area information captured by the vehicle 6th , 8th with the trained classifier K assigned to at least one of the area information classes. With the second system component 220 becomes the agricultural vehicle 10 based on the assigned area information 6th , 8th on the agricultural land 2 localized and navigated.

The system 200 is in 1 below and to the right of the agricultural area shown schematically 2 more detailed together with the agricultural vehicle 10 shown. The first system component 210 for assigning the area information captured by the vehicle 6th , 8th has an update input interface 214 to receive the over-the-air update OTA on.

In one embodiment, the over-the-air update OTA before operating the agricultural vehicle 10 on the agricultural land 2 and the system component 110 of the system 100 on the first system component 210 of the system 200 transfer. The trained classifier K is via the update input interface 214 in the first system component 210 deposited.

The first system component 210 has an input interface 212 on. With the vehicle-supported area detection sensors 12 recorded area information 6th , 8th is via the input interface 212 in the first system component 210 read in. The input interface 212 is with one in the first system component 210 stored filter F. connected. The filter F. filters the read-in area information 6th , 8th . In one embodiment it is the filter F. the first system component 210 and the filter F. the system component 110 of the system 100 around the same measurement data filter for preprocessing with the vehicle-supported area detection sensors 12 and with the air-supported area detection sensors 22nd each recorded raw measurement data.

The filtered area information 6th , 8th become the stored classifier K in the first system component 210 fed. This is what the filter is for F. in the first system component 210 with the classifier K in the first system component 210 connected. The classifier K arranges the area information recorded by the vehicle 6th , 8th at least one of the area information classes. The first system component 210 has an output interface 216 for outputting in this way with the classifier K assigned vehicle-supported area information recorded 6th , 8th on. Via the output interface 216 the vehicle-assisted recorded area information assigned in this way becomes 6th , 8th The second System component 220 fed. The agricultural vehicle 10 is based on the with the classifier K assigned vehicle-assisted detected area information localized.

For reading in the with the classifier K assigned, vehicle-supported recorded area information 6th , 8th into the second system component 220 assigns this to an input interface 222 on. In the second system component 220 becomes the position of the agricultural vehicle 10 on the agricultural land 2 calculated. In one embodiment, in addition to the position, there is also an orientation of the agricultural vehicle 10 to the agricultural area 2 calculated. The second system component 220 is set up to convert parameters between a vehicle-related coordinate system 11 and a space-related coordinate system 3 the agricultural area 2 to calculate. The transformation parameters can have corresponding translation parameters and rotation parameters.

The area information captured by the vehicle 6th , 8th is based on that in the first system component 210 stored and trained classifier K assigned to at least one of the area information classes. Based on the classification of the area information 6th , 8th the position, the orientation or the transformation parameters are calculated spatially. The agricultural vehicle 10 In one embodiment, it is based on the agricultural area boundary 4th when driving on the agricultural area 2 .

2 shows a flowchart of method steps for performing a method for providing the classifier K for the agricultural area 2 . In one embodiment, the method is also used to locate the agricultural vehicle 10 on the agricultural land 2 carried out.

In one step S1 an airborne area information acquisition takes place S1 . Area information 6th , 8th to the agricultural area 2 will in this step S1 air-supported with the air-supported area detection sensors 22nd detected. In a further step S1a georeferencing takes place S1a . In this step S1a is the airborne recorded area information 6th , 8th based on the usable area-related coordinate system 3 the agricultural area 2 georeferenced. The geo-referenced air-supported area information 6th , 8th becomes the training database T the system component 110 fed.

In a further step S2 classifier training takes place S2 . This is done in an intermediate step S1b a transformation S1b . In the intermediate step S1b the georeferenced airborne recorded area information 6th , 8th into the vehicle-related coordinate system 11 transformed. In addition to the transformed air-supported area information 6th , 8th is used for classifier training S2 further area information, which is in the training database T is already stored. To provide the trained classifier K is based on a pre-trained classifier K ' resorted to. In one embodiment, the pre-trained classifier is K ' also in the training database T deposited. In the step S2 of classifier training S2 becomes the area information 6th , 8th automatically or manually assigned to at least one of the area information classes. Thus the trained classifier represents K an assignment rule for assigning recorded area information 6th , 8th represent.

The trained classifier K will be in a further step S4 about that too 1 described over-the-air update OTA for locating the agricultural vehicle 10 on the agricultural land 2 transferred to the agricultural vehicle. In step S4 a transfer thus takes place S4 of the trained classifier K between the systems 100 , 200 . So that we are the trained classifier K into a process sequence for locating the agricultural vehicle 10 on the agricultural land 2 involved. This process sequence shows the further process steps S3 to S7 on.

In a further step S3 a vehicle-supported area information acquisition takes place S3 of area information 6th , 8th with the one on the agricultural vehicle 10 attached vehicle-based area detection sensors 12 . In one embodiment, a further step takes place S3a a georeferencing S3a . In this step S3a is the area information captured by the vehicle 6th , 8th analogous to georeferencing S1a in step S1a georeferenced. During the classifier K based on the vehicle-based coordinate system 11 is trained, becomes the agricultural vehicle 10 in the space-related coordinate system 3 localized and navigated.

In a further step S5 a class assignment takes place S5 the area information captured by the vehicle 6th , 8th with the on the agricultural vehicle 10 transferred classifier K , for example for area information 6th , 8th which refer to the agricultural area boundary 4th relates and which in one embodiment forms an area information class. Based on the recorded and with the classifier K assigned area information 6th , 8th the agricultural vehicle positions and orients itself 10 in one embodiment automatically and relative to the agricultural area boundary 4th . Positioning and orienting is a further step S6 being carried out in this step S6 a vehicle location S6 on the agricultural land 2 is carried out.

In a further step S7 navigation takes place S7 of the agricultural vehicle 10 on the agricultural land 2 based on the previous step S6 the vehicle location S6 . The agricultural vehicle 10 for this purpose, in one embodiment, compares the recorded and with the classifier K to the agricultural land border 4th assigned area information 6th , 8th with a known coordinate and on the agricultural vehicle 10 deposited agricultural land border 4th . This is how the agricultural vehicle orients itself 10 on the agricultural area 2 and continues route planning independently of position detection with a position detection system.

List of reference symbols

2

agricultural land

3

area-related coordinate system

4th

agricultural land border

6, 8

Area information

10

agricultural vehicle

11

vehicle-related coordinate system

12

vehicle-based area detection sensors

13th

vehicle-based detection area

20th

unmanned missile

22nd

air-supported area detection sensors

23

airborne detection area

100

Classifier provision system

110

System component

112

Input interface

114

Update outbound interface

200

Vehicle location system

210

first system component

220

second system component

212, 222

Input interface

214

Update input interface

216

Output interface

F.

filter

K

Classifier

K '

pre-trained classifier

OTA

Over-the-air update

S1

airborne area information acquisition

S1a

Georeferencing

S1b

transformation

S2

Classifier training

S3

vehicle-based area information acquisition

S3a

Georeferencing

S4

transmission

S5

Class assignment

S6

Vehicle location

S7

navigation

T

Training database

Claims (15)

Method for providing a classifier (K) for an agricultural area (2), comprising the steps: - Air-supported detection (S1) of area information (6, 8) on the agricultural area (2) with area detection sensors (22) mounted on an unmanned missile (20), - Defining a classifier (K) for assigning area information (6, 8) to at least one area information class, and - Training (S2) the classifier (K) with the air-supported area information (6, 8). Procedure according to Claim 1 wherein area information (6) relating to the agricultural area (2) forms an area information class. Procedure according to Claim 1 or 2 , with area information (8) not relating to the agricultural area (2) forming an area information class. Method according to one of the preceding claims, wherein area information relating to an agricultural area boundary (4) forms an area information class. Method according to one of the preceding claims, wherein the step of air-assisted recording (S1) comprises at least one of air-assisted recording of topography information on the agricultural area (2) and air-assisted recording of image information on the agricultural area (2). Method according to one of the preceding claims, with the further step of transforming (S1b) the area information (6, 8) recorded in the air into a vehicle-related coordinate system (11) of an agricultural vehicle (10). Method according to one of the preceding claims, wherein the step of training (S2) the classifier (K) is carried out based on at least one of training data stored in a training database (T) and a pre-trained classifier (K '). Method according to one of the preceding claims, with the further step of transmitting (S4) the trained classifier (K) as an over-the-air update (OTA) to an agricultural vehicle (10). Method according to one of the preceding claims, with the further steps: - Vehicle-based acquisition (S3) of area information (6, 8) on the agricultural area (2) with an area detection sensor system (12) mounted on an agricultural vehicle (10), - Assigning (S5) the area information (6, 8) recorded with the aid of a vehicle with the classifier (K) trained with the area information recorded with the aid of the air to at least one area information class, and - Localizing (S6) the agricultural vehicle (10) on the agricultural area (2) based on the area information (6, 8) associated with the classifier (K) with the aid of a vehicle. Procedure according to Claim 9 wherein the step of vehicle-assisted acquisition (S3) comprises at least one of vehicle-assisted acquisition of topographic information on the agricultural area (2) and vehicle-assisted acquisition of image information on the agricultural area (2). Method according to one of the preceding claims, with the further step of filtering area information (6, 8) recorded on the agricultural area (2) with a filter (F) for preprocessing recorded area information (6, 8). Method according to one of the preceding claims, wherein the classifier (K) has a neural network for assigning area information (6, 8) to at least one area information class. System (100) for providing a classifier (K) for an agricultural area (2), which has - an area detection sensor system (22) for air-supported detection of area information (6, 8) on the agricultural area (2) and - A system component (110) for training a classifier (K) for assigning area information (6, 8) to at least one area information class, the classifier (K) being stored in the system component (110), and the system component (110) being a Has an input interface (112) for reading in area information (6, 8) recorded air-supported. System (100, 200) Claim 13 which has - an area detection sensor system (12) for vehicle-supported detection of area information (6, 8) on the agricultural area (2), - a system component (210) for assigning vehicle-supported area information (6, 8) with the trained classifier (K ) for at least one area information class, the trained classifier (K) being stored in the system component (210), and - a system component (220) for localizing the agricultural vehicle (10) on the agricultural area (2) based on the classifier ( K) assigned vehicle-supported area information (6, 8), the system component (210) having an input interface (212) for reading in vehicle-supported area information (6, 8) for assigning, and the system component (220) having an input interface ( 222) for reading in with the classifier (K) assigned vehicle-supported detected ter area information (6, 8). System (100, 200) Claim 14 wherein the system component (210) has an update input interface (214) for receiving an over-the-air update (OTA), which has the trained classifier (K), for assigning area information (6, 8) detected vehicle-supported.

Images