WO2020182564A1

WO2020182564A1 - Vision-based steering assistance system for land vehicles

Info

Publication number: WO2020182564A1
Application number: PCT/EP2020/055651
Authority: WO
Inventors: Benedict Strohmaier; Costanza CASCELLI; Tommy Ikonen; Jan-Frederik Kuhn
Original assignee: Zf Friedrichshafen Ag
Priority date: 2019-03-11
Filing date: 2020-03-04
Publication date: 2020-09-17
Also published as: DE102019203247A1

Abstract

The invention relates to the autonomous operation of a vehicle (1). For example, the invention relates to a processor unit for autonomous operation of a vehicle (1). The processor unit comprises an interface, wherein the processor unit is configured to access, by means of the interface, an environment detection of the vehicle (1) generated by a sensor unit, to recognise a plurality of track lines (4.1, 4.2) in the environment detection, to select a track line (4.1) from the plurality of track lines (4.1, 4.2), and to control the vehicle (1) on the basis of the selected track line (4.1) in such a way that the vehicle (1) follows the selected track line (4.1) autonomously. Further claims are directed to a corresponding driver assistance system (5), a method as well as a computer program product and a vehicle (1).

Description

Vision-based steering assistance system for land vehicles

The invention relates to a processor unit, a driver assistance system, a method and a computer program product for the autonomous operation of a vehicle, in particular an agricultural utility vehicle.

There are driver assistance systems for agricultural vehicles or Maschi nen (agricultural technology) known. These driver assistance systems are intended to relieve the driver in particular as a fully automated or partially automated steering aid. The steering aid via GPS control has established itself in agricultural engineering. The driver assistance systems are sometimes very complex and, depending on the correction signal, enable accuracies of up to 2 cm. Problems with GPS are the partially non-existent signal coverage worldwide (e.g. also in mountainous regions) and the high acquisition and license costs, depending on the system.

Alternative driver assistance systems for agricultural vehicles that are not based on GPS provide optical systems that automate certain applications in agriculture. Examples are driver assistance systems based on cameras, lidar sensors or, more generally, laser sensors. However, such systems are complex and expensive (in particular when lidar sensors are used) and / or can only carry out or support a simple application. For example, these systems typically only allow a swath to be recognized in the center in front of the vehicle and not to the side of the vehicle. A “swath” can be understood as an agricultural crop that has been mown and lined up in a row, e.g. Grass, grain, etc. Furthermore, such systems are not able to process several applications (e.g. mowing, swathing, loading or baling) and thus do not offer a real alternative to GPS systems.

A driver of an agricultural utility vehicle typically has to concentrate on one operation for a whole day. In addition, in addition to controlling the vehicle, the driver usually has to simultaneously monitor the function of attachments in the vehicle. This can cause it to Operating errors mean that the machines are not used optimally efficiently and that work performance and quality of work suffer.

Expensive GPS systems are often not profitable for small or medium-sized farms or are useless in areas with poor signal coverage. Optical assistance systems have so far been very limited in their function and can often only implement very simple applications.

An object of the present invention can be seen in providing a technology that is an alternative to expensive GPS systems, in particular, which relieves the driver and offers him more comfort in various applications (e.g. mowing, swathing, loading or baling).

The object is achieved by the subjects of the independent claims. Advantageous embodiments are the subject of the dependent claims, the following description and the figures.

According to the present invention, a technology is proposed which records lane lines (e.g. swaths or mowed edges) in a field, processes corresponding image content, selects the correct lane and then controls the vehicle's steering accordingly. A core of the invention is a system of simple sensors (e.g. cameras) and state-of-the-art image processing (expanded in particular by methods of artificial intelligence, see below), which makes it possible to flexibly design and use the system. Depending on the type of application and the area of application, the system can be implemented in different ways and thus adapted to customer requirements.

In particular, according to the present invention, three functionalities or method steps are proposed, namely the recognition of the lane lines ("seeing" or in English: "see"), the selection of the correct lane line ("thinking" or in English: "think") and the control of the steering of the vehicle ("act"). The present invention makes it possible to relieve the driver and give him more comfort in various applications (e.g. mowing, swapping den, loading or pressing). The agricultural utility vehicle can thus be used more efficiently. The efficiency increases, work performance and quality of work improve significantly and fuel consumption decreases.

In this sense, according to a first aspect of the invention, a processor unit for the autonomous operation of a vehicle is provided. The processor unit can form an element of an electronic control unit of the vehicle. The processor unit can also form an element of a driver assistance system of the vehicle. The processor unit includes an interface. The interface is in particular a communication interface which enables the exchange of data, specifically an exchange of data between the processor unit on the one hand and a sensor unit on the other. Furthermore, the processor unit can transmit commands to actuators of the vehicle via the interface, for example to a steering actuator of the vehicle. The processor unit is set up in particular to use the interface to access a surrounding area detection of the vehicle generated by the sensor unit, to recognize several lane lines in the surrounding area detection, to select a lane line from the several lane lines and to control the vehicle based on the selected lane line in such a way that the vehicle autonomously follows the selected lane line, e.g. that the vehicle drives on or to the side of the selected track line. If the sensor unit detects only a single lane line in the vicinity of the vehicle, then the processing unit is also set up to recognize this single lane line and to control the vehicle in such a way that it follows it.

According to a second aspect of the invention, a driver assistance system for the autonomous operation of a vehicle is proposed. The driver assistance system comprises a sensor unit for detecting the surroundings of the vehicle and a processor unit according to the first aspect of the invention. The sensor unit is set up to generate a detection of the surroundings of the vehicle.

According to a third aspect of the invention, a corresponding method for the autonomous operation of a vehicle is proposed. The method can generate a detection of the surroundings of the vehicle by a sensor unit and a know several lane lines in the environment detection by means of a processor unit. Furthermore, a lane line can be selected from the multiple lane lines using the processor unit, and the vehicle can be controlled based on the selected lane line using the processor unit so that the vehicle autonomously follows the selected lane line, e.g. the vehicle drives on or to the side of the selected lane line .

According to a fourth aspect of the invention, a computer program product for the autonomous operation of a vehicle is also provided. When it is executed on a processor unit, the computer program product instructs the processor unit to use an interface to access a vehicle environment detection generated by a sensor unit, to recognize several lane lines in the area detection, to select a lane line from the several lane lines, and to control the vehicle based on the selected lane line in such a way that the vehicle autonomously follows the selected lane line, e.g. that the vehicle is driving on or to the side of the selected track line.

According to a fifth aspect of the invention, a vehicle is provided, in particular an off-road vehicle. The vehicle can comprise a processor unit according to the first aspect of the invention. Alternatively or additionally, the vehicle can include a driver assistance system according to the second aspect of the invention.

The following statements apply equally to the processor unit according to the first aspect of the invention, for the driver assistance system according to the second aspect of the invention, for the method according to the third aspect of the invention, for the computer program product according to the fourth aspect of the invention and for the vehicle according to the fifth aspect of the invention.

The vehicle can be controlled by means of the processor unit. “Controlled” or “steer” can in particular be understood to mean that the vehicle can be operated autonomously, ie it can in particular be automatically steered, accelerated and braked, including all of the necessary controls Actuators in particular of the drive train, the steering and signaling of the vehicle. The control of the vehicle by means of the processor unit can be activated automatically or by a user of the vehicle by means of an operating element when the vehicle is traveling along a lane line. If the vehicle turns at a headland of the track line (s), the control of the vehicle can be deactivated automatically or by the user of the vehicle using an operating element.

A track line can, for example, be a swath or a mowing edge or a cutting edge on an agricultural area, e.g. a field. Furthermore, the lane line can comprise a lane, a row of plants, a row of trees, a curb or a work path. The processor unit can be set up not only to recognize swaths and mowed edges as lane lines and instruct the vehicle to follow them, but also other applications in agriculture, in the municipal sector or in the construction machinery business can be successfully automated with simple application effort. For example, rows of maize, apple trees in plantations, vines in vineyards, cut edges of a combine harvester in grain, hoed crops (soybeans, sugar beets, etc.), potato ridges or curbs for road sweepers when mowing roadsides can be recognized. The track line can, in particular, be located in a field and, for example, have been caused by previous sowing. Plant rows are rows of the cultivated product in question, e.g. Corn or grapevines. Rows of trees are for example on orchards, e.g. Apple plantations to be found. You can find commutes to work, for example, in mining (especially in open-cast mining).

The vehicle can in particular be an off-road vehicle. An off-road vehicle can be understood to mean a vehicle whose primary area of application is not a road (as is the case, for example, with passenger cars, buses or trucks), but for example an agricultural area, e.g. a field to be worked / tilled or a Forest area, or a mining area (especially open-cast mining), or an industrial area, e.g. within a production facility or warehouse. For example The off-road vehicle can be an agricultural utility vehicle or an agricultural tractor such as a combine harvester or tractor. Furthermore, the vehicle can be an industrial truck, for example a forklift or a tractor. Furthermore, the vehicle can also be a municipal vehicle, for example a street sweeper, a garbage truck or a snow clearing vehicle.

In particular, the processor unit is set up to select that lane line which has the smallest distance from the vehicle. For example, the detection of the surroundings can comprise several camera images recorded one after the other (image sequence), which depict the surroundings of the vehicle. For example, a first track line and a second track line can be recognized in one of these camera images. The first track line and the second track line can run parallel to one another and to the side of the vehicle. This can be recognized by methods of image processing. Image processing methods can also be used to identify which of the two lane lines is closer to the side of the vehicle. If this is the first lane line, the processor unit can select the first lane line and control the vehicle in such a way that it drives on or parallel to the first lane line. However, if the processor unit determines that the second lane line has a smaller distance from the vehicle than the first lane line, then it can select the second lane line and control the vehicle in such a way that it drives, for example, on or at a lateral distance parallel to the second lane line .

The present invention supports several different applications (e.g., mowing, swathing, loading or baling). Furthermore, the present invention enables track lines to be followed at a certain distance, which is particularly important with different working widths of attachments that can be attached to the vehicle. Furthermore, the present invention enables track lines to be reliably recognized. Also can

Swaths and cut edges can be recognized, for example, with a lateral distance of up to 8 m, the agricultural utility vehicle being able to be controlled in such a way that the named recognized objects are followed. The sensor unit is set up to detect a local area surrounding the vehicle. The local surroundings of the vehicle can be referred to as the region of interest (ROI). For example, the region of interest can be a field or part of a field. The field or the part of the field can be intended to be worked in particular for agriculture by the vehicle. For example, the vehicle or an attachment of the vehicle can have means that enable it to carry out a specific operation within the work area, for example mowing, swathing, loading or baling. These means can be integrated into the vehicle or attached to the vehicle as attachments. The recordings or frames produced when the surroundings are recorded result in the recording of the surroundings. For example, the detection of the surroundings can be images if a camera or a camera system is used as the sensor. Furthermore, the detection of the surroundings can be frames if, for example, a radar or a lidar is used. The recordings, in particular the images or frames, each cover a limited area around the vehicle. This is what the “local” feature means. The local area around the vehicle is thus a limited area that extends around the outside of the vehicle.

The range or extent of the local environment can vary depending on the type of sensor used and can be adjusted if necessary. The sensor system disclosed in the context of the present application is designed to be arranged on the vehicle, in particular to be attached, in such a way that it can detect the local surroundings of the vehicle. The area of the environment which the relevant sensor detects can also be referred to as the so-called “field of view”. This area can be one-dimensional, two-dimensional or three-dimensional, depending on the sensor used. It is possible that the area that can be detected by the sensor can detect part of the surroundings of the vehicle, for example a sector in the front area and / or in the side area and / or in the rear area of the vehicle. Furthermore, the relevant sensor can also be set up to record the entire surroundings of the vehicle, for example when using so-called surround view systems. The processor unit is set up to evaluate the detection of the surroundings. The detection of the surroundings can, for example, be a recording of the local surroundings of the vehicle. In particular, the processor unit is set up to extract objects from recordings or frames generated by the sensor unit. For example, the processor unit can evaluate an image from a camera of the sensor unit and recognize all traces within the image.

The multiple lane lines can be recognized by means of image processing methods, for example within an image that has been recorded by a camera of the sensor unit and that depicts the surroundings of the vehicle. Examples of suitable methods of image processing are, for example, semantic segmentation or edge detection. The result of using such image processing methods can in particular be a polynomial that describes the extent and course of the track lines.

In one embodiment, the processor unit is set up to control the vehicle based on at least one of the following parameters, namely based on a curvature of the selected lane line, based on a course error of the vehicle or based on a difference from one determined from the surroundings lateral distance of the selected lane line to the vehicle including its attachments and a desired distance from the selected lane line to the vehicle including its attachments.

In a further embodiment, the processor unit is set up to recognize the multiple track lines in the detection of the surroundings by combining methods of image processing with methods of artificial intelligence. An artificial neural network can be trained, for example by making representative sample images with track lines available to the artificial neural network for training it. The extension of the “classic image processing” with elements of artificial intelligence makes it possible in particular to recognize several lane lines and to continuously improve the degree of recognition and thus the reliability by training a neural network, for example. An output through the combination of classic image processing and artificial intelligence Polynomial can correctly describe the curvature of the track line, the course of the track line and the lateral distance between the track line and the off-road vehicle including its attachments with a particularly high probability. Together with the desired distance between the track line and the off-road vehicle, including its attachments (“desired lateral offset”), the vehicle's steering can be controlled.

For example, images or frames generated by the sensor unit can be processed by means of so-called semantic segmentation (in English: “Semantic segmentation”), which uses an artificial neural network. With semantic segmentation, individual image contents are segmented. Adjacent pixels can be combined to form image regions with coherent content if they meet a defined homogeneity criterion. In particular, a pixel-precise classification of the images can take place, it being possible for each pixel to belong to a known object class. For example, the processor unit can receive an image sequence from a camera which records the surroundings of the vehicle in successive images of the image sequence. In the images, the processor unit can extract features using semantic segmentation, for example, and assign each pixel to an object class in a classification, for example an object class “track line”, an object class “field without track line”, “sky” etc. The semantic segmentation method is suitable especially for complex applications and provides a particularly high level of accuracy.

Alternatively, images or frames generated by the sensor unit can be processed using the edge detection method that uses an artificial neural network. A filter can be adapted using the artificial neural network. With the edge detection method, flat areas in a digital image can be separated from one another, especially if they differ sufficiently in terms of color or gray value, brightness or texture along straight or curved lines. Special edge operators, also known as filters, can recognize transitions between flat areas and mark them as edges. In other words you can in the method of edge detection, edges are detected in a two-dimensional image, for example, which has been recorded by a camera, for example. Information relating to recognized edges can be used to recognize certain objects in the image, with reference to the present invention in particular for recognizing track lines in a field. For this purpose, for example, the information relating to recognized edges can be compared with stored patterns.

By recognizing the track lines, the system also enables control or automation of an attachment in the vehicle. For example, a loader wagon pickup (as an example of an attachment) can e.g. be lifted via a CAN signal when the end of a swath is detected by the sensor unit. Furthermore, the loading wagon pick-up can e.g. can be lowered via a CAN signal when the beginning of a swath is detected by the sensor unit. In this sense, it is provided according to a further embodiment that the processor unit is set up to control an attachment mounted on the vehicle based on the selected track line and its course in the environment detection. The desired distance to the track lines ("Desired Lateral Offset") can be defined by the width of the attachment. This can either be entered manually by the driver via a terminal or through automatic attachment recognition, e.g. by means of a tag on the attachment, the tag being recognized and processed directly by the processor unit or the driver assistance system.

The processor unit can be integrated into a driver assistance system of the vehicle or be communicatively connected to the driver assistance system. The processor unit can be set up to initiate that the vehicle is transferred to a safe state when the processor unit detects at least one potential collision object in the surroundings detection. A potential collision object can be an object (stationary or movable, for example a tree, a power pole, a pole or another vehicle), a person or an animal that is located within the detected local area surrounding the vehicle. In this context, “initiate” can in particular be understood to mean that the processor unit a command is transmitted to the driver assistance system of the vehicle so that the driver assistance system transfers the vehicle to the safe state. For example, the driver assistance system can bring the vehicle to a standstill when it is brought into the safe state. Alternatively, the speed can be reduced (without having to stop the vehicle) so that the risk of collision can be significantly reduced. Furthermore, an optical or acoustic warning can be output, in particular to the user of the vehicle (for example via a display inside the vehicle) and / or to the local area around the vehicle, for example by a horn, a horn, a loudspeaker or by a Vehicle lighting system. The visual or acoustic warning can in particular be perceived by people and animals who are in the vicinity of the vehicle. In this way, people in the vicinity can be warned of the autonomously driving vehicle and collisions can be avoided.

The sensor unit for detecting the surroundings of the vehicle can comprise at least one of the following sensors, namely an image processing sensor, in particular a camera, a radar sensor or a laser-based sensor.

The image processing sensor (e.g. a camera) can be set up to record images of the surrounding area. Using known methods of image processing and image evaluation, features can be recognized in the images, in particular the track lines.

The radar-based sensor can be set up to recognize features in the detected surroundings of the vehicle, in particular the lane lines. The radar-based sensor can, for example, measure distances to objects within the detected surroundings. Furthermore, the radar-based sensor can also measure azimuth values, altitude values (elevation), intensity values and radial velocity values, for example. The radar sensor can be used to determine the incline. By including altitude values, it is possible in particular to determine a slope of the terrain which is present in the detected surroundings of the vehicle. In particular, the sensor unit can include at least one camera which detects track lines in a field of view of the camera, and in addition the sensor can Sensor unit comprise a radar-based sensor which is set up to determine the slope of the terrain in a field of view of the radar-based sensor. The fields of view of the camera and the radar-based sensor can be congruent, overlap or not overlap (complementary fields of view). A corresponding measurement cycle in which the radar-based sensor recorded the surroundings of the vehicle or measured it in the manner described can be referred to as a “frame”. The radar-based sensor can therefore scan or detect the environment in an N-dimensional manner, as a result of which point clouds can be generated. The radar-based sensor can extract features from recorded point clouds. The point cloud can accordingly comprise several dimensions (N-dimensional point cloud) if, for example, intensities and radial speeds are also taken into account.

The laser-based sensor (e.g. a lidar sensor) can be set up to recognize features in the detected surroundings of the vehicle, in particular the lane lines. The laser-based sensor can, for example, measure intensities in an x-direction, in a y-direction and in a z-direction of a Cartesian coordinate system of the laser-based sensor within the detected surroundings. A corresponding measuring cycle in which the laser-based sensor has recorded the environment or measured it in the described manner can be referred to as a “frame”. The laser-based sensor can scan or detect the environment N-dimensionally, whereby point clouds can be generated. The laser-based sensor can extract features from captured point clouds. The point cloud can accordingly comprise several dimensions (N-dimensional point cloud).

In addition to the automatic selection of the track line by the processor unit, a selection of a track line to be followed, eg a swath to be followed or a cutting edge to be followed, can be controlled by a driver or user of the agricultural utility vehicle via an operating aid. In this sense, the system comprises, in a further embodiment, an operating unit which is set up to enable a user to select a track line from the multiple track lines. The operating unit can comprise a display, for example a touch screen. The display can detect the plural ones by the sensor unit Show track lines. The user can select the lane line that the vehicle should follow autonomously from the multiple lane lines shown.

In the following, exemplary embodiments of the invention are explained in more detail with reference to the schematic drawings, identical or similar elements being provided with the same reference symbols. Here shows

1 shows a plan view of a field on which a tractor with an attachment drives autonomously,

FIG. 2 elements of a driver assistance system for the tractor according to FIG. 1,

3 shows steps of a method for detecting track lines and for controlling the tractor according to FIG. 1 based on a selected track line by means of the driver assistance system according to FIG. 2, and

FIG. 4 scenarios A to F, in which the driver assistance system according to FIG. 2 is active or passive in a T raktor.

1 shows an agricultural machine in the form of a tractor 1. In the rear area of the tractor 2, an attachment 2 is attached to the tractor 1. The tractor 1 pulls the attachment over a field 3 so that the attachment 2 can work on the field 3 and so that several track lines are created, in the exemplary embodiment shown, a first track line 4.1 and a second track line 4.2. The first track line 4.1 runs parallel to the second track line 4.2. The tractor 1 is located with the attachment 2 laterally spaced apart from the first track line 4.1. The second track line 4.2 is further away from the vehicle 1 and the attachment 2 than the first track line 4.1.

The vehicle 1 comprises a system 5, shown in more detail in FIG. 2, for the autonomous operation of the vehicle 1 (in certain situations / scenarios, see FIG. 4). In the exemplary embodiment shown, the system 5 comprises a processor unit 6, a memory unit 7, a communication interface 8 and a sensor unit 9. In the exemplary embodiment shown, the sensor unit 9 comprises a digital camera system which is referred to below as “camera” and is provided with the reference number 11 in FIG. 2. The camera 11 is arranged in particular on a roof of the tractor 1 so that it covers an environment 12 of the tractor 1 that is located in a forward area and on both sides of the tractor 1. However, this is purely exemplary. Alternatively, the digital camera system 1 1 can also, for example, be a

Surround view system that can capture an environment 12 that extends 360 ° around the tractor 1. The camera 11 successively and continuously records images of the surroundings 12 of the tractor 1, as a result of which a video or an image sequence of the surroundings 12 of the tractor 1 is generated. This video or this sequence of images is the detection of the surroundings of tractor 1. Within the individual images of this video or this image sequence, image sections can be defined (regions of interest, or ROI for short), within which the track lines 4.1 and 4.2 can be recognized using image recognition methods. Methods of artificial intelligence or machine learning can optionally be used. For example, semantic segmentation can be selected as the method of image recognition, whereby an artificial neural network is used (e.g. a convolutional neural network, or CNN for short) that is trained with reference images. In addition to the camera 9, the sensor unit 10 can have a radar-based sensor 13. The radar-based sensor 13 is arranged on the tractor 1 in such a way that it can determine altitude values which allow a calculation of a slope of the terrain which is present in the detected surroundings 12 of the tractor 1.

A computer program product 10 can be stored on the storage unit 7. The computer program product 10 can be executed on the processor unit 6, for which purpose the processor unit 6 and the memory unit 7 are connected to one another in a correspondingly communicative manner. When the computer program product 10 is executed on the processor unit 6, it instructs the processor unit 6 to carry out the functions or method steps described in connection with the drawing.

The tractor 1 can be assigned a coordinate system with an x-axis (longitudinal direction) and a y-axis (cross-direction or width direction) running transversely thereto. net that can be viewed as a reference coordinate system. A camera coordinate system can be assigned to each image recorded by the camera 11. Fig. 1 shows that this coordinate system can have a longitudinal axis L of the Ge voltage from tractor 1 and attachment 2. In the example according to FIG. 1, the longitudinal axis L of the image coordinate system coincides with the x-axis of the tractor coordinate system. The image coordinate system also has a width axis B of the attachment 1, which extends transversely to the longitudinal axis L and which extends laterally in width direction B than the tractor 1. In the exemplary embodiment according to FIG. 1, the width axis L does not match the y- Axis of the tractor coordinate system, but there is a longitudinal offset xO in the longitudinal direction x. The processor unit 6 is set up to take this offset into account by converting determined image coordinates into tractor coordinates. The processor unit 6 is also set up to take into account an offset in the width direction y by converting determined image coordinates into tractor coordinates (even if there is no offset in the width direction y in the present example according to FIG. 1).

The width axis B can for example run through the widest portion of the attachment 2, as shown by FIG. According to the exemplary embodiment according to FIG. 1, the wider attachment 2 should have a desired lateral offset (“desired lateral offset”, “target distance”) from a nearest track line (in the exemplary embodiment according to FIG. 1 this is the first track line 4.1), while the combination of tractor 1 and attachment 2 moves on field 3 and works on field 3 by means of attachment 2. This laterally desired distance can be a few meters, for example 8 m. For example, in a first method step 100 (FIG. 3), a distance BO can be defined which the closest track line 4.1 should have at least from the longitudinal axis L of the combination of tractor 1 and attachment 2. The desired lateral distance BO can be present as a distance data record to which the processor unit 6 has access. For example, the attachment 2 can comprise a tag, for example an RFID tag, which can be read out by the driver assistance system 5. The distance data record can be stored on the day. Furthermore, the distance data record can also be replaced by a User or a driver of the tractor 1 can be generated in the driver assistance system 5, for example by input via a corresponding terminal of the tractor 1.

In a second method step 200 (FIG. 3), the camera 11 records an image of the surroundings of the tractor 1. The second step 200 can be carried out after the first step 100. Alternatively, however, steps 100 and 200 can also be carried out at the same time or with a time overlap. In a third method step 300 (FIG. 3), the processor unit 6 can evaluate the image recorded by the camera 11 and recognize the track lines 4.1 and 4.2, whereby methods of artificial intelligence can be used as mentioned above.

This creates a classified image from the camera.

In a subsequent step 400 (FIG. 3), a polynomial fitting can be carried out. With polynomial fitting, the image content is further processed and polynomials are calculated from individual information (points, broken lines, etc.) that define a track line and its direction. The result of this fourth method step 400 are thus polynomials in the image coordinate system (L, B). In a fifth method step 500 (FIG. 3), the polynomials that have previously been calculated in the image coordinate system can be transformed into coordinates of the tractor coordinate system. The result of this fifth method step 500 are thus polynomials in the tractor coordinate system (x, y).

The track lines 4.1 and 4.2 can be swaths or cut edges, for example. The first track line 4.1 has a first lateral distance (first “lateral offset” / first “actual distance”) B1 in the width direction B from the longitudinal axis L of the combination of tractor 1 and attachment 2. The second track line 4.2 has a second lateral distance (second “lateral offset”; second “actual distance”) B2 to the longitudinal axis L of the combination of tractor 1 and attachment 2 in the width direction B. The first lateral distance B1 is smaller than the second lateral distance B2, ie the first track line 4.1 is closer to the longitudinal axis L of the combination of tractor 1 and attachment 2 in the width direction B than the second track line 4.2. In a further method step 600 (FIG. 3), the processor unit 6 can evaluate the image to determine which of the two track lines 4.1 and 4.2 has a smaller lateral distance (in the width direction B) to the combination of tractor 1 and attachment 2. In the exemplary embodiment according to FIG. 1, the processor unit 6 will determine that the first track line 4.1 has the smaller lateral distance to the combination of tractor 1 and attachment 2 than the second track line 4.2 (B1 <B2). The processor unit 6 will select the first track line 4.1 accordingly.

Based on the selected track line 4.1, the processor unit 6 can control the tractor 1 in a seventh method step 700 such that the combination of tractor 1 and attachment 2 moves autonomously along the selected first track line 4.1. The attachment 2 can pursue its main function, namely to work the field 3 and for example generate a new track line next to the first track line 4.1 or collect the first track line 4.1.

In the example shown by FIG. 1, the first lateral actual distance B1 is smaller than the lateral setpoint distance BO. However, it is required or desired that the actual distance B1 is at least as great as the setpoint distance BO. The processor unit 6 can determine this and, for example, have a corrective effect on the steering of the tractor 1 in such a way that the combination of tractor 1 and attachment 2 moves away from the first track line 4.1 (to the left according to the view in FIG. 1) that the distance B1 of the longitudinal axis L rises from the first track line 4.1 in subsequent images recorded by the camera 11 and that the target criterion B1> BO is met. If the value of the actual distance B1 and the target distance BO match, the steering accuracy provided by the driver assistance system 5 is high and the tractor 1 follows the track line 4.1 perfectly at the distance BO maintained. The greater the deviation between the desired distance BO and the actual distance B1, the less precise the control or regulation by the driver assistance system 5.

In the example shown by FIG. 1, the track lines 4.1 and 4.2 each have a curvature. The processor unit 6 is set up to determine the curvature of the track lines 4.1 and 4.2. For example, as shown by FIG. 1, the processor unit 6 can apply a tangent T to the first track line 4.1 where the latitude axis B intersects the first track line 4.1. This tangent T can be so be moved so that it lies at the intersection between the longitudinal axis L and the latitude axis B. The processor unit 6 can calculate an angle D between this shifted tangent T ′ and the longitudinal axis L. The angle D can be viewed as a heading error of the tractor 1. The angle D can be caused, for example, by drift, wind, traction problems or centrifugal forces. If the tractor 1 were to continue to move straight ahead in the direction of the longitudinal axis L, then the tractor 1 and the attachment 2 would approach the first track line 4.1 and possibly drive over it. However, in order to follow the curved first track line 4.1 with the lateral target distance BO, the processor unit 6 can, for example, have a corrective effect on the steering of the tractor 1 in such a way that the course of the shifted tangent T is subsequently recorded by the camera 11 Bil countries approximates the course of the longitudinal axis L, and that the course error in the form of the angle D is minimized.

The processor unit 6 can furthermore be set up to initiate that the tractor is brought into a safe state (e.g. stopped) when the processor unit 6 detects at least one potential collision object 15 in the detection of the surroundings, e.g. a person, stake, or other vehicle. Alternatively, a warning can also be given to the driver of the actuator 1 or its surroundings 12.

4 shows that the driver assistance system 5 of a tractor 1 can then be active and control the tractor 1 as described above when the tractor 1 is driving on the field 3 along the lane line 4 (scenarios A, B and C). If problems arise with the detection of the lane lines 4, the driver assistance system 5 transfers control of the control of the tractor 1 back to the driver of the tractor 1. Even while the driver is turning the tractor 1 at a headland 14, the driver assistance system 5 transfers control of the control of the tractor 1 back to the driver of the tractor 1. Furthermore, the driver assistance system 5 passes control of the control of the tractor 1 back to the driver of the tractor 1 if the driver does not turn at the headland 14 or takes a completely wrong course (scenarios D, E and F), because in these cases For example, no track line 4 in the vicinity of the tractor 1 can be detected by the sensor unit of the tractor 1 and the conditions for controlling the tractor 1 can therefore not be met.

Reference symbol

L Longitudinal axis tractor and attachment in the image coordinate system

B Latitude axis of attachment in the image coordinate system

BO Desired Lateral Offset

B1 Lateral offset to the first track line

B2 Lateral offset to the second track line

xO longitudinal offset in the tractor coordinate system

x Tractor longitudinal axis in the tractor coordinate system

y Cross axis / latitude axis in the tractor coordinate system

1 tractor

2 attachment

3 field

4.1 first track line

4.2 second track line

5 driver assistance system

6 processor unit

7 storage unit

8 communication interface

9 sensor unit

10 computer program product

1 1 digital camera system

12 The area around the tractor

13 Radar-based sensor

14 headlands

15 potential collision object

100 process step

200 process step

300 process step

400 process step

500 process step

600 process step

700 process step

Claims

1 . Processor unit (6) for the autonomous operation of a vehicle (1), the processor unit (6) comprising an interface (8),

wherein the processor unit (6) is set up to

- Using the interface (8) to access an environment detection of the vehicle (1) generated by a sensor unit (9),

- Recognize several track lines (4.1, 4.2) in the detection of the surroundings,

- select a track line (4.1) from the several track lines (4.1, 4.2), and

- To control the vehicle (1) based on the selected track line (4.1) in such a way that the vehicle (1) autonomously follows the selected track line (4.1).

2. processor unit (6) according to claim 1, wherein the processor unit (6) is set up to control the vehicle (1) based on at least one of the following Parame ter, namely

- a curvature of the selected track line (4.1),

- a course error of the vehicle (1),

- a difference between a lateral distance (B1), determined from the detection of the surroundings, between the selected track line (4.1) and the vehicle (1) including its attachments (2) and a desired distance (BO) between the selected track line (4.1) and the vehicle (1) ) including its attachments (2).

3. processor unit (6) according to any one of the preceding claims, wherein the processor unit (6) is set up to recognize the multiple lane lines (4.1, 4.2) in the environment detection using methods of image processing combined with methods of artificial intelligence.

4. processor unit (6) according to claim 3, wherein the method of semantic segmentation is combined with an artificial neural network.

5. processor unit (6) according to claim 3 or 4, wherein the method of edge detection is combined with an artificial neural network.

6. processor unit (6) according to any one of the preceding claims, wherein the processor unit (6) is set up to capture an attachment (2) mounted on the vehicle (1) based on the selected track line (4.1) and its course in the surroundings to control.

7. processor unit (6) according to any one of the preceding claims, wherein the processor unit (6) is set up to

- to check whether the environment detection depicts at least one potential collision object, and

- To transfer the vehicle (1) to a safe state or to reduce the speed of the vehicle (1) or to output an optical or acoustic warning to the user of the vehicle (1) or to the surroundings of the vehicle (1), if the environment detection depicts at least one potential collision object.

8. processor unit (6) according to any one of the preceding claims, wherein the track line (4.1, 4.2)

- a swath,

- a cut edge,

- a mowing edge,

- one lane,

- a row of plants,

- a row of trees,

- a curb or

- Includes a work path (15).

9. Driver assistance system (5) for the autonomous operation of a vehicle (1), comprising the driver assistance system (5)

- A sensor unit (9), wherein the sensor unit (9) is set up to generate a field detection of the vehicle (1), and

- A processor unit (6) according to one of the preceding claims.

10. The driver assistance system (5) according to claim 9, wherein the sensor unit (9) for detecting the surroundings (12) of the vehicle (1) comprises at least one of the following sensors, namely

- An image processing sensor, in particular a camera (1 1), preferably a stereo camera,

- a radar sensor (13),

- a laser-based sensor.

1 1. Driver assistance system (5) according to one of claims 9 or 10, the driver assistance system (5) further comprising an operating unit which is set up to enable a user to select a track line (4.1) from the multiple track lines (4.1, 4.2).

12. Method for autonomous operation of a vehicle (1), the method comprising the steps:

- Generating a detection of the surroundings of the vehicle (1) by a sensor unit (9),

- Recognition of several track lines (4.1, 4.2) in the detection of the surroundings by means of a processor unit (6),

- Selecting a track line (4.1) from the multiple track lines (4.1, 4.2) by means of the processor unit (6), and

- Controlling the vehicle (1) by means of the processor unit (6) based on the selected track line (4.1), so that the vehicle (1) autonomously follows the selected track line (4.1).

13. Computer program product (10) for the autonomous operation of a vehicle (1), the computer program product (10) instructing the processor unit (6) when it is executed on a processor unit (6),

- Using an interface (8) to access an environment detection of the vehicle (1) generated by a sensor unit (9),

- select a track line (4.1) from the several track lines (4.1, 4.2), and

14. Vehicle (1) comprising a processor unit (6) according to one of claims 1 to 7 or a driver assistance system (5) according to one of claims 8 to 11.