DE102019203247A1 - Vision-based steering assistance system for land vehicles - Google Patents

Abstract

The invention relates to the autonomous operation of a vehicle (1). For example, a processor unit for the autonomous operation of a vehicle (1) is proposed. The processor unit comprises an interface, the processor unit being set up to use the interface to access a detection of the surroundings of the vehicle (1) generated by a sensor unit, to recognize several lane lines (4.1, 4.2) in the detection of the surroundings, a lane line (4.1) from the to select several track lines (4.1, 4.2), and to control the vehicle (1) based on the selected track line (4.1) in such a way that the vehicle (1) autonomously follows the selected track line (4.1). Further claims are directed to a corresponding driver assistance system (5), a method and a computer program product and a vehicle (1).

Description

The invention relates to a processor unit, a driver assistance system, a method and a computer program product for the autonomous operation of a vehicle, in particular an agricultural utility vehicle.

Driver assistance systems for agricultural utility vehicles or machines (agricultural engineering) are known. These driver assistance systems are intended to relieve the driver in particular as a fully automated or partially automated steering aid. The steering aid via GPS control has established itself in agricultural engineering. The driver assistance systems are sometimes very complex and, depending on the correction signal, enable accuracies of up to 2 cm. Problems with GPS are the partially non-existent signal coverage worldwide (e.g. also in mountainous regions) and the high acquisition and license costs, depending on the system.

Alternative driver assistance systems for agricultural vehicles that are not based on GPS provide optical systems that automate certain applications in agriculture. Examples are driver assistance systems based on cameras, lidar sensors or, more generally, laser sensors. However, such systems are complex and expensive (in particular when lidar sensors are used) and / or can only carry out or support a simple application. For example, these systems typically only allow a swath to be recognized centrally in front of the vehicle and not to the side of the vehicle. A “swath” can be understood as an agricultural crop that has been mown and lined up in a row, e.g. Grass, grain, etc. Furthermore, such systems are not able to process several applications (e.g. mowing, swathing, loading or baling) and thus do not offer a real alternative to GPS systems.

A driver of an agricultural utility vehicle typically has to concentrate on one operation for a whole day. In addition, in addition to controlling the vehicle, the driver usually has to simultaneously monitor the function of attachments in the vehicle. This can lead to incorrect operation, the machines not being used optimally and efficiently and the work performance and quality of work suffer.

Expensive GPS systems are often not profitable for small or medium-sized farms or are useless in areas with poor signal coverage. Optical assistance systems have so far been very limited in their function and can often only implement very simple applications.

One object of the present invention can be seen as providing a technology that is an alternative to expensive GPS systems in particular, which takes the burden off the driver and offers him more comfort in various applications (e.g. mowing, swathing, loading or baling).

The object is achieved by the subjects of the independent claims. Advantageous embodiments are the subject of the dependent claims, the following description and the figures.

According to the present invention, a technology is proposed which detects track lines (e.g. swaths or mowed edges) in a field, processes corresponding image content, selects the correct track and then controls the steering of the vehicle accordingly. A core of the invention is a system of simple sensors (e.g. cameras) and state-of-the-art image processing (expanded in particular by methods of artificial intelligence, see below), which enables the system to be designed and used flexibly. Depending on the type of application and the area of application, the system can be implemented in different ways and thus adapted to customer requirements.

In particular, according to the present invention, three functionalities or method steps are proposed, namely the recognition of the lane lines ("seeing" or in English: "see"), the selection of the correct lane line ("thinking" or in English: "think") and the control of the steering of the vehicle ("act"). The present invention makes it possible to relieve the driver and to offer him more comfort in various applications (e.g. mowing, swathing, loading or baling). The agricultural utility vehicle can thus be used more efficiently. The efficiency increases, work performance and quality of work improve significantly and fuel consumption decreases.

In this sense, according to a first aspect of the invention, a processor unit for the autonomous operation of a vehicle is provided. The processor unit can form an element of an electronic control unit of the vehicle. The processor unit can also form an element of a driver assistance system of the vehicle. The processor unit includes an interface. The interface is, in particular, a communication interface which enables the exchange of data, specifically an exchange of data between the processor unit on the one hand and a sensor unit on the other. Furthermore, the processor unit can transmit commands to actuators of the vehicle via the interface, for example to a steering actuator of the vehicle. The processor unit is set up in particular to use the interface to access a surrounding area detection of the vehicle generated by the sensor unit, to recognize several lane lines in the surrounding area detection, to select a lane line from the several lane lines and to control the vehicle based on the selected lane line in such a way that the vehicle autonomously follows the selected lane line, eg that the vehicle drives on or to the side of the selected lane line. If the sensor unit only detects a single lane line in the vicinity of the vehicle, then the processor unit is also set up to recognize this single lane line and to control the vehicle in such a way that it follows it.

According to a second aspect of the invention, a driver assistance system for the autonomous operation of a vehicle is proposed. The driver assistance system comprises a sensor unit for detecting the surroundings of the vehicle and a processor unit according to the first aspect of the invention. The sensor unit is set up to generate a detection of the surroundings of the vehicle.

According to a third aspect of the invention, a corresponding method for the autonomous operation of a vehicle is proposed. The method can include generating a detection of the surroundings of the vehicle by means of a sensor unit and recognizing a plurality of lane lines in the detection of the surroundings by means of a processor unit. Furthermore, a lane line can be selected from the multiple lane lines by means of the processor unit, and the vehicle can be controlled by means of the processor unit based on the selected lane line, so that the vehicle autonomously follows the selected lane line, e.g. that the vehicle drives on or to the side of the selected track line.

According to a fourth aspect of the invention, a computer program product for the autonomous operation of a vehicle is also provided. When it is executed on a processor unit, the computer program product instructs the processor unit to use an interface to access an environment detection of the vehicle generated by a sensor unit, to recognize multiple lane lines in the area detection, to select a lane line from the multiple lane lines, and based the vehicle to steer on the selected lane line in such a way that the vehicle autonomously follows the selected lane line, e.g. that the vehicle drives on or to the side of the selected track line.

According to a fifth aspect of the invention, a vehicle is provided, in particular an off-road vehicle. The vehicle can comprise a processor unit according to the first aspect of the invention. Alternatively or additionally, the vehicle can comprise a driver assistance system according to the second aspect of the invention.

The following statements apply equally to the processor unit according to the first aspect of the invention, for the driver assistance system according to the second aspect of the invention, for the method according to the third aspect of the invention, for the computer program product according to the fourth aspect of the invention and for the vehicle according to the fifth aspect of the invention.

The vehicle can be controlled by means of the processor unit. Under “controlled” or “control” can be understood in particular that the vehicle can be operated autonomously, i.e. In particular, it can be automatically steered, accelerated and braked, including all the necessary controls of actuators, in particular of the drive train, of the vehicle's steering and signaling. The control of the vehicle by means of the processor unit can be activated automatically or by a user of the vehicle by means of an operating element when the vehicle is traveling along a lane. If the vehicle turns at a headland of the lane line (s), the control of the vehicle can be deactivated automatically or by the user of the vehicle by means of an operating element.

A track line can, for example, be a swath or a mowing edge or a cutting edge on an agricultural area, e.g. a field. Furthermore, the lane line can comprise a lane, a row of plants, a row of trees, a curb or a work path. The processor unit can not only recognize swaths and mowed edges as track lines and instruct the vehicle to follow them, but also other applications in agriculture, in the municipal sector or in the construction machinery business can be successfully automated with simple application effort. For example, rows of maize, apple trees in plantations, vines in vineyards, cut edges of a combine harvester in grain, chopped crops (soybeans, sugar beets, etc.), potato ridges or curbs for road sweepers when mowing roadsides can be recognized. The track line can, in particular, be located in a field and, for example, have been caused by previous sowing. Plant rows are rows of the cultivated product in question, e.g. Corn or grapevines. Rows of trees are for example on orchards, e.g. Apple orchards. For example, commutes to work can be found in mining (especially in opencast mining).

The vehicle can in particular be an off-road vehicle. An off-road vehicle can be understood to mean a vehicle whose primary area of use is not a road (as is the case, for example, with passenger cars, buses or trucks), but for example an agricultural area, e.g. a field to be worked / existing or a forest area, or a mining area (in particular open-cast mining), or an industrial area, for example within a production facility or warehouse. For example, the off-road vehicle can be an agricultural utility vehicle or an agricultural tractor such as a combine harvester or tractor. Furthermore, the vehicle can be an industrial truck, for example a forklift or a tractor. Furthermore, the vehicle can also be a municipal vehicle, for example a street sweeper, a garbage truck or a snow clearing vehicle.

In particular, the processor unit is set up to select that lane line which is the smallest distance from the vehicle. For example, the detection of the surroundings can comprise several camera images recorded one after the other (image sequence), which depict the surroundings of the vehicle. For example, a first track line and a second track line can be recognized in one of these camera images. The first track line and the second track line can run parallel to one another and to the side of the vehicle. This can be recognized by methods of image processing. Image processing methods can also be used to identify which of the two lane lines is closer to the side of the vehicle. If this is the first lane line, the processor unit can select the first lane line and control the vehicle in such a way that it drives on or parallel to the first lane line. However, if the processor unit determines that the second lane line is closer to the vehicle than the first lane line, then it can select the second lane line and control the vehicle such that it drives, for example, on or at a lateral distance parallel to the second lane line.

The present invention supports several different applications (e.g., mowing, swathing, loading or baling). Furthermore, the present invention enables track lines to be followed at a certain distance, which is particularly important in the case of different working widths of attachments that can be attached to the vehicle. Furthermore, the present invention enables track lines to be reliably recognized. In addition, swaths and cut edges can be recognized, for example with a lateral distance of up to 8 m, wherein the agricultural utility vehicle can be controlled in such a way that the named recognized objects are followed.

The sensor unit is set up to detect a local area surrounding the vehicle. The local surroundings of the vehicle can be referred to as the region of interest (ROI). For example, the region of interest can be a field or part of a field. The field or the part of the field can be intended to be worked in particular for agriculture by the vehicle. For example, the vehicle or an attachment to the vehicle may have means enabling it to perform a specific operation within the work area, e.g. Mowing, swathing, loading or baling. These means can be integrated into the vehicle or attached to the vehicle as attachments. The recordings or frames produced when the surroundings are recorded result in the recording of the surroundings. For example, the detection of the surroundings can be images if a camera or a camera system is used as the sensor. Furthermore, the detection of the surroundings can be frames if, for example, a radar or a lidar is used. The recordings, in particular the images or frames, each cover a limited area around the vehicle. This is what is meant by the “local” feature. The local environment of the vehicle is thus a limited area that extends around the outside of the vehicle.

The range or extent of the local environment can vary depending on the type of sensor used and can be adjusted if necessary. The sensor system disclosed in the context of the present application is designed to be arranged on the vehicle, in particular to be attached, in such a way that it can detect the local surroundings of the vehicle. The area of the environment which the relevant sensor detects can also be referred to as the so-called “field of view”. This area can be one-dimensional, two-dimensional or three-dimensional, depending on the sensor used. It is possible that the area detectable by the sensor can detect part of the surroundings of the vehicle, e.g. a sector in the front area and / or in the side area and / or in the rear area of the vehicle. Furthermore, the relevant sensor can also be set up to detect the entire surroundings of the vehicle, e.g. when using so-called surround view systems.

The processor unit is set up to evaluate the detection of the surroundings. The detection of the surroundings can be, for example, a recording of the local surroundings of the vehicle. In particular, the processor unit is set up to use recordings or frames generated by the sensor unit have been to extract objects. For example, the processor unit can evaluate an image from a camera of the sensor unit and in doing so recognize all traces within the image.

The multiple lane lines can be recognized by means of image processing methods, for example within an image which has been recorded by a camera of the sensor unit and which depicts the surroundings of the vehicle. Examples of suitable methods of image processing are, for example, semantic segmentation or edge detection. The result of using such image processing methods can in particular be a polynomial that describes the extent and course of the track lines.

In one embodiment, the processor unit is set up to control the vehicle based on at least one of the following parameters, namely based on a curvature of the selected lane line, based on a course error of the vehicle or based on a difference between a lateral distance of the determined from the surroundings detection selected lane line to the vehicle including its attachments and a desired distance of the selected lane line to the vehicle including its attachments.

In a further embodiment, the processor unit is set up to recognize the multiple lane lines in the detection of the surroundings by combining methods of image processing with methods of artificial intelligence. An artificial neural network can be trained, e.g. in that representative sample images with track lines are made available to the artificial neural network for its training. The extension of the "classic image processing" by elements of artificial intelligence enables in particular to recognize several lane lines and to continuously improve the degree of recognition and thus the reliability by training a neural network, for example. A polynomial output through the combination of classic image processing and artificial intelligence can correctly describe the curvature of the track line, the course of the track line and the lateral distance of the track line to the off-road vehicle including its attachments with a particularly high probability. The steering of the vehicle can be controlled together with the desired distance between the track line and the off-road vehicle including its attachments (“desired lateral offset”).

For example, images or frames generated by the sensor unit can be processed by means of so-called semantic segmentation (in English: “Semantic segmentation”), which uses an artificial neural network. With semantic segmentation, individual image contents are segmented. Adjacent pixels can be combined to form an image region with a coherent content if they meet a defined homogeneity criterion. In particular, a pixel-precise classification of the images can take place, it being possible for each pixel to belong to a known object class. For example, the processor unit can receive an image sequence from a camera which records the surroundings of the vehicle in successive images of the image sequence. In the images, the processor unit can extract features using semantic segmentation, for example, and assign each pixel to an object class in a classification, for example an object class “track line”, an object class “field without track line”, “sky” etc. The semantic segmentation method is particularly suitable for complex applications and delivers particularly high accuracy.

Alternatively, images or frames generated by the sensor unit can be processed using the edge detection method that uses an artificial neural network. A filter can be adapted by means of the artificial neural network. With the edge detection method, flat areas in a digital image can be separated from one another, especially if they differ sufficiently in color or gray value, brightness or texture along straight or curved lines. Special edge operators, also known as filters, can recognize transitions between two-dimensional areas and mark them as edges. In other words, with the edge detection method, edges can be detected in an e.g. two-dimensional image which e.g. was recorded by a camera. Information relating to recognized edges can be used for recognizing certain objects in the image, with reference to the present invention in particular for recognizing track lines in a field. For this purpose, for example, the information relating to recognized edges can be compared with stored patterns.

By recognizing the track lines, the system also enables control or automation of an attachment in the vehicle. For example, a loading wagon pick-up (as an example of an attachment) can be lifted, for example via a CAN signal, when the end of a swath is detected by the sensor unit. Furthermore, the loading wagon pickup can be lowered, for example via a CAN signal, when the sensor unit detects the beginning of a swath. In this sense, it is provided according to a further embodiment that the processor unit is set up to a to control the attachment attached to the vehicle based on the selected track line and its course in the environment detection. The desired distance to the track lines ("Desired Lateral Offset") can be defined by the width of the attachment. This can either be entered manually by the driver via a terminal or by automatic attachment recognition, for example by means of a tag on the attachment, the tag being recognized and processed directly by the processor unit or the driver assistance system.

The processor unit can be integrated into a driver assistance system of the vehicle or be communicatively connected to the driver assistance system. The processor unit can be set up to initiate that the vehicle is transferred to a safe state when the processor unit detects at least one potential collision object in the surroundings detection. A potential collision object can be an object (stationary or movable, e.g. a tree, a power pole, a pole or another vehicle), a person or an animal that is located within the detected local area around the vehicle. In this context, “initiate” can in particular be understood to mean that the processor unit transmits a command to the driver assistance system of the vehicle so that the driver assistance system transfers the vehicle to the safe state. For example, the driver assistance system can bring the vehicle to a standstill when it is brought into the safe state. Alternatively, the speed can also be reduced (without the vehicle having to be stopped), so that the risk of collision can be significantly reduced. Furthermore, an optical or acoustic warning can be issued, in particular to the user of the vehicle (e.g. via a display inside the vehicle) and / or to the local area surrounding the vehicle, e.g. by a horn, a horn, a loudspeaker or by a lighting device of the vehicle. The visual or acoustic warning can in particular be perceived by people and animals who are in the vicinity of the vehicle. In this way, people in the vicinity can be warned of the autonomously driving vehicle and collisions can be avoided.

The sensor unit for detecting the surroundings of the vehicle can comprise at least one of the following sensors, namely an image processing sensor, in particular a camera, a radar sensor or a laser-based sensor.

The image processing sensor (e.g. a camera) can be set up to record images of the surrounding area. Using known methods of image processing and image evaluation, features can be recognized in the images, in particular the track lines.

The radar-based sensor can be set up to recognize features in the detected surroundings of the vehicle, in particular the lane lines. The radar-based sensor can, for example, measure distances to objects within the detected surroundings. Furthermore, the radar-based sensor can also measure azimuth values, altitude values (elevation), intensity values and radial velocity values, for example. The radar sensor can be used to determine the slope. By including altitude values, it is possible in particular to determine a slope of the terrain which is present in the detected surroundings of the vehicle. In particular, the sensor unit can include at least one camera that detects lane lines in a field of view of the camera, and the sensor unit can additionally include a radar-based sensor that is set up to measure the gradient of the in a field of view of the radar-based sensor To determine the terrain. The fields of view of the camera and the radar-based sensor can be congruent, overlap or not overlap (complementary fields of view). A corresponding measurement cycle in which the radar-based sensor recorded the surroundings of the vehicle or measured it in the manner described can be referred to as a “frame”. The radar-based sensor can therefore scan or detect the environment in an N-dimensional manner, as a result of which point clouds can be generated. The radar-based sensor can extract features from recorded point clouds. The point cloud can accordingly comprise several dimensions (N-dimensional point cloud) if, for example, intensities and radial velocities are also taken into account.

The laser-based sensor (e.g. a lidar sensor) can be set up to recognize features in the detected surroundings of the vehicle, in particular the lane lines. The laser-based sensor can, for example, measure intensities in an x-direction, in a y-direction and in a z-direction of a Cartesian coordinate system of the laser-based sensor within the detected surroundings. A corresponding measuring cycle in which the laser-based sensor has recorded the environment or measured it in the described manner can be referred to as a “frame”. The laser-based sensor can scan or detect the environment N-dimensionally, whereby point clouds can be generated. The laser-based sensor can extract features from recorded point clouds. The point cloud can accordingly comprise several dimensions (N-dimensional point cloud).

In addition to the automatic selection of the track line by the processor unit, a Selection of a track line to be followed, for example a swath to be followed or a cutting edge to be followed, can be controlled by a driver or user of the agricultural utility vehicle via an operating aid. In this sense, the system comprises, in a further embodiment, an operating unit which is set up to enable a user to select a track line from the multiple track lines. The operating unit can comprise a display, for example a touch screen. The display can show the multiple track lines detected by the sensor unit. The user can select the lane line that the vehicle is to autonomously follow from the multiple lane lines shown.

• 1 eine Draufsicht auf ein Feld, auf welchem ein Traktor mit einem Anbaugerät autonom fährt,
• 2 Elemente eines Fahrerassistenzsystems für den Traktor nach 1,
• 3 Schritte eines Verfahrens zur Detektion von Spurlinien und zur Steuerung des Traktors nach 1 basierend auf einer ausgewählten Spurlinie mittels des Fahrerassistenzsystems nach 2, und
• 4 Szenarien A bis F, in welchen das Fahrerassistenzsystem nach 2 in einem Traktor aktiv bzw. passiv ist.

In the following, exemplary embodiments of the invention are explained in more detail with reference to the schematic drawing, the same or similar elements being provided with the same reference symbols. Here shows

• 1 a top view of a field on which a tractor with an attachment drives autonomously,
• 2 Elements of a driver assistance system for the tractor 1 ,
• 3 Steps of a method for the detection of track lines and for controlling the tractor according to 1 based on a selected track line by means of the driver assistance system 2 , and
• 4th Scenarios A to F, in which the driver assistance system after 2 is active or passive in a tractor.

1 Figure 3 shows an agricultural machine in the form of a tractor 1 . In the rear of the tractor 2 is an attachment 2 to the tractor 1 cultivated. The tractor 1 pulls the implement across a field 3 so that the attachment 2 the field 3 can edit and so that several track lines arise, in the embodiment shown, a first track line 4.1 and a second track line 4.2 . The first track line 4.1 runs parallel to the second track line 4.2 . The tractor 1 is located with the attachment 2 laterally spaced next to the first track line 4.1 . The second track line 4.2 is further away from the vehicle 1 and the attachment 2 as the first track line 4.1 .

The vehicle 1 includes a closer in 2 illustrated system 5 for the autonomous operation of the vehicle 1 (in certain situations / scenarios, cf. 4th ). The system 5 comprises in the exemplary embodiment shown a processor unit 6th , a storage unit 7th , a communication interface 8th and a sensor unit 9 .

The sensor unit 9 comprises, in the exemplary embodiment shown, a digital camera system, referred to below as “camera” and shown in FIG 2 with the reference number 11 is provided. The camera 11 is in particular on a roof of the tractor 1 arranged so that they are an environment 12 of the tractor 1 covers that is in a forward area and on both sides of the tractor 1 is located. However, this is purely exemplary. Alternatively, the digital camera system 11 also include, for example, a surround view system, which an environment 12 that is 360 ° around the tractor 1 extends around. The camera 11 takes pictures of the surroundings one after the other and continuously 12 of the tractor 1 on, creating a video or a sequence of images of the environment 12 of the tractor 1 is generated. This video or this sequence of images is the detection of the tractor's surroundings 1 . Within the individual images of this video or this image sequence, image sections can be defined (regions of interest, short: ROI), within which the track lines 4.1 and 4.2 can be recognized by means of image recognition methods. Methods of artificial intelligence or machine learning can optionally be used. For example, semantic segmentation can be selected as the method of image recognition, using an artificial neural network (eg a convolutional neural network, or CNN for short) that is trained with reference images. In addition to the camera 9 can the sensor unit 10 a radar-based sensor 13 exhibit. The radar-based sensor 13 is like that on the tractor 1 arranged so that he can determine height values that allow a calculation of a slope of the terrain, which in the detected environment 12 of the tractor 1 present.

On the storage unit 7th can be a computer program product 10 be saved. The computer program product 10 can be on the processor unit 6th are executed, including the processor unit 6th and the storage unit 7th are correspondingly communicatively connected to each other. When the computer program product 10 on the processor unit 6th is executed, it directs the processor unit 6th to carry out the functions or process steps described in connection with the drawing.

The tractor 1 a coordinate system with an x-axis (longitudinal direction) and a y-axis running transversely thereto (transverse direction or width direction) can be assigned that can be viewed as a reference coordinate system. Everyone from the camera 11 A camera coordinate system can be assigned to the recorded image. 1 shows that this coordinate system has a longitudinal axis L of the tractor combination 1 and attachment 2 may have. In the example after 1 the longitudinal axis L of the image coordinate system corresponds to the x-axis of the tractor coordinate system. The image coordinate system also has a width axis B of the attachment running transversely to the longitudinal axis L. 1 on, which extends laterally in width direction B than the tractor 1 . In the embodiment according to 1 if the latitude L does not coincide with the y-axis of the tractor coordinate system, there is a longitudinal offset x0 in the longitudinal direction x. The processing unit 6th is set up to take this offset into account by converting the determined image coordinates into tractor coordinates. The processing unit 6th is also set up to take into account an offset in the width direction y by converting the determined image coordinates into tractor coordinates (even if in the present example according to 1 there is no offset in width direction y).

The latitude axis B can, for example, pass through the widest section of the attachment 2 run through like this 1 is shown. The wider attachment 2 should according to the embodiment 1 have a desired lateral offset (“desired lateral offset”) to a closest track line (in the exemplary embodiment according to FIG 1 this is the first track line 4.1 ), while the tractor team 1 and attachment 2 on the field 3 drives and the field 3 by means of the attachment 2 processed. This desired lateral distance can be a few meters, for example 8 m. It can, for example, in a first process step 100 ( 3 ) a distance B0 can be defined, which is the closest track line 4.1 from the longitudinal axis L of the tractor combination 1 and attachment 2 should have at least. The desired lateral distance B0 can be present as a distance data record on which the processor unit 6th Has access. For example, the attachment 2 include a tag, for example an RFID tag, which is generated by the driver assistance system 5 can be read out. The distance data record can be stored on the day. Furthermore, the distance data record can also be provided by a user or a driver of the tractor 1 into the driver assistance system 5 can be generated, for example, by input via a corresponding terminal on the tractor 1 .

The camera 11 takes in a second step 200 ( 3 ) an image of the area around the tractor 1 on. The second step 200 can after the first step 100 be performed. Alternatively, the steps 100 and 200 however, they can also be carried out simultaneously or overlapping in time. The processing unit 6th can in a third process step 300 ( 3 ) that from the camera 11 evaluate the recorded image and the track lines 4.1 and 4.2 recognize, whereby, as mentioned above, methods of artificial intelligence can be used. This creates a classified image from the camera.

In a subsequent step 400 ( 3 ) a polynomial fitting can be carried out. With polynomial fitting, the image content is further processed and polynomials are calculated from individual information (points, broken lines, etc.) that define a track line or its direction. The result of this fourth step 400 are thus polynomials in the image coordinate system (L, B). In a fifth process step 500 ( 3 ) the polynomials that were previously calculated in the image coordinate system can be transformed into coordinates of the tractor coordinate system. The result of this fifth process step 500 are thus polynomials in the tractor coordinate system (x, y).

At the track lines 4.1 and 4.2 it can be swaths or cut edges, for example. The first track line 4.1 has in the width direction B a first lateral distance (first “lateral offset” / first “actual distance”) B1 to the longitudinal axis L of the combination of tractor 1 and attachment 2 on. The second track line 4.2 has in the width direction B a second lateral distance (second “lateral offset”; second “actual distance”) B2 to the longitudinal axis L of the combination of tractor 1 and attachment 2 on. The first lateral distance B1 is smaller than the second lateral distance B2, that is to say the first track line 4.1 is located in the width direction B closer to the longitudinal axis L of the tractor combination 1 and attachment 2 than the second track line 4.2 . The processing unit 6th can in a further process step 600 ( 3 ) evaluate the image to determine which of the two track lines 4.1 and 4.2 a smaller lateral distance (in the width direction B) to the combination of tractor 1 and attachment 2 having. In the embodiment according to 1 becomes the processor unit 6th while doing this, notice that the first track line 4.1 the smaller lateral distance to the tractor combination 1 and attachment 2 as the second track line 4.2 (B1 <B2). The processor unit becomes accordingly 6th the first track line 4.1 choose.

Based on the selected track line 4.1 can the processor unit 6th the tractor 1 in a seventh process step 700 control in such a way that the combination of tractor 1 and attachment 2 autonomously along the selected first track line 4.1 emotional. The attachment can 2 perform its main function, namely the field 3 edit and create a new track line next to the first track line, for example 4.1 create or the first track line 4.1 collect.

In that through 1 shown example, the first lateral actual distance B1 is smaller than the lateral setpoint distance B0. However, what is required or desired is that the actual distance B1 is at least as great as the setpoint distance B0. The processing unit 6th can determine this and, for example, corrective action on the steering of the tractor 1 act that the tractor-trailer combination 1 and attachment 2 from the first track line 4.1 moved away (according to the view of 1 to the left) that the distance B1 of the longitudinal axis L from the first track line 4.1 in subsequent from the camera 11 recorded images increases and that the target criterion B1 ≥ B0 is met. If the actual distance B1 and the setpoint distance B0 have the same value, that is determined by the driver assistance system 5 delivered steering accuracy high and the tractor 1 follows the track line perfectly 4.1 in the maintained distance B0. The greater the deviation between the setpoint distance B0 and the actual distance B1, the less precise the control or regulation by the driver assistance system 5 .

In that through 1 Example shown show the track lines 4.1 and 4.2 each has a curvature. The processing unit 6th is set up to control the curvature of the track lines 4.1 and 4.2 to investigate. For example, the processor unit 6th - how through 1 shown - a tangent T there to the first track line 4.1 Create where the latitude B is the first track line 4.1 cuts. This tangent T can be shifted in such a way that it lies at the point of intersection between the longitudinal axis L and the width axis B. The processing unit 6th can calculate an angle Δ between this displaced tangent T 'and the longitudinal axis L. The angle Δ can be seen as a heading error of the tractor 1 be considered. The angle Δ can be caused, for example, by drift, wind, traction problems or centrifugal forces. Would the tractor 1 move straight ahead in the direction of the longitudinal axis L, then the tractor would move 1 and the attachment 2 the first track line 4.1 approach and possibly run over them. However, around the curved first track line 4.1 with the lateral setpoint distance B0, the processor unit 6th for example, correcting the steering of the tractor in this way 1 act that the course of the shifted tangent T 'in subsequent from the camera 11 recorded images approximates the course of the longitudinal axis L, and that the course error in the form of the angle Δ is minimized.

The processing unit 6th can also be set up to initiate that the tractor is brought into a safe state (eg stopped) when the processor unit 6th at least one potential collision object in the environment detection 15th recognizes, for example, a person, a pole or another vehicle. Alternatively, a warning can also be sent to the driver of the tractor 1 or its environment 12 be handed in.

4th shows that the driver assistance system 5 of a tractor 1 then can be active and the tractor 1 can control as described above when the tractor 1 on the field 3 along the track line 4th drives (scenarios A, B and C). If problems with the recognition of the track lines 4th occur, transfers the driver assistance system 5 control over the control of the tractor 1 back to the driver of the tractor 1 . Even while the driver is the tractor 1 on a headland 14th turns, the driver assistance system hands over 5 control over the control of the tractor 1 back to the driver of the tractor 1 . The driver assistance system also transfers 5 control over the control of the tractor 1 back to the driver of the tractor 1 when the driver is not on the headland 14th turns around or takes a completely wrong course (scenarios D, E and F), because in these cases, for example, there is no track line 4th in the vicinity of the tractor 1 through the tractor's sensor unit 1 can be recorded and the conditions for controlling the tractor 1 thus cannot be met.

List of reference symbols

L.

Longitudinal axis tractor and implement in the image coordinate system

B.

Attachment latitude axis in the image coordinate system

B0

Desired lateral offset

B1

Lateral offset to the first track line

B2

Lateral offset to the second track line

x0

longitudinal offset in the tractor coordinate system

x

Longitudinal axis of the tractor in the tractor coordinate system

y

Cross axis / latitude axis in the tractor coordinate system

1

tractor

2

Attachment

3

field

4.1

first track line

4.2

second track line

5

Driver assistance system

6th

Processor unit

7th

Storage unit

8th

Communication interface

9

Sensor unit

10

Computer program product

11

digital camera system

12

Around the tractor

13

Radar-based sensor

14th

Headland

15th

potential collision object

100

Process step

200

Process step

300

Process step

400

Process step

500

Process step

600

Process step

700

Process step

Claims (14)

Processor unit (6) for the autonomous operation of a vehicle (1), the processor unit (6) comprising an interface (8), the processor unit (6) being set up to - to use the interface (8) to access a detection of the surroundings of the vehicle (1) generated by a sensor unit (9), - Recognize several track lines (4.1, 4.2) in the detection of the surroundings, - select a track line (4.1) from the several track lines (4.1, 4.2), and - To control the vehicle (1) based on the selected track line (4.1) in such a way that the vehicle (1) autonomously follows the selected track line (4.1). Processor unit (6) Claim 1 , the processor unit (6) being set up to control the vehicle (1) based on at least one of the following parameters, namely - a curvature of the selected lane line (4.1), - a course error of the vehicle (1), - a difference a lateral distance (B1) between the selected track line (4.1) and the vehicle (1) including its attachments (2) and a desired distance (B0) between the selected track line (4.1) and the vehicle (1) including its attachments, determined from the detection of the surroundings (2). Processor unit (6) according to one of the preceding claims, wherein the processor unit (6) is set up to recognize the multiple track lines (4.1, 4.2) in the environment detection using methods of image processing combined with methods of artificial intelligence. Processor unit (6) Claim 3 , whereby the method of semantic segmentation is combined with an artificial neural network. Processor unit (6) Claim 3 or 4th , whereby the method of edge detection is combined with an artificial neural network. Processor unit (6) according to one of the preceding claims, wherein the processor unit (6) is set up to control an attachment (2) mounted on the vehicle (1) based on the selected track line (4.1) and its course in the environment detection. Processor unit (6) according to one of the preceding claims, wherein the processor unit (6) is set up to - to check whether the environment detection shows at least one potential collision object, and - To transfer the vehicle (1) to a safe state or to reduce the speed of the vehicle (1) or to output an optical or acoustic warning to the user of the vehicle (1) or to the surroundings of the vehicle (1) when the surroundings are detected maps at least one potential collision object. Processor unit (6) according to one of the preceding claims, wherein the track line (4.1, 4.2) - a swath, - a cut edge, - a mowing edge, - one lane, - a row of plants, - a row of trees, - a curb or - Includes a work path (15). Driver assistance system (5) for the autonomous operation of a vehicle (1), comprising the driver assistance system (5) - A sensor unit (9), the sensor unit (9) being set up to generate a detection of the surroundings of the vehicle (1), and - A processor unit (6) according to one of the preceding claims. Driver assistance system (5) Claim 9 , wherein the sensor unit (9) for detecting the surroundings (12) of the vehicle (1) comprises at least one of the following sensors, namely - an image processing sensor, in particular a camera (11), preferably a stereo camera, - a radar sensor ( 13), - a laser-based sensor. Driver assistance system (5) according to one of the Claims 9 or 10 , the driver assistance system (5) further comprising an operating unit which is set up to enable a user to select a track line (4.1) from the multiple track lines (4.1, 4.2). A method for the autonomous operation of a vehicle (1), the method comprising the steps: - generating a detection of the surroundings of the vehicle (1) by a sensor unit (9), Recognition of several lane lines (4.1, 4.2) in the environment detection by means of a processor unit (6), - selecting a lane line (4.1) from the several lane lines (4.1, 4.2) by means of the processor unit (6), and - controlling the vehicle (1) by means of the processor unit (6) based on the selected track line (4.1), so that the vehicle (1) autonomously follows the selected track line (4.1). Computer program product (10) for the autonomous operation of a vehicle (1), the computer program product (10) instructing the processor unit (6) when it is executed on a processor unit (6), - to use an interface (8) to access a detection of the surroundings of the vehicle (1) generated by a sensor unit (9), - Recognize several track lines (4.1, 4.2) in the detection of the surroundings, - select a track line (4.1) from the several track lines (4.1, 4.2), and - To control the vehicle (1) based on the selected track line (4.1) in such a way that the vehicle (1) autonomously follows the selected track line (4.1). Vehicle (1) comprising a processor unit (6) according to one of the Claims 1 to 7th or a driver assistance system (5) according to one of the Claims 8 to 11 .

Images