DE112018001378T5 - IMPROVEMENTS IN VEHICLE CONTROL - Google Patents

Abstract

A computer system (10, 19, 185C) for a vehicle (100), the system comprising processing means (10, 19) arranged to receive terrain information from terrain data acquisition means (185C) comprising a stereoscopic camera system displaying the topography of the terrain extending ahead of the vehicle (100), the terrain information including at least one stereo image pair of terrain in front of the vehicle, the processing means (10, 19) being configured to: calculate a 3D point cloud data set in Relating to the terrain in front of the vehicle based on the terrain information; identifying a substantially continuous terrain area in front of the vehicle and within a predicted path of the vehicle in which the number of data points in the 3D point cloud corresponding to that area is less than a threshold for the data point density in the empty area, the area being an empty area when the number of data points in the 3D point cloud corresponding to this range is less than a threshold for the data point density in the empty area, determining whether the empty area corresponds to a body of water, which is an area, satisfying the criteria: the number of pixels in an image of this area that have been picked up by the camera system and that correspond to the specular reflection of a feature of the environment, including the terrain, exceeds a specular reflection density threshold; and a height difference between at least two positions of the range is less than a threshold height difference of the location, the system configured to output a signal depending on whether the terrain corresponding to a body of water has been identified in front of the vehicle.

Description

EDITING THROUGH REFERENCE

The content of the co-pending British patent applications GB2507622 and GB2499461 is hereby incorporated by reference. The content of US Pat. US7349776 and the pending international patent applications WO2013124321 and WO2014 / 139875 are hereby incorporated by reference. The content of the British patent applications GB2492748 . GB2492655 and GB2499279 and the British patent GB2508464 is also incorporated herein by reference.

FIELD OF THE INVENTION

The invention relates to a system for controlling a vehicle. More particularly, but not exclusively, the invention relates to a steering control system for a land vehicle capable of driving in a variety of different and extreme terrain and conditions.

BACKGROUND

In known vehicle steering control systems, such as lane keeping systems, a still-facing camera system detects lane markings on the road in front of the vehicle. In some systems, feedback is provided in the form of an audible alarm or haptic response, such as steering wheel vibration, when the vehicle deviates too much from a fictitious lane centerline or crosses a lane marker. Some steering control systems automatically control the steering angle of the road wheel to keep a vehicle in lane while driving on a highway based on lane markings.

The present Applicant has recognized that prior art steering control systems are unsuitable for off-road driving in which such systems may be of particular value for reducing driver fatigue.

Against this background, the present invention was conceived. Embodiments of the invention may provide a device, method, or vehicle that solves this problem. Other objects and advantages of the invention will be apparent from the following description, claims and drawings.

SUMMARY OF THE INVENTION

• Berechnen eines 3D-Punktwolkendatensatzes in Bezug auf das Gelände vor dem Fahrzeug basierend auf den Geländeinformationen;
• Identifizieren eines im Wesentlichen kontinuierlichen Geländebereichs vor dem Fahrzeug und innerhalb eines vorhergesagten Weges des Fahrzeugs, in dem die Anzahl der Datenpunkte in der 3D-Punktwolke, die diesem Bereich entspricht, kleiner als ein Schwellenwert für die Datenpunktdichte im leeren Bereich ist, wobei der Bereich als ein leerer Bereich betrachtet wird, wenn die Anzahl der Datenpunkte in der 3D-Punktwolke, die diesem Bereich entspricht, kleiner als ein Schwellenwert für die Datenpunktdichte im leeren Bereich ist;
• Bestimmen, ob der leere Bereich einem Gewässer entspricht, wobei es sich um einen Bereich handelt, der die folgenden Kriterien erfüllt:
  • die Anzahl der Pixel in einem Bild dieses Bereichs, die vom Kamerasystem aufgenommen wurden und der spiegelnden Reflexion eines Merkmals der Umgebung, einschließlich des Geländes, entsprechen, einen Schwellenwert der Spiegelreflexionsdichte überschreitet; und
  • ein Höhenunterschied zwischen mindestens zwei Positionen des Bereichs kleiner als ein Schwellenwert für die Höhendifferenz des Standorts ist,
  • wobei das System konfiguriert ist, um ein Signal in Abhängigkeit davon auszugeben, ob das einem Gewässer entsprechende Gelände vor dem Fahrzeug identifiziert wurde.

In one aspect of the invention for which protection is sought, a computer system is provided for a vehicle, the system comprising processing means arranged to receive terrain information from terrain data acquisition means comprising a stereoscopic camera system, which includes the topography of the vehicle Display terrain that extends in front of the vehicle, wherein the terrain information includes at least one stereo image pair of terrain in front of the vehicle, wherein the processing means is configured to:

• Calculating a 3D point cloud datum in relation to the terrain in front of the vehicle based on the terrain information;
• Identifying a substantially continuous terrain area in front of the vehicle and within a predicted path of the vehicle in which the number of data points in the 3D point cloud corresponding to that area is less than a threshold empty point data point density, the area being an empty area is considered if the number of data points in the 3D point cloud corresponding to this area is smaller than a threshold for the data point density in the empty area;
• Determine if the empty area corresponds to a body of water, which is an area that meets the following criteria:
  • the number of pixels in an image of this area taken by the camera system and corresponding to the specular reflection of a feature of the environment, including the terrain, exceeds a specular reflection density threshold; and
  • a height difference between at least two positions of the range is smaller than a threshold value for the altitude difference of the location,
  • wherein the system is configured to output a signal depending on whether the terrain corresponding to a body of water has been identified in front of the vehicle.

Embodiments of the present invention have the advantage that the presence of a body of water (body of water) in front of the vehicle can be reliably detected and an output signal can be generated in response thereto.

It is to be understood that when there is a body of water, the waters typically reflect terrain characteristics above the water such as sky, trees, shrubs or the like reflected. The apparent distance of these terrain features from the terrain data acquisition means, based on the disparity of the stereo image pair, is greater than the distance of the body of water from the terrain data acquisition means. Accordingly, the area of the 3D point cloud that corresponds to the location of the body of water may have a lower density of data points than if the area of the water were a non-reflecting body, such as an earth body. Embodiments of the present invention seek to detect the presence of a body of water based on typical features of such a body, which are the presence of specular reflections through the body of water in a captured image of the body of water and the property of flatness typically associated with a body of water connected is.

The system may be configured wherein, in order for the system to determine that an empty area corresponds to a body of water, the further condition must be satisfied that the height difference between at least two locations on substantially opposite sides of the area is less than a height difference threshold is.

In some embodiments, "opposite sides" refers to points along a particular column or row of pixels that includes a portion of the body of water that hits the ground. The pixels whose height is taken into account may be pixels closest to the blank area, on opposite sides of the blank area, along the column or row.

The system may be configured to determine the slope of the terrain to be experienced by the vehicle immediately before the empty area within the predicted path, wherein if the slope exceeds a predetermined slope, the system determines that the area is not a body of water equivalent.

The system may be further configured to determine the terrain slope to be experienced by the vehicle immediately before the empty area within the predicted path, wherein if the slope exceeds a predetermined slope value, the system determines that the area does not correspond to a body of water and a crest of a slope.

The system may be further configured to identify the empty area in at least one of the stereoscopic image pairs corresponding to the 3D point cloud and to determine whether the empty area is at least partially dependent on RGB color values (red, green , blue) of at least one pixel corresponding to the empty area.

The system may be configured to determine whether the empty area corresponds to a body of water, depending at least in part on an average RGB color value (red, green, blue) over a plurality of pixels corresponding to the empty area.

The system can be configured to determine that the empty area does not correspond to a body of water unless the average RGB color value (red, green, blue) over a plurality of pixels corresponding to the empty area is above one predetermined minimum average RGB color value and below a predetermined maximum average RGB color value.

Optionally, the system is configured to determine whether the number of pixels in an empty area image corresponding to the specular reflection of an environmental feature, including the terrain, exceeds a specular reflection density threshold, wherein the system is configured to Determine the difference between the corresponding pixels of the left and right stereo image pair, which corresponds to the empty area, wherein the number of pixels corresponding to the specular reflection, determined in dependence on the difference between the corresponding pixels of the left and right image pair.

Optionally, the system is further configured to require that a predetermined width condition be satisfied with respect to a width of the area of at least one of the stereoscopic image pairs corresponding to the blank area so that the blank area corresponds to an empty area.

It should be understood that in some embodiments, this requirement is a necessary condition but not a sufficient condition for the void area to correspond to a body of water.

Optionally, the predetermined width condition includes the condition that a width or average width of at least a part of the image including the blank area exceeds a predetermined threshold for the width.

In some embodiments, the system may be configured to require that at least a portion of the empty area have a width that exceeds the threshold for the width. The section may be a system-defined polygonal section passing through Extraction of pixels of the left or right image of the stereoscopic image pair is defined according to the empty area.

The system may include a cruise control, wherein the cruise control is configured to control vehicle speed based at least in part on whether a body of water has been identified in the path of the vehicle.

The speed controller may receive the signal output from the signal of the computer system depending on whether terrain corresponding to a body of water has been identified in the path of the vehicle and control the vehicle speed based on the signal.

The system may be configured to provide an output to the cruise control that indicates a recommended maximum speed that depends, at least in part, on whether a body of water has been identified in the path of the vehicle.

The system can be configured to alert a driver based on whether a body of water has been identified in the vehicle's driveway.

Optionally, the terrain data acquisition means comprises a stereoscopic camera system.

The system may further comprise the terrain data acquisition means.

• Berechnen eines 3D-Punktwolkendatensatzes in Bezug auf das Gelände vor dem Fahrzeug basierend auf den Geländeinformationen;
• Identifizieren eines im Wesentlichen kontinuierlichen Geländebereichs vor dem Fahrzeug und innerhalb eines vorhergesagten Weges des Fahrzeugs, in dem die Anzahl der Datenpunkte in der 3D-Punktwolke, die diesem Bereich entspricht, kleiner als ein Schwellenwert für die Datenpunktdichte im leeren Bereich ist, wobei der Bereich als ein leerer Bereich betrachtet wird, wenn die Anzahl der Datenpunkte in der 3D-Punktwolke, die diesem Bereich entspricht, kleiner als ein Schwellenwert für die Datenpunktdichte im leeren Bereich ist;
• Bestimmen, ob der leere Bereich einem Gewässer entspricht, wobei es sich um einen Bereich handelt, der die Kriterien erfüllt:
  • die Anzahl der Pixel in einem Bild dieses Bereichs, die vom Kamerasystem aufgenommen wurden und der spiegelnden Reflexion eines Merkmals der Umgebung, einschließlich des Geländes, entsprechen, einen Schwellenwert der Spiegelreflexionsdichte überschreitet; und
  • ein Höhenunterschied zwischen mindestens zwei Positionen des Bereichs kleiner als ein Schwellenwert für die Höhendifferenz des Standorts ist,
  • und gibt ein Signal in Abhängigkeit davon aus, ob vor dem Fahrzeug ein Gelände entsprechend einem Gewässer identifiziert wurde.

Optionally, the system includes an electronic processor having an electrical input for receiving terrain information indicating the topography of the terrain in front of the vehicle; and
an electronic storage device electrically coupled to the electronic processor, and
with instructions stored therein,
wherein the processor is configured to access the storage device and execute the instructions stored therein to be operable:

• Calculating a 3D point cloud datum in relation to the terrain in front of the vehicle based on the terrain information;
• Identifying a substantially continuous terrain area in front of the vehicle and within a predicted path of the vehicle in which the number of data points in the 3D point cloud corresponding to that area is less than a threshold empty point data point density, the area being an empty area is considered if the number of data points in the 3D point cloud corresponding to this area is smaller than a threshold for the data point density in the empty area;
• Determine if the empty area corresponds to a body of water, which is an area that meets the criteria:
  • the number of pixels in an image of this area taken by the camera system and corresponding to the specular reflection of a feature of the environment, including the terrain, exceeds a specular reflection density threshold; and
  • a height difference between at least two positions of the range is smaller than a threshold value for the altitude difference of the location,
  • and outputs a signal depending on whether a terrain corresponding to a body of water has been identified in front of the vehicle.

In one aspect of the invention for which protection is sought, a vehicle is provided that includes a system according to another aspect.

• Berechnen eines 3D-Punktwolkendatensatzes in Bezug auf das Gelände vor dem Fahrzeug basierend auf den Geländeinformationen;
• Identifizieren eines im Wesentlichen kontinuierlichen Geländebereichs vor dem Fahrzeug und innerhalb eines vorhergesagten Weges des Fahrzeugs, in dem die Anzahl der Datenpunkte in der 3D-Punktwolke, die diesem Bereich entspricht, kleiner als ein Schwellenwert für die Datenpunktdichte im leeren Bereich ist, wobei dieser Bereich als ein leerer Bereich betrachtet wird, wenn die Anzahl der Datenpunkte in der 3D-Punktwolke, die diesem Bereich entspricht, kleiner als ein Schwellenwert für die Datenpunktdichte im leeren Bereich ist;
• Bestimmen, ob der leere Bereich einem Gewässer entspricht, wobei es sich um einen Bereich handelt, der die Kriterien erfüllt, die:
  • die Anzahl der Pixel in einem Bild dieses Bereichs, die vom Kamerasystem aufgenommen wurden und der spiegelnden Reflexion eines Merkmals der Umgebung, einschließlich des Geländes, entsprechen, einen Schwellenwert der Spiegelreflexionsdichte überschreitet; und
  • ein Höhenunterschied zwischen mindestens zwei Positionen des Bereichs kleiner als ein Schwellenwert für die Höhendifferenz des Standorts ist,
  • wobei das Verfahren das Ausgeben eines Signals in Abhängigkeit davon umfasst, ob das einem Gewässer entsprechende Gelände vor dem Fahrzeug identifiziert wurde.

In one aspect of the invention for which protection is sought, a method of identifying the terrain corresponding to a body of water implemented by means of a computer system of a vehicle is provided,
the method comprising receiving terrain data acquisition means ( 185C ) comprising a stereoscopic camera system of terrain information indicating the topography of the terrain in front of the vehicle ( 100 ), the terrain information including at least one stereo image pair of terrain in front of the vehicle, the method comprising:

• Calculating a 3D point cloud datum in relation to the terrain in front of the vehicle based on the terrain information;
• Identifying a substantially continuous terrain area in front of the vehicle and within a predicted path of the vehicle in which the number of data points in the 3D point cloud corresponding to that area is less than a threshold empty point data point density, said area an empty area is considered if the number of data points in the 3D point cloud corresponding to this area is smaller than a threshold for the data point density in the empty area;
• Determine if the empty area corresponds to a body of water, which is an area that meets the criteria that:
  • the number of pixels in an image of this area taken by the camera system and corresponding to the specular reflection of a feature of the environment, including the terrain, exceeds a specular reflection density threshold; and
  • a height difference between at least two positions of the range is smaller than a threshold value for the altitude difference of the location,
  • the method comprising outputting a signal depending on whether the terrain corresponding to a body of water has been identified in front of the vehicle.

In one aspect of the invention for which protection is sought, there is provided a non-transitory computer-readable carrier medium carrying a computer-readable code for controlling a vehicle to perform the method of another aspect.

In one aspect of the invention for which protection is desired, a computer program product executable on a processor is provided to implement the method of another aspect.

In one aspect of the invention for which protection is desired, there is provided a non-transitory computer-readable medium loaded with the computer program product of another aspect.

In one aspect of the invention for which protection is sought, a processor is provided that is adapted to implement the method of another aspect or the computer program product of another aspect.

It is expressly intended in the context of this application that the various aspects, embodiments, examples and alternatives set forth in the preceding paragraphs, in the claims and / or in the following descriptions and drawings, and in particular the individual features thereof, be made independently or in part any combination can be accepted. That is, all embodiments and / or features of an embodiment may be combined in any manner and / or combination, unless these features are incompatible. The Applicant reserves the right to change a claim originally filed or to submit a new claim accordingly, including the right to change a claim originally filed in order to be dependent on another claim and / or to receive a property of another claim, although it was not initially claimed in this way.

list of figures

• 1 ist eine schematische Darstellung eines Fahrzeugs gemäß einer Ausführungsform der Erfindung in der Draufsicht;
• 2 zeigt das Fahrzeug von 1 in der Seitenansicht;
• 3 ist ein schematisches Übersichtsdiagramm einer Ausführungsform des Fahrzeuggeschwindigkeitsregelsystems der vorliegenden Erfindung, einschließlich eines Geschwindigkeitsregelsystems und eines langsamen Fortschrittsregelsystems;
• 4 veranschaulicht ein Lenkrad eines Fahrzeugs gemäß der Ausführungsform von 1;
• 5 ist eine schematische Darstellung in Draufsicht einer Geländehöhenkarte vor dem Fahrzeug;
• 6 zeigt (a) ein Beispiel für ein Bild, das von einer linken Kamera eines stereoskopischen Kamerasystems des Fahrzeugs von 1 aufgenommen wurde, und (b) eine Darstellung der entsprechenden, nach der MLS-Methode erzeugten Höhenkarte in Bezug auf das unter (a) dargestellte Bild;
• 7 ist eine schematische Darstellung einer MLS-Karte, die Zellen C umfasst, wobei Zellen, die der Lage eines Hindernis OB entsprechen, in relativ dunkler Schattierung dargestellt sind, während Zellen, die dem Schatten SH des Hindernisses entsprechen, in mittlerer Schattierung dargestellt sind;
• 8 zeigt (a) den Abschnitt des Bildes von 6(a), der dem in 6(b) dargestellten leeren Bereich ER entspricht, in Form eines zugeschnittenen polygonalen Bereichs des Kamerabildes von 6(a) und (b) einen verlängerten zugeschnittenen polygonalen Bereich, basierend auf dem in (a) dargestellten zugeschnittenen polygonalen Bereich;
• 9 ist ein Diagramm, das eine Domäne von Werten der SpeckleZahl und der durchschnittlichen Intensität von R, G, B Farbwerten abbildet, die gemäß einer Ausführungsform der vorliegenden Erfindung bestimmt wurden, die mit dem Vorhandensein eines Grabens oder Gewässers in einem stereoskopischen Bildpaar von Gelände vor einem Fahrzeug übereinstimmen; und
• 10 ist ein Flussdiagramm, das den Betrieb eines Computersystems des Fahrzeugs mit der Ausführungsform von 1 veranschaulicht.

The present invention will now be described by way of example only with reference to the accompanying drawings, in which:

• 1 is a schematic representation of a vehicle according to an embodiment of the invention in plan view;
• 2 shows the vehicle from 1 in the side view;
• 3 Fig. 10 is a schematic overview diagram of an embodiment of the vehicle speed control system of the present invention including a speed control system and a slow progress control system;
• 4 FIG. 10 illustrates a steering wheel of a vehicle according to the embodiment of FIG 1 ;
• 5 is a schematic representation in plan view of a terrain elevation map in front of the vehicle;
• 6 FIG. 11 shows (a) an example of an image taken by a left camera of a stereoscopic camera system of the vehicle of FIG 1 and (b) a representation of the corresponding MLS altitude map with respect to the image shown at (a);
• 7 is a schematic representation of an MLS card, the cells C includes, taking cells that are the location of an obstacle IF are shown in relatively dark shade, while cells corresponding to the shadow SH of the obstacle are shown in middle shading;
• 8th shows (a) the section of the image of 6 (a) who in the 6 (b) shown empty area HE corresponds, in the form of a trimmed polygonal area of the camera image of 6 (a) and (b) an elongated trimmed polygonal region based on the trimmed polygonal region shown in (a);
• 9 is a diagram that is a domain of values of the speckle number and the average intensity of R . G . B Mapping color values determined according to an embodiment of the present invention that match the presence of a trench or body of water in a stereoscopic image pair of terrain in front of a vehicle; and
• 10 FIG. 10 is a flowchart illustrating the operation of a computer system of the vehicle having the embodiment of FIG 1 illustrated.

DETAILED DESCRIPTION

References thereon to a block, such as a functional block, are understood to include reference to software code for performing the specified function or action, which may be an output provided in response to one or more inputs. The code may be in the form of a software routine or function called by a host program, or it may be code that is part of a code stream and is not a separate routine or function. For a better explanation of the operation of the embodiments of the present invention, reference is made to the functional block.

PICTURE 1 shows a vehicle 100 according to an embodiment of the present invention. The vehicle 100 has a drive train 129 on, the one engine 121 that involves a powertrain 130 with an automatic transmission 124 connected is. It should be understood that the embodiments of the present invention are also suitable for use in vehicles having manual transmissions, continuously variable transmissions, or other suitable transmission.

In the embodiment of 1 can the gearbox 124 by means of a transmission mode selector wheel 124S be set to one of a plurality of transmission modes, namely, a parking mode, a reverse mode, a neutral mode, a drive mode, or a sport mode. The dial 124S provides an output signal to a powertrain control 11 , whereupon the powertrain control 11 causes the gearbox 124 operates according to the selected transmission mode.

The powertrain 130 is arranged so that it has a pair of front vehicle wheels 111112 by means of a front differential 137 and a pair of front drive shafts 118 drives. The powertrain 130 also includes an auxiliary drive train section 131 which is arranged so that he has a pair of rear wheels 114 . 115 by means of an auxiliary drive shaft or shaft 132 , a rear differential 135 and a pair of rear drive shafts 139 drives.

Embodiments of the invention are suitable for use with vehicles in which the transmission is arranged to drive only a pair of front wheels or just a pair of rear wheels (ie, front wheel drive vehicles or rear wheel drive vehicles) or selectable two wheel drive / four wheel drive vehicles. In the embodiment of 1 is the transmission 124 by means of a power transmission unit (PTU) 131P detachable with the auxiliary drive train section 131 connectable, which allows operation in a two-wheel drive or four-wheel drive. It is to be understood that embodiments of the invention may be suitable for vehicles with more than four wheels or when only two wheels are driven, eg two wheels of a three-wheeled vehicle or a four-wheeled vehicle or a vehicle with more than four wheels.

A control system for the vehicle engine 121 includes a central control 10 used as vehicle control unit (VCU) 10 is called the powertrain control 11 , a brake control 13 (an antilock brake system (ABS) control) and a steering control 170C , The ABS control 13 is part of a braking system 22 ( 3 ). The VCU 10 receives and outputs a variety of signals to and from various sensors and subsystems (not shown) on the vehicle. The VCU 10 includes a in 3 illustrated slow cruise control (LSP) control system 12 , a stability control system (SCS) 14 , a cruise control system 16 and a Hill Decent Control (HDC) system 12HD , The SCS 14 improves the safety of the vehicle 100 by detecting and controlling the loss of traction or steering control. If a reduction in traction or steering control is detected, then the SCS 14 automatically able to control the ABS 13 to instruct one or more brakes of the vehicle to operate the vehicle 100 to steer in the direction in which the user wants to drive. In the illustrated embodiment, the SCS 14 through the VCU 10 implemented. In some alternative embodiments, the SCS 14 through the ABS control 13 be implemented.

Although in 3 not shown in detail, includes the VCU 10 furthermore a function block Traction Control (TC). The TC function block is implemented in software code that is provided by a VCU computing device 10 is performed. The ABS control 13 and the TC functional block provide outputs that include, for example, TC activity, ABS activity, single wheel braking interventions, and engine torque requests from the VCU 10 to the engine 121 in the case of a wheel slip event. Each of the aforementioned events indicates that a wheel slip event has occurred. In some embodiments, the ABS controller implements 13 the TC function block. Other vehicle subsystems, such as a rollover stability control system or the like, may also be included.

As already mentioned, the vehicle includes 100 also a speed control system 16 which is operable to automatically maintain the vehicle speed at a selected speed when the vehicle is traveling at speeds in excess of 25 km / h. The Speed control system 16 is with a cruise control HMI (Human Machine Interface) 18 equipped with the user with a target vehicle speed in a known manner in the cruise control system 16 can enter. In one embodiment of the invention, the input controls of the cruise control system are on a steering wheel 171 mounted (FIG 4 ). The speed control system 16 can by pressing a selection key 176 of the cruise control system are turned on. If the speed control system 16 is on, sets the operation of a speed controller 173 the current value of a cruise control on the current vehicle speed. By pressing a '+' button 174 the value of the cruise_set-speed can be increased while pressing a '-'-button 175 the value of the cruise_set-speed can be reduced. It is a continuation button 173R provided for controlling the speed control system 16 is operable to resume the cruise control to the immediate value of the cruise_set-speed after a driver crossing. It should be understood that known on-highway cruise control systems including the present system 16 are configured so that, in the event that the user applies the brake or, in the case of vehicles with manual transmission, a clutch pedal, the control of the vehicle speed through the cruise control system 16 is canceled and the vehicle 100 returns to a manual mode which requires the user to enter accelerator pedal or brake pedal to maintain vehicle speed. Moreover, the detection of a wheel slip event, such as may be triggered by a loss of traction, also has the effect of controlling the vehicle speed through the cruise control system 16 will be annulled. The speed control by the system 16 is resumed when the driver subsequently presses the resume button 173R suppressed.

The speed control system 16 monitors the vehicle speed and any deviation from the desired vehicle speed is automatically adjusted so that the vehicle speed is maintained at a substantially constant value, which is typically above 25 km / h. In other words, the cruise control system is ineffective at speeds below 25 km / h. The cruise control HMI 18 can also be configured to give the user a visual indication of the HMI 18 about the status of the cruise control system 16 to inform. In the present embodiment, the speed control system is 16 configured so that the value of the cruise_set-speed can be set to any value in the range of 25-150 km / h.

The LSP control system 12 Also provides a speed-based control system to the user that allows the user to select a very low target speed that the vehicle can travel on without the user needing any pedal inputs to maintain vehicle speed. The functionality of slow speed cruise control (or slow speed) is provided by on-highway cruise control 16 not provided, which works only at speeds above 25 km / h.

In the present embodiment, the LSP control becomes 12 by pressing on the steering wheel 171 attached selector switch 178 activated. The system 12 is operable to selectively drive train, traction control and braking actions on one or more wheels of the vehicle 100 apply together or individually.

The LSP control system 12 is configured to allow a user to set a desired vehicle speed setpoint in the form of a setpoint speed parameter, user_set-speed, via a slow HMI (LSP HMI) scheduler 20 ( 1 . 3 ) entering certain input keys 173-175 with the speed control system 16 and the HDC control system 12HD Splits. If the vehicle speed is within the permissible operating range of the LSP control system 12 is (which is in the present embodiment, the range of 2 to 30 km / h, although other ranges are also useful) and no other limitation on vehicle speed exists while coming from the LSP control system 12 is controlled, controls the LSP control system 12 the vehicle speed according to an LSP control system target speed value LSP_set-speed set substantially equal to the user target speed. In contrast to the cruise control system 16 is the LSP control system 12 configured to operate independently of the occurrence of a traction event. That is, the LSP control system 12 does not cancel the speed control when detecting the wheel slip. Rather, the LSP control system controls 12 active the vehicle behavior with slip detection.

The LSP control HMI 20 is arranged in the vehicle cabin so that it is easily accessible to the user. The user of the vehicle 100 can via the LSP HMI 20 the desired value of the user_set-speed, as described above, with the button "set-speed" 173 and the keys "+" / "-" 174 . 175 similar to the cruise control 16 into the LSP control system 12 enter. The LSP HMI 20 also includes a visual Display that informs the user about the status of the LSP control system 12 can be informed and guided.

The LSP control system 12 receives an input from the ABS controller 13 of the brake system 22 of the vehicle, indicating the extent to which the user is using the brake pedal 163 has slowed down. The LSP control system 12 also receives an input from an accelerator pedal 161 , indicating to what extent the user the accelerator pedal 161 has pressed, and an input from the transmission or transmission 124 , This latter input may include signals, for example, the speed of an output shaft of the transmission 124 , a slip of the torque converter and a translation request. Further inputs for the LSP control system 12 include an input of cruise control HMI 18 , which is for the state (ON / OFF) of cruise control 16 representative of an input of the LSP control system HMI 20 and an input of a gradient sensor 45 indicating the slope of the running surface over which the vehicle is 100 moves. In the present embodiment, the gradient sensor is 45 a gyroscopic sensor. In some alternative embodiments, the LSP control system receives 12 a signal indicative of the road inclination from another controller such as the ABS controller 13 , The ABS control 13 may determine the gradient based on a plurality of inputs, optionally based at least in part on signals indicative of the longitudinal and lateral acceleration of the vehicle, and a signal indicative of the reference vehicle speed of the vehicle (v_actual) is a signal representing the actual vehicle speed indicating over reason. Methods for calculating the vehicle reference speed based, for example, on wheel speeds of vehicles are known. For example, in some known vehicles, the vehicle reference speed may be determined as the speed of the second slowest wheel or the average speed of all wheels. Other methods of calculating the vehicle reference speed may be useful in some embodiments, including with the aid of a camera device or a radar sensor.

The HDC system 12HD is activated by pressing the key 177 enabled, from the HDC system HMI 20HD exists and on the steering wheel 171 is mounted. If the HDC system 12HD is active controls the system 12HD the brake system 22 to limit the vehicle speed to a value corresponding to that of a target HDC set speed parameter HDC_soll speed that is similar to the target speed of the cruise control system by a user 16 and the LSP control system with the same control buttons 173 . 173R . 174 . 175 can be controlled. The HDC system 12HD is functional so that the value of the HDC_set-speed can be set to any value in the range of 2-30 km / h. The parameter HDC setpoint speed can also be referred to as setpoint HDC speed. Unless the user has the HDC system 12HD not by pressing the accelerator pedal 161 overrides when the HDC system 12HD is active controls the HDC system 12HD the brake system 22 ( 3 ) to prevent the vehicle speed from exceeding the HDC_soll speed. In the present embodiment, the HDC system is 12HD unable to apply a positive drive torque. Rather, the HDC system 12HD only so operable that about the brake system 22 a negative braking torque is applied.

It is understood that the VCU 10 is configured to implement a known Terrain Response (TR) (RTM) system of the type described above, in which the VCU 10 the settings of one or more vehicle systems or subsystems such as powertrain control 11 in response to a selected drive mode. The driving mode may be selected by a user using a driving mode selector 141S ( 1 ) to be selected. The driving modes may also be referred to as Terrain Modes, Terrain Response (TR) Modes, or Control Modes.

In the embodiment of 1 four driving modes are provided: a driving mode on the road suitable for driving on a relatively hard, smooth running surface having a relatively high surface friction coefficient between the running surface and the wheels of the vehicle; a driving mode on sand suitable for driving over sandy terrain characterized at least in part by relatively high air resistance, relatively high ductility or compliance and relatively low surface friction coefficients; a driving mode on grass, gravel or snow (GGS) that is suitable for driving over grass, gravel or snow and is relatively slippery surfaces (ie with a relatively low friction coefficient between surface and wheel and typically a lower drag than sand); a rock-crawl (RC) drift mode capable of driving slowly over a rocky surface; and a mud and rattle drive mode (MR) capable of driving in muddy, tattered terrain. In addition or instead, other modes of operation may be provided. In the present embodiment, the selector switch allows 141S the user also the selection of an "automatic drive mode selection condition" in which the VCU 10 automatically selects the most appropriate driving style, as described in more detail below. On-road driving mode may be referred to in some embodiments as " Special Programs Off Mode (SPO), as it corresponds to a standard or standard driving mode and is not required to consider special factors such as a relatively low surface friction coefficient or surfaces with high roughness.

The LSP control system 12 causes the vehicle 100 according to the value of the LSP set-speed works.

To effect the application of the required positive or negative torque to the wheels, the VCU 10 instruct that through the powertrain 129 a positive or negative torque is applied to the vehicle wheels and / or that by the braking system 22 a braking force is applied to the vehicle wheels, any or both of which may be used to implement the change in torque required to achieve and maintain a required vehicle speed. In some embodiments, the torque is applied individually to the vehicle wheels, eg, by the torque vectoring of the powertrain to maintain the vehicle at the required speed. Alternatively, in some embodiments, torque may be applied to the wheels to maintain the required speed, e.g. As in vehicles with powertrain, where a torque transmission is not possible. In some embodiments, the powertrain control 11 be operable to implement torque vectoring to control an amount of torque acting on one or more wheels by controlling a powertrain component such as a rear drive unit, a front drive unit, a differential or other suitable component. For example, one or more components of the powertrain 130 include one or more clutches operable to vary a torque applied to one or more wheels. Other regulations may be useful.

If a powertrain 129 one or more electric machines, for example, one or more drive motors and / or generators, may include the powertrain control 11 be operated to modulate the torque applied to one or more wheels to realize torque vectoring by means of one or more electric machines.

In some embodiments, the LSP control system 12 a signal wheel_slip (also with 48 in 3 designated) indicative of a occurred wheel slip event. This signal 48 is also the on-highway cruise control system 16 supplied in the latter case, an override or blocking of the mode in the on-highway cruise control system 16 triggers, allowing automatic control of vehicle speed through the on-highway cruise control system 16 suspended or suspended. The LSP control system 12 however, it is not arranged to stop operation after receiving the wheel_slip signal 48 aborts or suspends. Rather, the system is 12 arranged so that it monitors the wheel slip and then managed to relieve the driver.

During a slip event, the LSP control system compares 12 continue to measure the measured vehicle speed with the value of the LSP_Sollgeschwindigkeit and continue to automatically control the torque acting on the vehicle wheels (through the drive train 129 and the brake system 22 ) to keep the vehicle speed at the selected value. It is therefore to be understood that the LSP control system 12 configured differently than the cruise control system 16 in which a wheel slip event causes the cruise control function to be overridden, so that the manual operation of the vehicle must be resumed, or the speed control by the cruise control system 16 by pressing the continue button 173R or the target speed button 173 is resumed.

The vehicle 100 is also equipped with additional sensors (not shown) that are representative of a variety of parameters related to vehicle motion and vehicle status. This includes a in 3 illustrated inertial measurement unit (IMU) 23 which provides information indicative of the yaw, roll and pitch angles and speed and velocity of the vehicle's longitudinal acceleration. The signals from the sensors provide or are used for calculation, a variety of driving status displays (also called terrain indexes) indicating the type of terrain conditions that the vehicle is about 100 moves.

The sensors (not shown) on the vehicle 100 include, but are not limited to, sensors that provide continuous sensor outputs to the VCU 10 As previously mentioned, including wheel speed sensors, an ambient temperature sensor, an atmospheric pressure sensor, tire pressure sensors, wheel joint sensors, an engine torque sensor (or engine torque estimator), a steering angle sensor, a steering wheel speed sensor, a gradient sensor (or gradient estimator), a lateral acceleration sensor, the portion of the SCS 14 can be a brake pedal position sensor, a brake pressure sensor, an accelerator pedal position sensor, longitudinal, transverse and Vertical motion sensors and water sensing sensors that are part of a vehicle watzhilfssystems (not shown). In other embodiments, only a selection of the aforementioned sensors may be used.

The VCU 10 also receives a signal from the steering control 170C , The steering control 170C is in the form of an electronic power steering (ePAS unit) 170C executed. The steering control 170C delivers to the VCU 10 a signal indicative of the steering force applied to the steerable road wheels 111 . 112 of the vehicle 100 is applied. This force corresponds to that of a user on the steering wheel 171 in combination with that of the ePAS unit 170C generated steering force exerts. The ePAS unit 170C Also provides a signal that indicates the rotational position or angle of the steering wheel. The steering control 170C is also configured to set the steering angle of the steerable road wheels with electric motors that are part of the ePAS unit to a desired value. Thus, the vehicle 100 configured to implement autonomous steering control when needed.

In the present embodiment, the VCU evaluates 10 the various sensor inputs to determine the likelihood that each of the plurality of different TR modes (control modes or driving modes) will be appropriate for the vehicle subsystems, each control mode corresponding to a particular terrain type over which the vehicle is traveling (eg, mud and ruts, Sand, grass / gravel / snow / snow) as described above.

If the user has selected the operation of the vehicle in the automatic driving mode selection condition, the VCU selects 10 then the most appropriate one of the control modes and is automatically configured to control the subsystems according to the selected mode. This aspect of the invention is described in our co-pending patent applications GB2492748 . GB2492655 and GB2499279 described in detail, the contents of each of which is incorporated herein by reference as mentioned above.

As indicated above, the type of terrain over which the vehicle is traveling (determined by the selected control mode) may also be in the LSP control system 12 used to determine a reasonable increase or decrease in vehicle speed. For example, if the user selects a value for the user_set-speed that is not appropriate for the type of terrain the vehicle is traveling over, the system may 12 automatically set the value of LSP_set-speed to a value lower than user_set-speed. In some cases, such as uneven or rough surfaces, the user-selected speed may not be achievable or appropriate over certain terrain types. If the system 12 A setpoint speed (a value of LSP_set-speed) that differs from the user-selected setpoint speed is provided to the user via the LSP HMI 20 provided a visual indication of the speed limit to indicate that an alternative speed was assumed.

Other precautions may be useful.

In the present embodiment, the vehicle is 100 with a stereoscopic camera system 185C configured to couple stereo color image pairs by means of a pair of front-facing color video cameras of the system 185C to create. Suitable stereo image pairs relative to the terrain in front of the vehicle are useful sources of terrain information indicating the topography of the terrain. A stream of two video image data is sent from the cameras to the VCU 10 passed, which in a processing section 19 received image data processed and repeatedly generated from the received images a 3D point cloud data set. Techniques for generating 3D point cloud data sets based on stereoscopic image data are known. Each point in the 3D point cloud data set corresponds to a 3D coordinate of a point on a terrain surface in front of the vehicle 100 from each of the front-facing video cameras of the stereoscopic camera system 185C is looked at.

In the present embodiment, the 3D point cloud dataset is transformed such that the origin of the frame of reference of the dataset is the midpoint of a line connecting the points where the two front wheels connect 111 . 112 of the vehicle 100 touch the ground over which the vehicle 100 moves. In the present embodiment, the frame of reference is with respect to the Cartesian coordinates X . Y . Z defined, where X is an axis transverse to the direction of travel of the vehicle, ie, along a transverse direction with respect to the vehicle 100 . Y an axis is that in an upward direction with respect to the vehicle 100 oriented, which corresponds to a substantially vertical upward direction when the vehicle 100 parked on level ground, and Z parallel to or simultaneously with a longitudinal axis of the vehicle 100 along the direction of travel of the vehicle 100 is.

The processing section 19 is configured to compute a terrain elevation map filled with data points of the point cloud dataset. PICTURE 5 is a schematic representation in plan view of a height map of the terrain in front of the vehicle 100 , In the present Embodiment becomes the height map with respect to the vehicle axles X . Y . Z (ie in the frame of reference of the vehicle). Following the MLS (Multi-Level Surface) methodology, the elevation map is fictitiously considered to be composed of square cells C in the XZ plane of predetermined size, as shown schematically in FIG 5 shown (see attachment). In the present embodiment, the cells are 0.25m from the side, although other sizes may be useful in some embodiments.

In the present embodiment, each 3D point is associated with the height map of a cell C corresponding to its position with respect to the X-Z plane. A particular cell may contain points that are at multiple levels or heights, i. have different values of the Y coordinate. The points within a given cell are grouped according to the value of the Y-coordinate into one or more corresponding "patches", whereby points with a Y-coordinate within a predetermined value range of the Y-coordinate are assigned to a patch which corresponds to this value range of the Y-coordinate. Coordinate corresponds. For example, if patches of points with different y-coordinates are identified in the same cell, the range can be defined by a +/- range in the y-axis around an average y-value of each patch. For example, points within a given patch may have a different y coordinate because they are assigned within a range, but are due to the same surface. Each patch may be defined to be substantially parallel to the X-Z plane, e.g. it can be defined to have the size of the patch at the x-z window with a y-coordinate, which is the average value of the y-coordinate for that patch. In contrast, points within the same cell that correspond to another surface, e.g. a bridge over the running surface, or an overhanging tree branch, should not be part of the same area as points falling on the running surface. It should be understood that each patch contains information about the geometry of the points that fall within this patch.

It is understood that data structures other than MLS cards may be used, e.g. Voxel maps or any other electronic surface mapping technology that reduces the number of data points to analyze, the purpose of the map being to reduce the number of data points.

The processing section 19 analyzes the cells and a cell is called an "obstruction" cell when the variance and the range of data points in the lowest part of the cell exceed the respective predetermined thresholds. In addition, there are some arrangements in which a cell is referred to as an "obstacle" cell when an overhanging spot having a height below a vehicle-dependent threshold is detected. In some alternative embodiments, the average height may be used instead. In some embodiments, the height difference (mean height) between a current cell and neighboring cells may be used to detect obstacles, ie, relatively large height differences between adjacent cells may indicate the presence of an obstacle. Cells that are not marked as obstruction cells may be referred to as a "horizontal patch." If a cell does not contain any points, it can be referred to as an "empty patch."

In addition to implementing the MLS methodology, as described above, the elevation map is refined to discard overhanging patches, which are falsely referred to as horizontal patches. An overhanging patch is a patch that is at a different elevation than a corresponding lower patch of the same cell, which coincides with the presence of an object that overhangs the surface over which the vehicle 100 drives, such as a branch of a tree or a bridge that the vehicle 100 underpasses.

Cells labeled as obstruction cells are considered cells that can not correspond to a body of water and are therefore not taken into account when identifying areas that may correspond to a body of water. In addition, "empty" cells that are located behind obstacles and closed from the camera's point of view ("obstacle shadows" or "shadow cells") are identified and also ignored. It should be understood that these "empty" cells typically have a relatively low (if not all) number of data points per horizontal unit area of terrain.

In the present embodiment, the processing section determines 19 a predicted way of the vehicle 100 based on the instantaneous value of the steering angle of the vehicle 100 , In the present embodiment, the steering angle becomes directly related to the rotation angle of the steering wheel 171 of the vehicle 100 considered. The predicted path is considered a path to which the vehicle 100 follows when the vehicle with the immediate steering angle and essentially without slip or slip of the wheels of the vehicle 100 drives on. It is to be understood that the predicted path is that of a wheelbase of the vehicle 100 is discharged, can be regarded as an area whose width is substantially equal to a maximum track of the vehicle (maximum lateral distance between left and right Front or rear wheels) and that on a center line of the vehicle 100 is centered.

The processing section 19 then identifies substantially continuous areas or terrain areas defined by empty cells that are not shadow cells and that have at least a portion that is within the predicted path from the vehicle 100 is swept. These regions or territories are hereinafter referred to as "empty regions". Thus, each empty area is defined by a substantially continuous area of empty cells that are not shadow cells and have at least a portion that lies within the predicted path from the vehicle 100 is going through.

The reason for identifying empty cells, which are not shadow cells, is that these cells are candidate cells for cells, which are the geographical location of a water body, a slope, a ditch, over bright "regions or dark" Regions associated comb, as described in more detail below.

It should be understood that in the case of a body of water, the water typically reflects objects above the water. The disparity of the objects in a stereographic image pair is typically less than for all objects floating on the surface of the water, and therefore the location of the data points in the 3D point cloud that correspond to the terrain areas visible in the stereoscopic image pair does not actually become that Area of the terrain coincide, as the objects as farther away from the stereoscopic camera system 185C to be viewed as. Accordingly, empty cells are typically present in the region of the terrain that corresponds to the body of water.

In the event that the empty area is connected to a ridge of a slope, measuring the slope of the terrain immediately in front of the empty area may provide useful indication of whether the empty area corresponds to a body of water or an area behind the ridge of a slope.

FIGURE 6 (a) is an example of an image taken by a left camera of the stereoscopic camera system 185C of the vehicle 100 has been recorded. FIGURE 6 (b) is a representation of the corresponding elevation map generated by the MLS methodology, ie the map in 5 in terms of the picture of 6 (a) equivalent. For the map of 6 (b) be areas through the processing section 19 were identified as corresponding to the areas of different types, with different contrast or gray levels. Areas corresponding to the obstacles OB are hatched and marked with OB, areas corresponding to the obstacle shadows are indicated by SH, and areas corresponding to the empty cells which are not obstruction shadow cells are indicated by ER. The left and right wheel tracks are superimposed on the map. The corresponding features are in the camera image of 6 (a) indicated for easy correlation by the reader.

FIGURE 7 illustrates the effect of having obstacles on the 3D point cloud data using a relatively simple example. In 7 is a schematic representation of an MLS card with cells C shown. cell C , which correspond to the location of an obstacle OB, are displayed in relatively dark shading, while cells corresponding to the shadow SH of the obstacle, are shown in medium shading. Other cells are displayed without shading. The position of the stereoscopic camera system 185C in relation to the terrain represented by the MLS map is shown schematically in the lower part of the figure. The "fan shape" of the shadow area may be out of the picture as a result of the position of the camera 185C be understood, and their position within the limits of the MLS card is at least partially dependent on the position of the camera system 185C relative to the object. The vehicle axles X and Y ( 2 ) are also superimposed with 7 shown.

For every empty area HE the processing section determines 19 from the 3D point cloud data, the gradient of the terrain just ahead of the area along the predicted path and within that through the left and right wheel tracks LW . RW defined way. If the slope exceeds a predetermined crest slope value and a width of the area exceeds a predetermined crest slope value, then the terrain just ahead of the area is considered a crest area corresponding to the crest of a slope rather than a body of water. In the present embodiment, the predetermined peak gradient value is substantially 10% and the predetermined peak width value is substantially 2m ,

When the processing section 19 determines that the area is not a comb area, the processing section tries 19 to determine if the area can correspond to a body of water or a ditch. For this purpose, the processing section identifies 19 in one of the stereoscopic camera system 185C For example, captured 2D image pairs have an area corresponding to the empty area ER identified in the MLS map. Of the processing section 19 extracts from this 2D image the area that is the empty area HE corresponds, and additionally an area of the image immediately around the area HE , This is to try to include more reflections of objects in the water, when in fact the empty region HE corresponds to a body of water.

As an example shows 8 (a) the part of the picture of 6 (a) who in the 6 (b) shown empty area HE equivalent. The image is in the form of a trimmed polygonal area of the camera image of 6 (a) , The processing section 19 defines an extended cropped polygonal area of the image 6 (a) which includes an area of the image immediately adjacent to the trimmed polygonal area of 8 (a) lies. The extended, truncated polygonal area in this example is in 8 (b) shown.

The processing section 19 then tries to determine the number of pixels in the extended trimmed polygonal area corresponding to the reflection of an image of the environment, such as an image of a tree (which is a near object) or a sky (a distant object). Such pixels are referred to herein as reflection pixels, and the number of pixels is referred to as the "speckle number" for the extended trimmed polygonal area. The presence of reflection pixels is determined taking into account the corresponding area of a disparity image which is a data set of similar dimensions to each of the image pairs captured by the stereoscopic camera system, the data points of the data set corresponding to the spacing (disparity) between the respective pixels of the respective images. It should be understood that the closer the object is to the camera system, the greater the disparity between the pixels corresponding to a particular point on an object 185C lies. Therefore, the disparity map is expected to have relatively low disparity values with respect to reflection pixels in the blank area.

wobei Disparität und Brennweite in Pixeleinheiten angegeben sind, während Basislinie und Tiefe in Metereinheiten angegeben sind.In the present embodiment, the processing section is 19 configured to identify reflection pixels with respect to distant objects by identifying pixels with a disparity at least 40 meters from the camera system 185C equivalent. The following equation can be used: $$\text{disparity}<\left(\text{focal length}*\left(\text{baseline}\right)/\left(\text{depth}\right)\right)$$

where disparity and focal length are given in pixel units, while baseline and depth are given in units of meters.

The processing section 19 then identifies reflection pixels with respect to nearby objects. The processing section does this by taking into account the disparity values for each pixel along each substantially vertical column of the image, ie from the top of the image to the bottom of the image (or vice versa). The processing section 19 checks the variation of the disparity values from top to bottom (or from bottom to top). If there are no reflections in the image, the disparity values gradually increase from top to bottom (or decrease from bottom to top), ie areas of the image corresponding to objects closer to the camera are more disparate than objects that are further away from the camera. However, if there are reflections, the disparity values over at least some pixels moving from top to bottom may decrease rather than increase. In the present embodiment, the processing section looks 19 the disparity values at the top and bottom of each column of the image that correspond to the areas that are not part of a body of water, such as soil or objects adjacent to a puddle. The processing section 19 then determines the number of pixels in a given column where the disparity is less than the disparity of the pixels at the top and bottom of the column. That is, pixels representing an image of an object farther away than objects corresponding to the pixels at the top and bottom of the 2D image. These pixels correspond to the reflections of nearby objects.

The processing section 19 then sums up the number of near and far reflection pixels, those in the empty area HE were identified to get a 'speckleCount' value.

The processing section 19 is configured to then determine if the empty area HE corresponds to a body of water. The processing section 19 does this by first removing "dark" areas and "over bright" areas from the 3D point cloud data set as areas that could correspond to a body of water.

So determines the processing section 19 First, whether the empty areas correspond to a "dark" image area or a "brighter" image area. The processing section 19 does this by first calculating the average color value of each of the R, G and B channels in the trimmed polygonal region, in the present example this is the in 8 (a) shown area. If the average color values are less than a lower color threshold value and the value of speckleCount is below a lower speckleCount threshold, then the polygonal area is classified as a "darker" area. In the present embodiment, the lower speckleCount threshold is a value of essentially 10 Disparity points per selected polygon range, although other values may be useful. The lower color value threshold is in the present embodiment 35 set any units, although other values may be useful.

If the average color values are greater than an upper color threshold value and the value of speckleCount is below the lower speckleCount threshold, the polygonal region is classified as an "overbell" region.

1. (a) der Wert von speckleCount einen vorgegebenen oberen speckleCount-Schwellenwert überschreitet; und
2. (b) die durchschnittliche Breite des erweiterten zugeschnittenen polygonalen Bereichs entlang der Fahrtrichtung des Fahrzeugs des Bildes größer als ein Breitenschwellenwert und die Höhendifferenz (in Bezug auf die MLS-Karte) zwischen Zellen an gegenüberliegenden Enden einer Spalte von Zellen, die den erweiterten polygonalen Bereich überspannen, kleiner als ein Höhendifferenzschwellenwert ist.

The processing section 19 then determines if areas are not eliminated as potential areas corresponding to a body of water. The processing section 19 determines that an area not already eliminated may be an area corresponding to a body of water, if the following conditions are met:

1. (a) the value of speckleCount exceeds a predetermined upper speckleCount threshold; and
2. (b) the average width of the expanded trimmed polygonal region along the vehicle's travel direction of the image is greater than a latitude threshold and the height difference (with respect to the MLS map) between cells at opposite ends of a column of cells spanning the extended polygonal region is less than a height difference threshold.

In the present embodiment, the predetermined upper speckleCount threshold is substantially equal 50 Points per polygon area, although other values may be useful. In the present embodiment, the width threshold is set to 20 pixels and the height difference threshold is set to 0.25 m.

When the processing section 19 determines that the empty area ER corresponds to a body of water, gives the processing section 19 a signal to the LSP control system 12 out, that's the LSP control system 12 informed that a body of water in front of the vehicle 100 within the predicted path of the vehicle 100 .... The processing section 19 also provides an output for the LSP control system 12 available, the removal of the water body from the vehicle 100 displays.

1. (a) die durchschnittliche Breite des ausgedehnten zugeschnittenen polygonalen Bereichs entlang der Fahrzeugrichtung kleiner als ein Grabenbreitenschwellenwert ist;
2. (b) Der Wert von speckleCount ist kleiner als der vorgegebene untere speckleCount-Schwellenwert.

In the present embodiment, when the processing section 19 determines that the empty area HE does not correspond to a body of water and is neither a dark nor an over-bright area, the processing section determines 19 whether the area HE meets the criteria for classification as a trench. The processing section 19 determines that the empty area HE a trench, if the following conditions are met:

1. (a) the average width of the extended trimmed polygonal region along the vehicle direction is less than a trench width threshold;
2. (b) The value of speckleCount is less than the default lower speckleCount threshold.

In the present embodiment, the trench width threshold is 20 pixels, which is the same threshold as the polygonal area used to identify puddles, although other values may be useful. In the present embodiment, the predetermined lower speckleCount threshold is 5 points. Other values may be useful. In some embodiments, the value of speckleCount must be zero.

When the processing section 19 determines that the empty area ER corresponds to a trench, the processing section gives 19 a signal to the LSP control system 12 out, that's the LSP control system 12 informed that digging in front of the vehicle 100 within the predicted path that the vehicle 100 has been identified. The processing section 19 also provides an output for the LSP control system 12 ready the distance of the trench from the vehicle 100 displays.

FIG. 9 is a diagram depicting the values of speckleCount and the average intensity of the color values R, G, B in an area of an image illustrating the requirements of these parameters when an empty area ER of an image is shown as a dark area (DR). over brightening area (BR) or as area corresponding to a ditch or puddle (DP).

In the present embodiment, the LSP control system is 12 configured to respond to signals from the processing section 19 to respond indicating that a body of water or a ditch is in the path of the vehicle in front. In the present embodiment, the LSP control system is 12 configured to provide the driver with an audible alarm in the form of a chime and a visual alarm in the form of a graphic displayed on the LSP HMI 20 is displayed and indicates the nature of the terrain feature in front of it, ie whether there is a body of water or a ditch in front of it. The LSP HMI 20 also provides an indication of the distance of the terrain feature from the vehicle 100 and updates this value in real time as the vehicle approaches the feature.

In the present embodiment, the LSP control system is 12 configured to the speed of the vehicle 100 to reduce to a given water transit speed if a body of water has been identified, or a predetermined digger speed if a ditch has been identified. In the present embodiment, the predetermined water passage velocity and the predetermined trench speed are each substantially 5 km / h, although other values may be useful in some embodiments.

PICTURE 10 is a flow chart illustrating the operation of the stereoscopic camera system 185C and the processing section 19 of the vehicle 100 out 1 illustrated.

At step S101 captures the stereoscopic camera system 185C a stereoscopic image pair, which are spatially separated images from the left and right cameras of the system 185C be absorbed, which are mounted side by side.

In the present embodiment, the images are sent to the processing section 19 for step S103 although in some embodiments step S103 from the camera system 185C is performed. The camera system 185C is repeatedly configured to capture stereoscopic image pairs and to the processing section 19 issue. In the present embodiment, image pairs at a frame rate of 25 pairs per second are sent to the processing section 19 output. Other values may be useful, such as 10 pairs per second, 12 pairs per second, or any other suitable value.

At step S103 the processing section calculates 19 in the present embodiment, a disparity image based on the last received stereoscopic image pair.

At step S105 the processing section calculates 19 a 3D point cloud based on the one at step S103 calculated disparity image.

At step S107 the processing section calculates 19 an MLS map based on the 3D point cloud.

At step S109 recognizes the processing section 19 Obstacle shadows SH in the MLS map and ignores these areas for the purpose of detecting terrain that corresponds to a body of water or a ditch.

At step S111 identifies the processing section 19 the next empty area ER of the MLS map that corresponds to the terrain that lies within the predicted path, which is a substantially continuous range of cells of the MLS map with a number of data points smaller than a given one Threshold for the data point density in the empty area.

At step S113 the processing section determines 19 the slope of the terrain immediately before the next empty area ER of the terrain in a direction parallel to the instantaneous longitudinal axis of the vehicle.

At step S115 the processing section determines whether the slope exceeds a predetermined peak gradient value and a width of the blank area exceeds a predetermined peak width value.

If the slope does not exceed the predetermined peak gradient value or the width of the empty area does not exceed the predetermined peak width value, the processing section moves 19 at step S117 continued.

When the slope exceeds the predetermined peak slope value and the width of the blank area exceeds the predetermined peak width value, the processing section gives 19 a signal indicating that a vertex in front of the vehicle 100 was recorded. The processing section 19 Also outputs a signal indicating the distance of the comb from the vehicle 100 displays. The processing section 19 then moves to step S101 continued.

At step S117 the processing section checks the RGB color values of each of the pixels present in the area of one of the stereoscopic image pairs (in the present embodiment, the left image) shown in step S101 which corresponds to the empty area of the considered MLS card. The processing section 19 calculates an average of the R, G and B color values of the pixels in this area, which in the above discussion has been termed a trimmed polygonal area.

At step S119 the processing section attempts to determine whether the pixels identified as corresponding to the empty area in the MLS card correspond to the reflections of objects (near the terrain area corresponding to the empty area of the MLS card, such as a bush or tree which adjoins the area or far from it, such as the sky). The processing section 19 achieves this by defining an extended polygonal area in the manner described above and the corresponding area in a disparity image which is generated based on the stereoscopic image pair from which the left image is being analyzed. The processing section 19 identifies pixels of the left image pair corresponding to the blank area corresponding to one pixel of the disparity image with a disparity that is greater than a predetermined disparity image deviation from the camera system 185C corresponds to a distance of 40 m in the present embodiment. Other values may be useful in some embodiments. Such pixels are considered reflections of distant objects.

The processing section 19 identifies pixels that correspond to mirror reflections of nearby objects by taking into account the disparity values in columns of pixels of the left image that correspond to the empty area. The processing section 19 compares the disparity values of the pixels in each column within the empty area with the disparity values of the "end" pixels of each column, ie the pixels at the top and bottom of each column, at the boundary of the empty area with respect to that column. The processing section then counts the number of pixels within the column that have disparity values that are less than the values of the disparity of the end pixels. These pixels correspond to the mirror reflections of nearby objects.

The processing section 19 then sums the number of pixels corresponding to mirror reflections from near and far objects to produce the speckleCount value.

At step S121 the processing section determines 19 then whether the blank area corresponds to a dark area or an oversized area by comparing the value of the average RGB color value with the upper and lower color thresholds and the value of speckleCount with the upper and lower speckleCount thresholds. When the average RGB color value. As described above, when the average color value is smaller than the lower color value threshold and the value of speckleCount is below the lower SpeckleCount threshold, the polygonal region is classified as a dark region. If the average color value is greater than the upper color threshold and the value of speckleCount is below the lower speckleCount threshold, the polygonal region is classified as an over-bright region. If the blank area corresponds to a dark or oversized area, the area is considered not to be a trench or body of water and the processing section 19 moves at step S121 continued.

At step S123 the processing section determines 19 the processing portion that the empty area corresponds to a body of water when (a) the value of speckleCount exceeds the upper speckle count threshold and (b) the average width of the extended trimmed polygonal area is greater than the width threshold and the height difference (determined from the MLS Card) is smaller than the height difference threshold between cells C at opposite ends of a cell gap spanning the empty area. In the present embodiment, the processing section looks 19 each column of cells across the width of the empty area. In some embodiments, a reduced number of columns is considered, eg, columns at locations where there is a distance of (eg) 25%, 50%, and 75% of the page width of the empty area from a side edge of the empty area. Other arrangements may be useful in some embodiments.

If at step S123 the processing section 19 has determined that the empty area corresponds to a body of water, gives the vehicle 100 a signal indicating the presence of a body of water in front of the vehicle 100 and a signal representing the distance of the empty area from the vehicle 100 displays. The processing section 19 then moves to step S101 continued.

If at step S123 the processing section 19 has determined that the empty area does not correspond to a body of water, moves the processing section 19 at step S127 continued. At step S127 The processing section outputs a signal indicating that a trench exists in front of the vehicle and a signal indicating the distance of the empty area from the vehicle 100 displays. The processing section 19 then moves to step S101 continued.

Some embodiments of the present invention enable vehicle operation with increased peace of mind when driving through terrain. This is at least partially relieved of the driver when operating with active LSP control system 12 due. Because a driver does not have to manually reduce the vehicle speed when approaching a terrain function in the form of a body of water or a ditch. Rather, the LSP control system causes 12 automatically reducing speed in response to detection of the off-road function.

It should be understood that the above-described embodiments are given by way of example only and are not intended to limit the invention, the scope of which is defined in the appended claims.

Throughout the specification and claims of this specification, the words "comprise" and "contain" and variations of the words, For example, "comprise" and "comprise" means "including, but not limited to," and are not intended to exclude (other than) other entities, additives, components, integers, or steps.

Throughout the specification and claims of this specification, the singular includes plural, unless the context requires otherwise. In particular, when the indefinite article is used, the specification should be understood to consider both plurality and singularity unless the context requires otherwise.

Features, integers, features, compounds, chemical entities, or groups described in connection with a particular aspect, embodiment, or example of the invention are to be understood to be in any other aspect, every alternate embodiment, or any other are applicable, unless they are incompatible with it.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• GB 2507622 [0001]
• GB 2499461 [0001]
• US 7349776 [0001]
• WO 2013124321 [0001]
• WO 2014/139875 [0001]
• GB 2492748 [0001, 0062]
• GB 2492655 [0001, 0062]
• GB 2499279 [0001, 0062]
• GB 2508464 [0001]

Claims (22)

A computer system for a vehicle, the system comprising processing means arranged to receive terrain information from terrain data acquisition means comprising a stereoscopic camera system indicative of the topography of the terrain extending in front of the vehicle, the terrain information being at least one stereo image pair terrain in front of the vehicle, the processing means being configured to: Calculating a 3D point cloud datum in relation to the terrain in front of the vehicle based on the terrain information; Identifying a substantially continuous terrain area in front of the vehicle and within a predicted path of the vehicle in which the number of data points in the 3D point cloud corresponding to that area is less than a threshold empty point data point density, the area being an empty area is considered if the number of data points in the 3D point cloud corresponding to that area is smaller than a data point density threshold for an empty area; Determine if the empty area corresponds to a body of water, which is an area that meets the criteria: the number of pixels in an image of this area taken by the camera system and corresponding to the specular reflection of a feature of the environment, including the terrain, exceeds a specular reflection density threshold; and a height difference between at least two positions of the range is smaller than a threshold value for the altitude difference of the location, wherein the system is configured to output a signal depending on whether the terrain corresponding to a body of water has been identified in front of the vehicle. System after Claim 1 wherein, in order that the system may determine that an empty area corresponds to a body of water, the further condition that the height difference between at least two positions on substantially opposite sides of the area is smaller than a height difference threshold is configured , System after Claim 1 or Claim 2 , which is further configured to determine the terrain slope to be experienced by the vehicle immediately before the empty area within the predicted path, wherein if the slope exceeds a predetermined slope value, the system determines that the area does not correspond to a body of water , The system of any preceding claim, further configured to determine the terrain slope that the vehicle has to experience immediately prior to the empty area within the predicted path, wherein if the slope exceeds a predetermined slope, the system determines that the area does not correspond to a body of water and can correspond to a crest of a slope. The system of any preceding claim, further configured to identify the empty area in at least one of the stereoscopic image pairs corresponding to the 3D point cloud and to determine whether the empty area corresponds to a body of water in further dependence on at least partially RGB. Color values of at least one pixel that corresponds to the empty area. System after Claim 5 configured to determine whether the empty area corresponds to a body of water at least partially corresponding to an average RGB color value over a plurality of pixels corresponding to the empty area. System after Claim 5 configured to determine that the blank area does not correspond to a body of water unless the average RGB color value over a plurality of pixels corresponding to the blank area is above a predetermined minimum average RGB color value and below a predetermined maximum average RGB color value. The system of any preceding claim, wherein the system is configured to determine whether the number of pixels in an image of the blank area captured by the camera system corresponding to the specular reflection of a feature of the environment, including the terrain Threshold of mirror reflection density, wherein the system is configured to determine the difference between the corresponding pixels of the left and right stereo image pairs corresponding to the blank area, the number of pixels corresponding to the mirror reflection being dependent on the difference between the mirror reflections corresponding pixels of the left and right image pair is determined. The system of any preceding claim, wherein the system is further configured to require that a predetermined width condition must be satisfied with respect to a width of the area of at least one of the stereoscopic image pairs corresponding to the blank area to empty the blank area Range corresponds. System after Claim 9 wherein the predetermined width condition includes the condition that a width or average width of at least a portion of the image including the blank area exceeds a predetermined threshold. The system of any preceding claim, comprising a speed controller, wherein the speed controller is configured to control the vehicle speed based at least in part on whether a body of water in the path of the vehicle has been identified. System after Claim 11 that is configured to provide a speed control output indicative of a recommended maximum speed depending, at least in part, on whether a body of water has been identified in the vehicle's driveway. System after Claim 11 or 12 , which is configured to alert a driver depending on whether a body of water has been identified in the path of the vehicle. A system according to any preceding claim, wherein the terrain data acquisition means comprises a stereoscopic camera system. The system of any preceding claim, further comprising the terrain data acquisition means. The system of any preceding claim, wherein the system includes an electronic processor having an electrical input for receiving terrain information indicating the topography of the terrain in front of the vehicle; and an electronic storage device electrically coupled to the electronic processor and instructions stored therein; wherein the processor is configured to access the storage device and execute the instructions stored therein so as to be operable to: Calculating a 3D point cloud datum in relation to the terrain in front of the vehicle based on the terrain information; Identifying a substantially continuous terrain area in front of the vehicle and within a predicted path of the vehicle in which the number of data points in the 3D point cloud corresponding to that area is less than a threshold empty point data point density, the area being an empty area is considered if the number of data points in the 3D point cloud corresponding to this area is smaller than a threshold for the data point density in the empty area; Determine if the empty area corresponds to a body of water, which is an area that meets the criteria: the number of pixels in an image of this area taken by the camera system and corresponding to the specular reflection of a feature of the environment, including the terrain, exceeds a specular reflection density threshold; and a height difference between at least two positions of the range is smaller than a threshold value for the altitude difference of the location, and Output of a signal depending on whether a terrain corresponding to a body of water has been identified in front of the vehicle. A vehicle comprising a system according to any preceding claim. A method for identifying terrain corresponding to a body of water, which method is implemented by means of a computer system of a vehicle, the method comprising receiving terrain information from terrain data acquisition means comprising a stereoscopic camera system indicating the topography of the terrain that preceded the terrain Vehicle, wherein the terrain information includes at least one stereo image pair of terrain in front of the vehicle, the method comprising: calculating a 3D point cloud data set relative to the terrain in front of the vehicle based on the terrain information; Identifying a substantially continuous terrain area in front of the vehicle and within a predicted path of the vehicle in which the number of data points in the 3D point cloud corresponding to that area is less than a threshold empty point data point density, said area an empty area is considered if the number of data points in the 3D point cloud corresponding to this area is smaller than a threshold for the data point density in the empty area; Determining whether the empty area corresponds to a body of water, which is an area that meets the following criteria: the number of pixels in an image of that area taken by the camera system and the specular reflection of a feature of the environment, including the Terrain, exceeds a threshold of specular reflection density; and a height difference between at least two positions of the range is smaller than a threshold value for the altitude difference of the location, the method comprising outputting a signal depending on whether the terrain corresponding to a body of water has been identified in front of the vehicle. A non-transitory computer-readable carrier medium having computer-readable code for controlling a vehicle to perform the method of Claim 18 , A computer program product executable on a processor to perform the method Claim 18 to implement. Non-volatile, computer-readable medium that copes with the computer program product Claim 20 loaded. Processor that is arranged to follow the procedure Claim 18 or the computer program product Claim 20 implemented.

Images