FR3072931A1 - DATA PROCESSING METHOD FOR VEHICLE DRIVER ASSISTANCE SYSTEM AND DRIVING ASSISTANCE SYSTEM - Google Patents

Abstract

The invention relates to a data processing method for a driving assistance system (1) of a vehicle (3) comprising the following steps: a) acquisition of a set of data relating to the environment of the vehicle, said data set comprising a first set of data relating to a first part of the vehicle environment and a second set of data relating to a second part of the vehicle environment, b) processing the first set of data according to a first process and at a first instant, c) processing the second set of data according to a second process distinct from the first process or at a second distinct time from the first instant. Such a driver assistance system is also described.

Description

The present invention relates generally to the field of processing the data acquired by a sensor.

It relates more particularly to a method of processing data acquired by the sensor of a driving assistance system for a vehicle and such a driving assistance system.

Technological background

It is known to equip vehicles with a driving assistance system which makes it possible to analyze the environment of the vehicle and to provide the driver with information facilitating driving.

Such driving assistance systems generally comprise a data sensor, of the video camera type, capable of acquiring images of the environment of the vehicle, a processing unit capable of applying processing algorithms to the data acquired in order to extract the relevant information therefrom, as well as a display unit allowing the driver to view the extracted information.

However, the amount of data to be processed as well as the increasing complexity of the processing algorithms used require powerful hardware resources (microprocessor, memory, etc.).

It is also desirable in some cases to display relevant information quickly after data acquisition (if possible almost in real time).

Object of the invention

In this context, the present invention provides a data processing method for a vehicle driving assistance system comprising a step of acquiring a set of data relating to the environment of the vehicle, said set of data. comprising a first set of data relating to a first part of the vehicle environment and a second set of data relating to a second part of the vehicle environment,

According to the invention, the method further comprises steps of:

- processing of the first set of data according to a first process and at a first instant,

processing of the second set of data according to a second process distinct from the first process or at a second instant distinct from the first instant.

By distinguishing several sets of data and applying different treatments to them and / or at different times, the driver assistance system makes it possible to reduce the volume of data processed and to increase the speed of display of the information obtained. Indeed, the system allows to allocate more material resources to the processing of data sets allowing to obtain the most relevant information than to those containing little interest.

Other non-limiting and advantageous characteristics of the process according to the invention, taken individually or in any technically possible combination, are the following:

- the second set of data is processed at the second instant and the second instant is after the first instant,

- the first process comprises a sampling of the first set of data according to a first sampling rate,

the second process comprises sampling the second set of data according to a second sampling rate, the first sampling rate being greater than the second sampling rate,

- the sampling of the first data set and the sampling of the second data set are spatial samplings,

- the sampling of the first data set and the sampling of the second data set are temporal samples,

the method further comprises a step of detecting an object located in the environment of the vehicle and / or a step of determining a subset of data corresponding to said object in the data set,

- a number of algorithms is applied to the data subset, and in which said number of algorithms is a function of a location of the object,

the location of the object comprises a membership of the data subset in the first data set (the data subset then forming a first data subset), or in the second data set (the subset then forming a second subset of data),

- a first number of algorithm (s) is applied to said first subset of data,

- a second number of algorithm (s) is applied to said second subset of data, the first number of algorithms being greater than the second number of algorithms,

- the location of the object includes a distance between the object and the vehicle,

- the distance is measured using a telemetry device,

the method further comprises a step of superimposing information obtained by processing the first set of data on the object by means of a head-up display (for example an head-up display with augmented reality),

- the set of acquired data represents an acquired image,

- the first set of data represents a first part of the image,

- the first part of the image corresponds to a display area of the driving assistance system.

The invention also provides a vehicle driving assistance system comprising:

a sensor adapted to acquire a set of data relating to the environment of the vehicle, said set of data comprising a first set of data relating to a first part of the environment of the vehicle and a second set of data relating to a second part vehicle environment,

an electronic processing unit capable of processing the first set of data according to a first process and at a first instant, and capable of processing the second set of data according to a second process distinct from the first process or at a second instant distinct from the first instant .

Other non-limiting and advantageous characteristics of the system according to the invention, taken individually or in any technically possible combination, are the following:

- the system includes a head-up display (for example an augmented reality head-up display),

- the electronic processing unit is capable of producing information by means of the first process,

- said head-up display (augmented reality) is capable of superimposing said information on an object located in the environment of the vehicle.

Detailed description of an exemplary embodiment

The description which follows with reference to the appended drawings, given by way of nonlimiting examples, will make it clear what the invention consists of and how it can be carried out.

In the accompanying drawings:

- Figure 1 shows schematically, a vehicle seen from above equipped with a driving assistance system,

FIG. 2 schematically represents an image acquired by the driving assistance system and processed using the data processing method according to the invention,

FIG. 3 schematically represents an image acquired by the driving assistance system and processed according to a variant of the data processing method, and

- Figure 4 schematically represents an image acquired by the driving assistance system and processed according to another variant of the data processing method.

In Figure 1, there is shown schematically the main elements of a driving assistance system 1 intended to equip a vehicle 3, for example a motor vehicle, a train, a boat such as a barge, a tram or a bus, to facilitate driving.

This driving assistance system 1 here includes a head-up display. The head-up display includes a light source, a projection unit and a partially transparent slide. The partially transparent blade serves as display area 5 (see FIG. 2) and allows the display of information 2 relating to the vehicle 3 and / or to its environment in the form of a virtual image. These elements of the head-up display are known to a person skilled in the art and will not be described in more detail.

The driving assistance system 1 described here is particularly advantageous when the aforementioned head-up display is an augmented reality head-up display. Such an augmented reality head-up display makes it possible to superimpose (for the driver of the vehicle) the information (s) displayed (s) 2 on an object 7 (pedestrian, animal, other vehicle, etc.) located in front of the vehicle 3.

For this, the partially transparent blade comprises for example the windshield 9 of the vehicle 3, and the display area 5 can then extend over at least part of said windshield 9.

The driving assistance system 1 also comprises a detection device. Said detection device typically comprises a sensor adapted to acquire a set of data relating to the environment of the vehicle 3. The sensor comprises for example an image acquisition unit 11, for example a video camera. In this case, the set of data relating to the environment represents an acquired image 13 (see FIG. 2) and / or a sequence of images 13. The acquired image 13 comprises a plurality of pixels, generally arranged in the form of 'a matrix of pixels.

The image acquisition unit 11 is disposed at the front of the vehicle 3, for example at the level of the interior (central) rear view mirror of the vehicle 3. The image acquisition unit 11 has a field of view 111 which extends opposite the vehicle 3. Advantageously, the field angle 111 covers the solid angle corresponding to the windshield 9 of the vehicle (as seen from the driver) and extends beyond the solid angle corresponding to said windshield 9.

The driving assistance system 1 further comprises an electronic processing unit 15. The electronic processing unit 15 is programmed to process data of the acquired image 13.

This processing comprises at least one algorithm selected from a plurality of algorithms. Said plurality of algorithms is stored by the driving assistance system 1, for example in a memory unit 17.

The plurality of algorithms includes, for example, algorithms capable of obtaining information 2 on the acquired image 13, for example detecting the presence or absence of an object 7, identifying the object 7, tracking the object 7 on a sequence of acquired images 13, to predict the evolution of the position of the object 7 over time, to extract characteristics of the object 7 (distance, orientation, size, speed etc.), to be calculated the time remaining before a potential collision between the object 7 and the vehicle 3. The plurality of algorithms also includes sampling algorithms making it possible to sample the acquired image 13 and thus to reduce the volume of data to be processed by the after.

We now describe with reference to FIGS. 2 to 4 different possible examples for the image processing method implemented by the driving assistance system 1 of the vehicle 3.

Generally, during a step a) of acquiring the data set, the image acquisition unit 11 acquires the image 13 of the environment located at the front of the vehicle 3.

The data set includes a first data set and a second data set. The first set of data here represents a first part of image 19 relating to a first part of the environment. The second set of data represents a second part of image 21 relating to a second part of the environment.

In some of the examples described below, in order to distinguish the first image part 19 from the second image part 21, said first and second image parts 19, 21 have coordinates in pixels stored in the unit of memory 17.

In the case of a head-up display (possibly with augmented reality), the first image part 19 corresponds to the display area 5 of the head-up display, which corresponds as already indicated to at least one part of the windshield 9 of the vehicle 3.

It is here on this first part of image 19 that the rapid obtaining of information 2 relating to the environment is desired. Indeed, in the case of an augmented reality head-up display, the display area 5 is the area over which the information 2 is superimposed on the environment and therefore the area for which the image analysis acquired 13 benefits the driver the most.

The second image part 21 includes a peripheral zone surrounding the first image part 19.

The acquired image 13 is transmitted to the electronic processing unit 15.

During a step b) of processing the first image part 19, the electronic processing unit 15 applies a first processing process to the first image part 19 at a first instant h. More specifically, the first process is applied to the pixels of the first image part 19. The first process can in practice comprise a single first algorithm or a plurality of first algorithms.

During a detection sub-step, the first process uses for example an algorithm making it possible to detect whether an object 7 located in the environment of the vehicle 3 is more precisely in the solid angle corresponding to the area d 'display 5.

Then, during a determination sub-step, if an object 7 has been detected, the first process implements a second algorithm making it possible to extract a first subset of data 719 corresponding to the object 7 in the first part of image 19.

During a step c) of processing the second image part 21, the electronic processing unit 15 applies the second process to the pixels of the second image part 21.

According to a first embodiment of the invention, the electronic processing unit 15 applies the first process to the pixels of the second image part 21 at a second instant l ₂ distinct from the first instant h, the first instant h being earlier at the second instant l ₂ . Thus, the first image part 19 is processed as a priority, which allows rapid display of the information 2 obtained by the first process.

It is understood that it is proposed in this first embodiment to apply the same first process to the pixels of the first image part 19 and the second image part 21 (but at separate times).

To use the algorithms previously given as an example, during a detection sub-step, the first process implements the algorithm making it possible to detect whether an object 7 located in the environment of the vehicle 3 is more precisely outside of the solid angle corresponding to the display area 5.

Then, during a determination sub-step, if an object 7 has been detected, the first process makes it possible to extract a second subset of data 721 corresponding to the object 7 in the second image part 21 .

Finally, during a display step d), the driving assistance system 1 projects the information 2 onto the display area 5, which allows the driver to view it in the form of virtual images.

In the case of an augmented reality head-up display, the information 2 is projected so that it appears superimposed on the objects 7 located in the environment of the vehicle 3. According to the example described, the information 2 contains for example a symbol or an outline which, superimposed on the object 7 or placed near it, makes it possible to underline its presence. This information 2 is particularly relevant in the case of objects 7 that are difficult for the driver to see.

When this first embodiment is implemented (alone or in combination with others which will be described later), it is advantageous to perform the display step at the end of the treatment according to the first process and before initiate treatment according to the second process. Thus the information 2 obtained by means of the first process can be quickly displayed to the driver.

We now describe other embodiments of steps b) and c) of processing.

According to a second embodiment, the electronic processing unit 15 applies at the first instant h a second process distinct from the first process to the pixels of the second image part 21. The second process can comprise a single second algorithm or a plurality of second algorithms. In the case where the electronic processing unit 15 applies a plurality of second algorithms, the plurality of second algorithms may include one or more algorithms from the plurality of first algorithms.

This embodiment is particularly advantageous when an object 7 is detected in the environment of the vehicle 3. The electronic processing unit 15 can then apply a plurality of algorithms to the first subset 719 or to the second subset 721 in order to measure a plurality of characteristics of the object 7.

Indeed, it is more advantageous to obtain information 2 for a first subset 719 than for a second subset 721. Indeed, for such a second subset, no information 2 can be superimposed by the augmented reality head-up display to the object 7 which it represents (these objects 7 being outside the solid angle corresponding to the display area 5).

For example, the electronic processing unit 15 can apply a first number of first algorithms to the pixels of the first subset

719, and a second number of second algorithms to the pixels of the second subset 721. The first number of first algorithms is greater than the second number of second algorithms, thus more information 2 is obtained on a first subset 719 (by example the size, the distance, the nature, etc. of the object 7 which it represents) than on a second subset 721 (for example only the distance)

Thus the data processing method makes it possible to reduce the number of operations performed by the electronic processing unit 15.

According to a third embodiment, the first process and the second process comprise a sampling of the pixels of the acquired image 13. During this sampling, the second image part 21 is undersampled with respect to the first image part 19 Only the sampled pixels will subsequently be processed according to other algorithms. This makes it possible to reduce the volume of data to be processed and thus speed up image processing while reducing the electronic resources required.

The sampling is for example implemented by the electronic processing unit 15.

According to a first variant of the third embodiment, the sampling comprises a spatial sampling of the pixels. The pixels of the first image part 19 are sampled according to a first spatial sampling rate t _s i. The pixels of the second image part 21 are sampled according to a second spatial sampling rate t _s2 .

The first spatial sampling rate t _s i is greater than the second spatial sampling rate t _S 2, thus the resolution of the first sampled image part is greater than the resolution of the second sampled image part. For example, the first spatial sampling rate t _s i is equal to 1, and the second spatial sampling rate t _s2 is equal to 1/2, 1/3, 1/4, etc.

For example, when the algorithm applied makes it possible to measure the size of an object 7, the size of an object 7 corresponding to the first subset 719 is calculated with more precision than the size of an object 7 corresponding to the second subset 721 (this size can then be underestimated or overestimated).

Thanks to these different sampling rates, the volume of data to be processed and therefore the number of operations to be carried out in order to obtain information 2 relating to the environment of the vehicle 3 is reduced, while maintaining good accuracy in the information 2 relevant.

According to a second variant of the third embodiment, the sampling comprises a temporal sampling of the pixels.

The pixels of the first image part 19 are sampled according to a first temporal sampling rate t _t i. The pixels of the second image part 21 are sampled according to a second temporal sampling rate t _t 2 The first temporal sampling rate tu is greater than the second temporal sampling rate t _t 2, thus the information 2 obtained is put updated more frequently for the first image part 19 than for the second image part 21. For example, the first temporal sampling rate t _t i is equal to 1, and the second temporal sampling rate t _t 2 equals 1/2, 1/3, 1/4, etc.

For example, when the algorithm applied makes it possible to measure the position of an object 7, the position of an object 7 corresponding to the first subset 719 is calculated on all the first parts of images 19 when the first rate of temporal sampling tu is equal to 1. The position of an object 7 corresponding to the second subset 721 is calculated one on a second part of the image out of 21 two when the second temporal sampling rate t _t 2 is equal to 1 / 2.

Thus, the position of the object 7 corresponding to the first subset 719 is known with precision over time, which allows for example a precise monitoring of said first object 719. On the other hand, the monitoring of object 7 corresponding to the second subset 721 is less precise.

By not processing all the pixels of the acquired image 13, the volume of data to be processed and therefore the number of operations to be carried out to obtain information 2 relating to the environment of the vehicle 3 is reduced.

The different embodiments as well as the variants can be combined with one another in order to further reduce the number of operations carried out. Thus the process gains speed and the need for material resources necessary for its implementation decreases.

According to an advantageous example illustrated in FIG. 4, the second image part 21 can comprise a plurality of zones, for example a first zone and a second zone. Here, the zones of the second image part 21 form peripheral zones 211, 213 arranged concentrically around the first image part 19, as shown in FIG. 3. According to this example, a first peripheral zone 211 surrounds the first image part 19 and a second peripheral zone 213 surrounds the first peripheral zone 211.

According to this example, the further the peripheral zone 211,213 is from the first image part 19, the less the information 2 that it contains is considered important.

The previously described embodiments and their variants can be applied to the peripheral areas 211,213 in order to further reduce the number of operations performed and increase the speed of calculation and display of information 2 on the display area 5 .

In the case of the first embodiment, the first image part 19 is processed at the first instant h, the first peripheral zone 211 is processed at the second instant h distinct from the first instant, and the second peripheral zone 213 is processed at a third instant l ₃ , said third instant being distinct from the first instant h and the second instant l ₂ . In addition, the third instant l ₃ is after the second instant l ₂ , the second instant l _{2 itself} being after the first instant h.

In the case of the second embodiment, the third number of third algorithms applied to the second peripheral area 213 is less than the second number of second algorithms applied to the first peripheral area 211. The second number of algorithms applied to the first peripheral area 211 is less than the first number of first algorithms applied to the first image part 19.

In the case of the third embodiment, the spatial sampling rates t _s and the temporal sampling rates t _t decrease as a function of the distance from the first image part 19.

For example, all the images are selected for the first image part 19, only one image out of two is selected for the first peripheral zone 211, and one image out of four is selected for the second peripheral zone 213. The information 2 extracted from peripheral areas 211,213 are therefore less precise than those of the first image part 19.

FIG. 4 illustrates the positions 7 _N , 7 _{N + 1} , 7 _{N + 2} , 7 _{N + 3} of an object 7 on a superposition of a sequence of acquired images N, N + 1, N + 2, N 3. Thus the initial position represents the object 7 on an initial image N etc.

The object 7, for example a pedestrian, is first of all detected at a first position 7 _N in the second peripheral zone 213 of the first acquired image N.

In this example, the second peripheral zone 213 has a temporal sampling rate t _s2 i3 is equal to 1/4, then the second position 7 _{N + 1} of the object 7 will not be known on the second captured image N + 1. Indeed, the object 7 is still in the second peripheral zone 213.

The temporal sampling rate t _s2 n of the first peripheral zone is equal to 1/2.

Between the acquisition of the second acquired image N + 1 and that of the third acquired image N + 2, the object 7 entered the first peripheral zone 211. The object 7 is therefore detected at a third position 7n + 2 in the first peripheral zone 211 on the third acquired image N + 2.

Then the object 7 enters the first image part 19, and its fourth position 7 _{N + 3} can be detected on the fourth acquired image N + 3.

If the object had remained in the first peripheral zone 211 or had returned to the second peripheral zone, its presence would not have been detected on the fourth image N + 3.

By further reducing the processing frequency of the peripheral areas 211, 213, the computational load necessary for processing the acquired image 13 is reduced.

It is also possible to increase the number of peripheral zones 211, 213 included in the second image part 21 in order to precisely control the number of operations performed.

The embodiments described and their variants can be combined.

According to a fourth embodiment, the driving assistance system 1 comprises a hybrid detection device. In this case, the hybrid detection device includes multiple sensors. For example, in addition to the image acquisition unit 11, the hybrid detection device comprises a telemetry device 23 making it possible to measure a distance between the object 7 and the vehicle

3.

The telemetry device 23 comprises for example a lidar (according to the English acronym of "Llght Detection And Ranging"), or even a radar. Said telemetry device 23 is capable of emitting an electromagnetic wave and of acquiring the electromagnetic wave reflected by the object 7 present in a detection field of the telemetry device 23. The distance between the object 7 and the vehicle 3 is then calculated by measuring a time of flight of the light wave or of the radio wave reflected by this object 7, this time of flight being obtained from the echo signal captured by this lidar or this radar.

It is proposed in this embodiment that each image part is determined as a set of pixels corresponding to objects located over a certain range of distances as measured by the telemetry device 23. (Each image part can then be made up of separate subparties.)

Thus the first part of image 19 can comprise all of the pixels corresponding to objects located at a measured distance less than a threshold value (for example 20 m).

The telemetry device 23 makes it possible to determine a first image portion 19 of reduced size and thus to increase the speed of the process and to reduce the number of operations.

For example, a plurality of objects 7 is detected by telemetry, each object 7 being at a different distance from the vehicle 3. The first image part 19 then corresponds to the first set of pixels representing an object, or to a plurality of 'sets of pixels representing a plurality of objects, located at a distance less than the threshold value. The second image part 21 then corresponds to the second set of pixels representing an object, or to a plurality of sets of pixels representing a plurality of objects, located at a distance greater than the threshold value. .

Advantageously, a plurality of threshold values can be stored in the memory unit 17 of the vehicle. Thus, the second image part 21 can comprise a plurality of zones defined as a function of the threshold values.

For example, the first area of the second image part 21 comprises the set or sets of pixels representing one or more objects located between 20 m and 30 m from the vehicle 3. The second area corresponds to the third set of pixels representing one or more objects located more than 30 m from the vehicle 3 etc.

This fourth embodiment can be combined with the other embodiments and their variants. Thus, the processes applied to the first image part 19, to the first zone and to the second zone can be such as those previously described. In addition, the information 2 corresponding to the object closest to the vehicle 3 can be displayed in priority.

The telemetry device 23 can also make it possible to confirm certain information 2 obtained thanks to the image acquisition unit 11, for example the distance between the object 7 and the vehicle 3.

For all these different embodiments, display step d) is carried out as described above.

Claims (15)

1. Data processing method for driving assistance system (1) of a vehicle (3) comprising the following steps: a) acquisition of a set of data relating to the environment of the vehicle (3), said set of data comprising a first set of data relating to a first part of the environment of the vehicle (3) and a second set of data relating to a second part of the environment of the vehicle (3), b) processing of the first set of data according to a first process and at a first instant (h), c) processing of the second set of data according to a second process distinct from the first process or at a second instant (l ₂ ) distinct from the first instant (h). 2. A data processing method according to claim 1, in which the second set of data is processed at the second instant (l ₂ ) and in which the second instant (l ₂ ) is after the first instant (h). The image processing method according to claim 1 or 2, in which the first process comprises sampling the first set of data according to a first sampling rate (t _s i, t _t i), and in which the second process includes sampling the second data set according to a second sampling rate (t _s2 , t _t2 ), the first sampling rate (t _s i, t _t i) being greater than the second sampling rate (t _s2 , W, 4. The image processing method according to claim 3, in which the sampling of the first set of data and the sampling of the second set of data are spatial samplings. 5. The image processing method according to claim 3, in which the sampling of the first set of data and the sampling of the second set of data are temporal samplings. 6. Image processing method according to one of the preceding claims, further comprising the following steps: - detection of an object (7) located in the environment of the vehicle (3), - determination of a subset of data (719, 721) corresponding to said object (7) in the data set. 7. The image processing method according to claim 6, in which a number of algorithms is applied to the subset of data (719, 721), and in which said number of algorithms is a function of a location of l object (7). 8. The image processing method according to claim 7, wherein the location of the object (7) comprises belonging to the subset of data (719, 721) to the first set of data or to the second set of data. 9. The image processing method according to claim 7, wherein the location of the object (7) comprises a distance between the object (7) and the vehicle (3). 10. The image processing method according to claim 9, wherein said distance is measured using a telemetry device (23). 11. Image processing method according to one of claims 6 to 10, further comprising a step of superimposing information (2) obtained by processing the first set of data on the object (7) by means a head-up display. 12. Image processing method according to one of the preceding claims, in which the set of acquired data represents an acquired image (13), in which the first set of data represents a first image part (19), and wherein said first image part (19) corresponds to a display area (5) of the driving assistance system (1). 13. Driving assistance system (1) for vehicle (3) comprising: - a sensor adapted to acquire a set of data relating to the environment of the vehicle (3), said set of data comprising a 5 first set of data relating to a first part of the vehicle environment (3) and a second set of data relating to a second part of the vehicle environment (3), an electronic processing unit (15) capable of processing the first set of data according to a first process and at a first instant (h), and 10 capable of processing the second set of data according to a second process distinct from the first process or at a second instant (l ₂ ) distinct from the first instant (h). 14. Driving assistance system (1) according to claim 13, 15 comprising a head-up display, the electronic processing unit (15) being capable of producing information by means of the first process, said head-up display being able to superimpose said information (2) on an object (7) located in the environment of the vehicle (3). 1/2