WO2008149049A1

WO2008149049A1 - A vehicle safety system

Info

Publication number: WO2008149049A1
Application number: PCT/GB2007/002057
Authority: WO
Inventors: Elisabeth Agren; Stefan Johansson; Jonas Elbornsson Skogsstjarna; Mattias Hanqvist
Original assignee: Autoliv Development Ab; Beattie, Alex
Priority date: 2007-06-05
Filing date: 2007-06-05
Publication date: 2008-12-11

Abstract

A vehicle safety system, comprising: an image capture device to capture image data of an object in the vicinity of the vehicle; and a processor operable to: analyse the image data using a plurality of filters, in parallel, each filter assuming that the object is a certain type of object and calculating at least the distance to the object on the basis of this assumption; and input the results from the filters into an evaluation algorithm, the evaluation algorithm using the results to determine a probable distance to the object.

Description

"A Vehicle Safety System"

Description of Invention

THIS INVENTION relates to a vehicle safety system, and in particular concerns a system for detecting, and determining the distance to, objects around a vehicle.

When using a vehicle-mounted "mono" camera, i.e. when stereo cameras are not available, it is a challenge to determine accurately the distance from the vehicle to an object that has been detected by the camera. From a single frame image from a mono camera, the angular position and the angular side of an object (i.e. the amount of the image which is taken up by the object) can readily be determined. If the real size of the object is known, then the distance to the object can be accurately determined. Unfortunately, in most cases the real size of the object will not be known.

If the object appears to be of a certain type (for instance, an adult human), an approximate determination of the distance to the object can be determined by assuming that the object is the same height as an average adult.

If the real height of the object is not known, and cannot be approximated, the distance of the object can be estimated by analysing the rate of change of the angular size of the object as the vehicle approaches the object, and hence the distance from the vehicle to the object decreases (i.e. the scale change of the object). This technique assumes that the speed of the object is much lower than that of the vehicle. The scale change of an object can be calculated by comparing consecutive frames from a mono camera. However, this method is generally ineffective if the distance between the vehicle and the object is relatively large.

If the distance between the vehicle and an object can be determined accurately, then a decision can be made reliably as to whether any action is to be taken. For instance, a warning could be given to the driver that the speed of the vehicle must be reduced, emergency braking of the vehicle may be applied in an attempt to bring the vehicle to a halt before the vehicle strikes the object, or a pedestrian safety system such as a hood lifter may be activated. However, if the distance to the object is calculated incorrectly, distracting warnings may be activated needlessly, emergency braking or a hood lifter may be activated unnecessarily, which can be dangerous for the driver and other road users, or indeed a vehicle may be involved in an accident without adequate warning having been given or preventative measures having been taken.

It is an object of the present invention to seek to provide an improved vehicle safety system of this type.

Accordingly, one aspect of the present invention provides a vehicle safety system, comprising: an image capture device to capture image data of an object in the vicinity of the vehicle; and a processor operable to: analyse the image data using a plurality of filters, in parallel, each filter assuming that the object is a certain type of object and calculating at least the distance to the object on the basis of this assumption; and input the results from the filters into an evaluation algorithm, the evaluation algorithm using the results to determine a probable distance to the object. Advantageously, at least a first filter assumes that the object to an adult human, and a second filter assumes that the object is a child.

Preferably, at least one further filter assumes that the object is a cyclist or one of a plurality of different types of vehicle.

Conveniently, each filter is a Kalman filter.

Advantageously, the image data comprises at least the angular position and angular size of the object as seen by the image capture device.

Preferably, the system further comprises a classification algorithm which analyses the image data to provide a determination as to the classification into which the object falls.

Conveniently, the classification algorithm is applied in parallel with the analysis of the image data by the filters.

Advantageously, the classification algorithm applies a set of rules to the apparent shape of the object to determine the likely classification into which the object falls.

Preferably, the classification algorithm compares the apparent shape of the object to a number of templates to determine the likely classification into which the object falls.

Conveniently, at least some of the rules or templates correspond to different poses or positions of the same object. Advantageously, the determined probable distance to the object is calculated by selecting from the outputs of the filters, based on the output from the classification algorithm.

Preferably, the output from one of the filters is selected, based on the output from the classification algorithm.

Conveniently, a weighted average of the outputs of the filters is used to calculate the probable distance to the object, based on the output from the classification algorithm.

Advantageously, an output is generated if it is determined that a hazardous situation may arise.

Preferably, the output is a warning output, to cause a warning to be provided to a driver of the vehicle.

Conveniently, the output is a control output, to cause one or more active vehicle control systems to be activated.

Advantageously, the active vehicle control systems comprise active braking or active steering.

Preferably, the output signal is a pedestrian protection signal, causing the activation of one of more pedestrian protection system of the vehicle.

Conveniently, an output signal is generated if the object is located within a first region defined in front of the vehicle. Advantageously, if the object is determined to be in a second region which is not directly in front of the vehicle, an output signal will be generated only if it is determined that the object is moving towards the first region.

Preferably, the image capture device is sensitive to visible light.

Conveniently, the image capture device is sensitive to infrared radiation.

Another aspect of the present invention provides a computer program comprising a computer program code adapted to cause all of the features of any one of the preceding claims to occur when the program is run on a computer.

In order that the present invention may be more readily understood, embodiments thereof will now be described, by way of example, with reference to the accompanying drawings. In which:

Figure 1 shows a schematic view of a situation in which a system embodying the present invention may be used;

Figure 2 is a schematic view of components of a system embodying the present invention; and

Figure 3 is a further schematic view of a situation in which embodiments of the present invention may be useful.

With reference firstly to figure 1 , a vehicle 1 is shown, travelling in a forward direction generally designated by an arrow 2. A camera 3 is mounted on the vehicle, and arranged so that the field of view 4 of the camera 3 points generally forwardly of the vehicle 1. The camera 3 may be adapted to detect light in optical frequencies, or alternatively may detect alternative radiation, for instance infrared radiation. The camera 3 is a mono camera.

A pedestrian 5 and another vehicle 6 are ahead of the vehicle 1 , and fall within the field of view 4 of the camera 3.

The shapes of the pedestrian 5 and the other vehicle 6 will appear in image data captured by the camera 3.

With reference to figure 2, image data gathered by the camera 3 is passed to an object detection and classification device 7 as is known in the art. Objects that are "of interest" can be extracted from the image data.

The image data relating to the objects that are of interest are passed to a filter bank 8. Within the filter bank 8, the image data is analysed in parallel according to several filters or algorithms, which may be Kalman filters, as will be understood by those skilled in the art. Each of the filters attempts to calculate the distance to the object based upon the assumption that the object is of a particular type. In preferred embodiments of the invention, one of the filters assumes that the object is an adult human, another filter assumes that the object is an older child, and a third filter assumes that the object is a younger child.

The invention is not, of course, limited to these filters, and further filters may assume that the object is a cyclist, or a vehicle of a certain type (for instance, a saloon car, an SUV, a van or a lorry).

The image data is processed in parallel (i.e. substantially simultaneously) by each of the filters, and each of the filters generates a calculated distance to the object. Each filter may also, in preferred embodiments of the invention, B2007/002057

calculate a position of the object, and/or a velocity of the object relative to the vehicle or to the ground.

The results from the filters are passed to an evaluation unit 9, the function of which will be described in more detail below.

The image data is also passed to a classification unit 10, which operates in parallel with the filter bank 8. The classification unit 10 compares the apparent shape of the object with various templates or rules and attempts to classify the object. For instance, the classification unit 10 may have a template or set of rules relating to an adult human, a further template or set of rules relating to an older child, and another template or set of rules relating to a younger child. The classification unit 10 attempts to determine the category of objects to which the object that has been detected by the camera 3 belongs.

For instance, the ratio of head size to body size of an adult human is generally rather less than that for children, whose heads are relatively large compared to their bodies, and hence if a set of rules is used then the ratio of head size to body size may be used as one criterion for distinguishing between these two types of object. Other factors may be used, of course, to attempt to classify the object correctly, and preferably several different criteria are applied.

If the shape of an object is considered against one or more templates of shapes, then several templates may be provided for each shape. For instance, the shape of a cyclist when seen side-on is likely to be rather different than the shape of the cyclist when seen head-on. In preferred embodiments, the template against which an object fits most closely may be used to gather information about likely movement of the object. It will be understood that a cyclist that is side-on is likely to move across the path of the vehicle 1 , whereas a cyclist that appears to be head-on is more likely to be moving towards or away from the vehicle 1.

In generating rules or templates to determine the class of a detected object, the rules or templates will preferably be "trained" using a large number of real objects.

The classification unit 10 may arrive at a decision as to the most likely correct classification of the object. Alternatively, the classification unit 10 could provide a series of outputs, each relating to one potential object classification, providing an indication of the likelihood that the object falls into that particular classification. In either event, the output from the classification unit 10 is also passed to the evaluation unit 9.

The evaluation unit 9 uses at least the outputs from the filter bank 8 to determine a most probable distance to the object. If the distance to the object calculated by each of the filters is identical, or approximately the same, then the evaluation unit 9 will simply conclude that the distance to the object is that arrived at by each of the filters.

However, if there is a significant difference in the distance calculated by each of the filters, the evaluation unit 9 will use the output from the classification unit 10 to decide which of the filters is likely to produce the correct result.

In certain embodiments, a most likely classification will be selected, and the calculated distance produced by the corresponding filter will be used. For instance, if it is considered most likely that the object is an adult human, the result from the filter that works on the assumption that the object is an adult human will be used, with the results from other filters being discarded. Alternatively, a weighted average of the outputs from the filters may be used. The classification unit may indicate that there is an 80% probability that the object is an adult human, and a 20% probability that the object is an older child. In this case, the most probable distance to the object may be calculated as being an average of the distances calculated by the filters which assume that the object is an adult human and an older child respectively, with the average being weighted appropriately towards the figure derived from the filter that assumes the object is an adult human.

It will be appreciated that this method confers significant advantages when compared to prior art techniques. In known safety systems, a classification algorithm must first be carried out, and only when the object has been classified can a further algorithm be performed to attempt to calculate the distance to the object. However, by performing calculations in parallel based on several different classifications types, and combining these results (if necessary) from the output of a classification routine which is also run in parallel, a reliable calculation of the distance to the object an be arrived at in a short length of time.

In preferred embodiments of the invention, an algorithm which attempts to determine the distance to the object using the scale change of the object, as discussed above, may also be applied to the image data of successive frames gathered by the camera 3. The result generated by this algorithm may be used as an additional factor in determining the most probable distance to the object

The evaluation unit 9 also makes a determination as to whether any vehicle systems should be activated in response to the most probable distance from the vehicle to the object. For instance, if it appears that the driver will soon need to reduce the speed of the vehicle in order to avoid a collision with the object, or another potentially hazardous situation, a warning may be provided to the driver. The warning may take the form of a visual, audible or haptic alert, or any combination of these.

Alternatively, if it appears that action must be taken very swiftly in order to avoid a collision, active braking and/or steering systems may be activated in order to bring the vehicle to a halt and hopefully avoid a collision with the object.

Further, if it appears that a collision with the object is unavoidable, then one or more vehicle safety system may be activated. These safety systems may operate to the benefit of the vehicle occupants, for instance air-bags or seatbelt pretensioners. They may also act for the benefit of pedestrians or cyclists, and for instance may comprise a bonnet lifter to lessen the likely harm of the impact.

In preferred embodiments of the invention, the generation of an output signal by the evaluation unit 9 to cause one or more of these events to happen will be considered if the vehicle is likely to arrive at the object within a time period of three to five seconds, and most preferably four seconds. At 70 km per hour for instance, objects less than 78m from the vehicle 1 will fall into this category. The invention is not limited to this, and objects further away from the vehicle 1 than this may also be considered, particularly if the vehicle 1 is travelling at higher speed.

With reference to figure 3, in preferred embodiments of the invention a first region 11 is defined in front of the vehicle 1 , with second regions 12 being defined to either side of the first region 11. In general terms, the first region 11 is defined such that, if an object placed within the first region 11, the object is likely to be struck by the vehicle as the vehicle 1 proceeds.

In preferred embodiments of the invention, objects that are determined to be within the second region 12 will not give rise to a warning or the activation of any vehicle systems, unless it appears that the object is moving towards the first region 11 , i.e. into the path of the vehicle 1. In this case, such objects will give rise to a warning, or to the activation of vehicle systems, as discussed above.

It will be appreciated, with reference to figure 3, that the correct distance to the object will often need to be determined before it can be decided whether an object is within the first region 11 or the second region 12. If an object is not straight ahead of the vehicle, then an object of a certain apparent size may fall within the first region 11 if it is a relatively small object 13 which is close to the vehicle, but may fall within the second region 12 if it is a relatively large object 14 further from the vehicle 1.

It will be appreciated that embodiments of the present invention provide an effective system for determining the distances of objects from a vehicle, allowing accurate determinations of distances to be arrived at more swiftly than is possible with known systems.

When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components.

The features disclosed in the foregoing description, or the following claims, or the accompanying drawings, expressed in their specific forms or in terms of a means for performing the disclosed function, or a method or process for attaining the disclosed result, as appropriate, may, separately, or in any combination of such features, be utilised for realising the invention in diverse forms thereof.

Claims

CLAIMS:

1. A vehicle safety system, comprising: an image capture device to capture image data of an object in the vicinity of the vehicle; and a processor operable to: analyse the image data using a plurality of filters, in parallel, each filter assuming that the object is a certain type of object and calculating at least the distance to the object on the basis of this assumption; and input the results from the filters into an evaluation algorithm, the evaluation algorithm using the results to determine a probable distance to the object.

2. A system according to claim 1 , wherein at least a first filter assumes that the object to an adult human, and a second filter assumes that the object is a child.

3. A system according to claim 1 , wherein at least one further filter assumes that the object is a cyclist or one of a plurality of different types of vehicle.

4. A system according to any preceding claim, wherein each filter is a Kalman filter.

5. A system according to any preceding claim, wherein the image data comprises at least the angular position and angular size of the object as seen

K b\y/ t thhαe i immaαπg_es r caanpftiu llr"oe H dθeΛv/ii/c"ϊβe.

6. A system according to any preceding claim, further comprising a classification algorithm which analyses the image data to provide a determination as to the classification into which the object falls.

7. A system according to claim 5, wherein the classification algorithm is applied in parallel with the analysis of the image data by the filters.

8. A system according to claim 6 or 7, wherein the classification algorithm applies a set of rules to the apparent shape of the object to determine the likely classification into which the object falls.

9. A system according to any one of claims 6 to 8, wherein the classification algorithm compares the apparent shape of the object to a number of templates to determine the likely classification into which the object falls.

10. A system according to claim 8 or 9, wherein at least some of the rules or templates correspond to different poses or positions of the same object.

11. A system according to any one of claims 6 to 10, wherein the determined probable distance to the object is calculated by selecting from the outputs of the filters, based on the output from the classification algorithm.

12. A system according to claim 11, wherein the output from one of the filters is selected, based on the output from the classification algorithm.

13. A system according to claim 11 , wherein a weighted average of the outputs of the filters is used to calculate the probable distance to the object, based on the output from the classification algorithm.

14. A system according to any preceding claim, in which an output is generated if it is determined that a hazardous situation may arise;

15. A system according to claim 14, wherein the output is a warning output, to cause a warning to be provided to a driver of the vehicle.

16. A system according to claim 14 or 15, in which the output is a control output, to cause one or more active vehicle control systems to be activated.

17. A system according to claim 12, wherein the active vehicle controi systems comprise active braking or active steering.

18. A system according to any one of claims 14 to 17, wherein the output signal is a pedestrian protection signal, causing the activation of one of more pedestrian protection system of the vehicle.

19. A system according to any one of claims 14 to 18, wherein an output signal is generated if the object is located within a first region defined in front of the vehicle.

20. A system according to any one of claims 14 to 19wherein, if the object is determined to be in a second region which is not directly in front of the vehicle, an output signal will be generated only if it is determined that the object is moving towards the first region.

21. A system according to any preceding claim, wherein the image capture device is sensitive to visible light.

22. A system according to any preceding claim, wherein the image capture device is sensitive to infrared radiation.

23. A computer program comprising a computer program code adapted to cause all of the features of any one of the preceding claims to occur when the program is run on a computer.