GB2610384A

GB2610384A - Camera position review for vehicle driver assistance systems

Info

Publication number: GB2610384A
Application number: GB2112432.6A
Authority: GB
Inventors: Davies Christopher; Daniel Gwen
Original assignee: Belron International Ltd
Current assignee: Belron International Ltd
Priority date: 2021-09-01
Filing date: 2021-09-01
Publication date: 2023-03-08
Also published as: GB202112432D0; AU2022336119A1; CA3228672A1; WO2023031264A1; EP4367636A1; CN118215938A

Abstract

A method and system 1 of investigating the accuracy of fitment for a vehicle replacement windscreen (10, Fig.3) having a mounted driver assistance camera 4,5 (DAC) utilising an illuminating radiation delivery device 2 and a spectral reflection profile (SRP) 7,8 from the DAC is imaged 3 and compared to a datum SRP. The datum SRP may result from a DAC fitted to the windscreen before replacement. A first SRP may be imaged with DAC in position on a damaged windscreen fitted to the vehicle and a second SRP may be imaged with DAC in position on the replacement windscreen fitted to the vehicle. The SRP may be captured using a camera, where the imaging and illumination source are positioned in a controlled or fixed relationship relative to one another and the vehicle. The imaged spectral reflection data may be processed to identify pitch 15 and yaw 14 components of the DAC about a first and second axis 12,13.

Description

Camera Position Review for Vehicle Driver Assistance Systems The present invention relates to camera position review/determination for vehicle driver assistance systems.

Vehicles may utilise advanced driver assistance systems (ADAS) which have components mounted to the windscreen. Forward facing cameras for ADAS systems are often windscreen mounted and provided with mounting brackets secured to the windscreen.

Typically, an Advanced Driver Assistant System (ADAS) uses a camera housed in the windscreen of modern cars to help the driver and vehicle have greater awareness of the driving conditions. They are usually positioned at the top of a windscreen looking forwards towards the road ahead. When windscreens are subjected to damage beyond repair, they are replaced. However, the ADAS (including the camera) will be transferred to the new replacement screen Due to the adhesive compound which secures the windscreen glazing panel to the vehicle windscreen frame, when a windscreen is replaced, it will be fitted to within a reasonably large tolerance. This can translate and rotate the camera away from its previous position.

The pitch angle tolerance can vary greatly depending on the Vehicle Manufacturer -from +/-0.15deg to +/-5deg. If a screen is fitted and the camera is outside of the acceptable tolerance, the screen must be removed and fitted again. Based upon investigation regarding which type of movement introduces the most challenge in re-alignment, it has been identified that primarily, pitch, yaw, and roll, are the most important parameters to accurately check in a camera fitted to a replacement windscreen. The detection for X & Y translation is a trivial activity using fundamental machine vision algorithms, a combination of edge detection and circularity measurements of the ADAS camera lens.

According to the invention, there is provided, in a first aspect, a method for investigating 30 the accuracy of fitment for a vehicle replacement windscreen having a mounted driver assistance camera (DAC); wherein an illuminating radiation delivery device is used and the spectral reflection profile from the DAC is imaged and compared to a datum spectral reflection profile (SAP).

In a preferred realisation, the datum spectral reflection profile results from the or a DAC 5 fitted to the windscreen before replacement.

In a preferred realisation: a) a first spectral reflection profile is imaged with DAC in position on a damaged windscreen fitted to the vehicle; and, b) a second spectral reflection profile is imaged with DAC in position on the replacement windscreen fitted to the vehicle.

It is preferred that a processor operates to compare image data relating to the datum spectral reflection profile with image data relating to the spectral reflection profile of the 15 DAC fitted to the replacement windscreen The illuminating radiation is preferably delivered from a device spaced from the windscreen; and an imaging device (such as a camera) is used to capture the spectral reflection pattern.

The imaging device and the illumination delivery device may be positioned in controlled/fixed relationship to one another and to the vehicle such that accurately repeatable images can be taken with the damaged and replacement windscreens fitted to the vehicle The image data is preferably captured using an imaging device which is wavelength tuned to a wavelength (or wavelength band) of the illuminating radiation The imaged spectral reflection data is processed to identify one or more (or a combination 30 of) roll, pitch and yaw components.

In an embodiment, the spectral reflection profile from the lens of the DAC is imaged and compared to a datum spectral reflection profile (SRP).

According to another aspect, the invention provides a method for calibration of a DAC of a 5 vehicle diver assistance system comprising: i) a first step comprising investigating the accuracy of fitment for a vehicle replacement windscreen having a mounted driver assistance camera (DAC); wherein an illuminating radiation delivery device is used and the spectral reflection profile from the 10 DAC is imaged and compared to a datum spectral reflection profile (SRP); ii) a subsequent step comprising calibration of the DAC.

According to another aspect, the invention provides a system for investigating the accuracy of fitment for a vehicle replacement windscreen having a mounted driver assistance camera (DAC), wherein an illuminating radiation delivery device (such as a light source) is used to direct light to impinge on the lens of the DAC and the spectral reflection profile from the DAC is imaged using an imaging device (such as a camera) and compared (for example using a processor) to a datum spectral reflection profile (SRP).

The system preferably further includes means for assuring the spatial consistency of the illuminating radiation delivery device, the windscreen, and the imaging device for imaging the DAC on the windscreen before replacement and for imaging the DAC on the windscreen following replacement.

Preferably, the system further comprises means for calibration of the DAC.

Preferred and optional features with respect to the first aspect of the invention are also envisaged as beneficial with respect to the second aspect of the invention as defined.

The invention will now be further described, by way of example only with reference to the accompanying drawings.

Figure 1 shows an ADAS driver assistance camera with a spectral reflection profile; Figure 2 shows a system set-up for use in accordance with theinvention; Figure 3 is a schematic representation of the system set-up of figure 2; Figure 4 is a plot showing specular line width and height, Figure 5 is a plot showing specular line angle from vertical with pitch, Figure 6 is a plot showing specular line angle from vertical with yaw, Figure 7 is a plot showing absolute specular line angle from vertical with yaw; Figure 8 is a graphical representation of theta angle of specularity line from vertical; Figure 9A shows the specular reflection from the lens of a camera at a first yaw position, Figure 9B shows the specular reflection from the lens of a camera at a second yaw 20 position; Figure 10 presents the raw images captured at each increment of pitch and yaw. The progression of images show how the angle of the lens reflection increases/decreases with a change in pitch and yaw, Figure 11 shows the image from sample where the extreme specularity dots are highlighted with bounding boxes which have been identified using a machine vision thresholding process; Figures 12 and 13 are tables relevant to the specularity measurement.

In the general and evolving area of this technology, a number of ADAS camera 5 variants exist, but all consistently have one (sometimes two for stereo camera systems and even three for tri-focal cameras) camera lens 6 is visible from the front of the housing camera 5 housing mounted to the windscreen. The camera 5 is housed within a bracket 4 which is connected directly to the windscreen 12, see Figures 1 and 3. This allows for the camera to be removed from the screen, and installed in a new windscreen bracket 4, or alternatively a new bracket 4 incorporating a camera 5 to be secured as a replacement to the windscreen 12. The exterior of the camera from the front of the vehicle is typically black and plain. From the front of the vehicle, much of the ADAS bracket and camera structure is covered 10 by window trim and tint.

The system and apparatus for investigating the accuracy of fitment for a vehicle replacement windscreen having a mounted driver assistance camera (DAC) is shown in figures 2 and 3.

The system operates by first being set up to image the specular reflection from the lens 6 of a DAC 5 with the windscreen to be replaced in position mounted in the vehicle before replacement. With reference to figure 1 and 2, the system comprises an illuminating light source 2 positioned in a fixed location with respect to the windscreen 12 and DAC 5, An imaging device (camera 3) is also positioned in a fixed orientation with respect to the light source 2 and the windscreen 12 and DAC 5. The specular reflection from the lens 6 of the DAC 5 is then imaged and recorded electronically. When fitted to the vehicle, replacement windscreen, is then viewed with the same, replicated system set-up and the specular reflection from the DAC 5 mounted to the replacement windscreen 12 is then imaged and recorded electronically. The spectral images are compared by a processor 11, and if they are in concurrence to within a predetermined degree of tolerance or acceptability, then the system outputs that the replacement fitting of the DAC.5 mounted to the replacement windscreen 12 is acceptable. The DAC.5 may then undergo a recalibration process in accordance with known practices and techniques. In the alternative if the comparison of the before and after fitment spectral profiles in not within the accepted tolerance then a negative determination may result in the procedure being restarted. The technique gives a first indication concerning acceptable fitment of the windscreen with the mounted DAC 5 before proceeding to the standard calibration determination of the DAC 5 The apparatus and system has a source of illuminating radiation (e.g. light) 2 which is 5 disposed to ensure illuminating radiation is reflected from the lens 6 of the DAC 5. An imaging device (such as a camera 3) is positioned to image the specular reflection from the lens 6 wherein an illuminating radiation delivery device is used and the spectral reflection profile from the DAC 5. Before replacement of the windscreen the spectral reflection profile from the lens 6 of the camera 5 is imaged in order to provide a datum spectral 10 reflection profile (SRP).

The illuminating radiation from the source 2 can be in the visible spectrum although wavelengths in the non-visible portion of the spectrum may be used.

Such illumination produces a noticeable spectral reflection of light source 2 in the lens 6. Not only is it noticeable that when the light source 2 changes position, but also when the lens 6 moves in relation to the static light source 2. The specular reflection is similar to the concept of lens flare. This phenomenon in the DAC is shown in Figure 1. The reflection takes the form of several bright dots 7, which from a distance, look to follow a straight line. With closer inspection however, this line can arc or deform in certain orientations between DAC, capture camera 3 and light source 2.

In Figures 9A and 9B the DAC camera is shown at two different yaw positions. Inside the lens the reflection can be seen rotating around the centre clockwise from figure 9A to figure 9B. The amount of rotation is relational to the yaw rotation (reference 14 in figure 2). Additionally, pitch also has an influence the formation of the reflection, either with the length of the reflection line, or rotation angle. The pitch is identified by reference 15 in figure 2. This response determines that pitch and yaw can be inferred by the formation of the reflection in the lens, such that, after installation of the replacement windscreen carrying the DAC 5, any inaccuracy in the new position of the DAC 5 can be determined by a deviation in pitch and yaw.

The difference between the spectral reflection images of figures 9A and 9B demonstrates how the reflection moves as the DAC 5 moves through pitch and yaw. The position of the light source 2 also has an impact on the spectral reflection profile. With the light source 2 close to the imaging camera 3 reference frame, the reflection is more distinct, and the dots are more in line. When the light source moves away from the reference frame, the reflection is less distinct, and the dots are less uniformly arranged.

According to the invention, the reflection of a light source 2 can be seen (and imaged by a camera 3) in the lens 6 of the DAC 5 of the ADAS system, and the form of the reflection 10 will change in a measurable way in relation to the pitch and yaw of the DAC S. The following description explores the range of pitch and yaw and observes the measurable effects of the specular reflection. Figure 2 shows how the system is arranged. The light source 2 and external camera 3 are kept at a constant position with respect to the windscreen 12 when the damaged windscreen is removed, and the replacement windscreen fitted. To test the relationship between the specular reflection that would occur from a misalignment of the DAC between the original windscreen and the fitment of the replacement windscreen, the DAC 5 of the ADAS system camera is rotated through a range of positions, -10 to +10 degrees. Both the DAC 5 and external (imaging) camera 3 are placed on the same surface and light source is close to the camera reference frame without being eclipsed by the camera. Exploration of the arrangement showed that to maximise the movement of the lens 6 reflection and achieve the most distinct dots (line 7 in figure 1), the light source 2 must be as close to the camera 3 reference as possible. For testing with respect to pitch and yaw the ADAS DAC 5 is mounted within an adjustable gimbal fixture that allows for controlled positioning through a range of pitch and yaw.

Using the adjustable fixture, the ADAS DAC 5 can be moved through a range of pitch and yaw angles, -10 degree to +10 degree, with 5-degree increments. At each position, an image is captured with the external camera. The lens reflection seen in each image is 30 measured, as shown in Figures 5 to 8.

Figurel0 presents the raw images captured at each increment of pitch and yaw. The progression of images show how the angle of the lens reflection increases/decreases with a change in pitch and yaw. It is worth noting that when the lens of the ADAS and lens of the external camera are aligned and facing each other (see Figure 8 T0:P-5:Y-10 to T0:P- 5:Y+10), the length of the reflection is shortest, and so more challenging to infer the angle. In the case of this experiment, that is most noticeable when the ADAS is at pitch -.5o It is also worth noting, that when the ADAS camera goes beyond the alignment with the external camera, the direction of rotation flips, and goes in the opposite direction. This can be seen in Figure 10 where pitch -10 the rotation is above the centre, pitch -5 is ambiguous, 10 and 0 to 10 is below the centre.

With reference to figure 1 for explanation, in determining the angle of the specularity, the extreme dots at either end of the line 7 are captured, then their relative distances along x and y axis are measured. Pythagoras theorem can then be used to determine the angle theta based upon the opposite and adjacent dimensions. However, both these dimensions are indicative of a decoupling between pitch and yaw and their influence on the final angle theta. Figure 4 presents a scatter plot of each combination of the opposite and adjacent (x 8z y) for each sample taken, where the specularity line projected back on to the plot. This data clearly shows a relationship where the opposite dimension (y) increases with pitch, and adjacent dimensions (x) increased with yaw. Whilst these relationships are not linear through their range, they are characterizable through calibration of the system, and enable decoupling of pitch from yaw by comparing the x & y measurements.

These results show that there is a relationship between the measured angle of the lens reflection and both pitch and yaw of the lens. Furthermore, the relationship can be defined mathematically. The results have shown that there are regions within the range of sensing which are less reliable, especially when both cameras are directly aligned. This determines a 'sweet spot' of alignment to avoid less sensitive regions, or regions where reflection is too poor to interpret. Exploration of the experiment arrangement also showed that light position has an effect on lens reflection range. A light source close to the DAC camera 5 reference frame produces greater range in the lens reflection, however it is not possible to have the light source directly inline without the light being blocked by the external camera.

Determining accuracy with the configuration described is only indicative of more sophisticated camera equipment, but assumptions can be made to help arrive at an accuracy figure. Figure 11 shows the image from sample TO:PO:YO, where the extreme dots (A & B) are highlighted with bounding boxes which have been identified using a machine vision thresholding process. It is acknowledged that the accuracy of these bounding boxes is subjected to image noise and finely tuned thresholding parameters. With that in mind, this discussion will assume that a bounding box will have a +/-5% accuracy. Each box has been annotated with relative pixel dimensions, alongside the relative x and y distance as measured previously as part of the data analysis.

Fig 12 presents the propagation of the assumed 5% machine vision accuracy to the bounding box measurements. Fig 13 calculates the maximum and minimum measurements for both dots, A & B. Using these limits, it presents the maximum and minimum possible theta angles with a +/-0.87 degrees. If we relate this to the earlier discussion, and the trend line for pitch 5 degrees, it has an approximate range of +/-40 degrees range of specularity rotation for a physical yaw range of +/-10 degrees. The approximate relationship between specularity rotation and yaw, disregarding linearity is 4 degrees to 1 degree (specularity to yaw) with an accuracy of +/-0.87 degrees specularity rotation, which propagates to +/-0.22 degrees real yaw rotation.

Whilst this calculation is based upon assumptions, it does explore the effect of the vision equipment accuracy and provides opportunity to explore what measures can be taken to improve this figure. For example, a pixel measurement is proportional to the image sensor size used in the external camera. More pixels used in the captured image will provide a higher resolution for pixel measurements. Using a suitable lens will allow the image to zoom to and focus on the ADAS lens specifically. The sample images were taken with a 5568x3712 image sensor, but the ADAS lens occupies a 250x250 region. If a correct lens were employed to zoom to the limits of the sensor, the ADAS lens could occupy 3500x3500 with some space left to allow for mechanical alignment of the cameras. This would be an improvement of x14 resolution. Optimistically the machine vision dot detection accuracy could be improved by x14, providing a +/-0.36% tolerance. This would propagate through to a theta angle measurement accuracy of +/-0.063 degrees and real yaw accuracy of +/-0.016 degrees. These assumptions are based upon ideal conditions of camera equipment, however, demonstrate a means to achieve improved accuracy to a high tolerance.

These results suggest that ADAS camera orientation is clearly measurable using this specular reflection technique. It is worth noting that a windscreen offers additional geometric markers which can be used in addition to or in isolation to make inferences on positional change. For example, the aperture of the window tint shifts with a slight change in ADAS camera 5 pitch. Detection of the camera 5 lens circularity is a trivial machine vision task and offers a reference for x and y translation. Using such methods of following additional markers in combination with the reflection measure both X and Y linear translation and yaw and pitch rotation can be monitored.

The invention provides that the orientation and translation of a camera 5 in a newly fitted windscreen can be determined with machine vision equipment. In order to achieve validation, measurement of the pitch and yaw of the camera S are beneficially required alongside a measurement of x and y translation. Translation of x and y within a static camera 3 view is a process of matching the ADAS camera 5 profile in two images and measuring the distance between the two instances. Observations and measurements have been made which investigate a relationship between the form of specularities seen in the ADAS camera 5 lens 6 when there is a presence of an external light source 2, and the pitch and yaw of the ADAS camera 5 housing 4. This has been explored and rationalised with a demonstration of the specularities seen in the lens 6 of an ADAS camera 5 housed within a windscreen. It has been demonstrated that these specularities provide a consistence relationship with pitch and yaw which can be decoupled, that a preferred system depends on that to determine the orientation of a newly fitted ADAS camera system.

It should be noted that the above-mentioned embodiments illustrate rather than limit the 30 invention, and that those skilled in the art will be capable of designing many alternative embodiments without departing from the scope of the invention as defined by the appended claims. In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The word "comprising" and "comprises", and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. In the present specification, "comprises" means "includes or consists of' and -comprising" means -including or consisting of'. The singular reference of an element does not exclude the plural reference of such elements and vice-versa. The mere fact that certain measures arc recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS 2. 4 5
A method for investigating the accuracy of fitment for a vehicle replacement windscreen having a mounted driver assistance camera (DAC); wherein an illuminating radiation delivery device is used and the spectral reflection profile from the DAC is imaged and compared to a datum spectral reflection profile (SRP) A method according to claim 1, wherein the datum spectral reflection profile results from the or a DAC fitted to the windscreen before replacement.
A method according to claim 1 or claim 2, wherein: a) a first spectral reflection profile is imaged with DAC in position on a damaged windscreen fitted to the vehicle; b) a second spectral reflection profile is imaged with DAC in position on the replacement windscreen fitted to the vehicle.
A method according to any preceding claim wherein a processor operates to compare image data relating to the datum spectral reflection profile with image data relating to the spectral reflection profile of the DAC fitted to the replacement windscreen.
A method according to any preceding claim, wherein, the illuminating radiation is delivered from a device spaced from the windscreen; and an imaging device (such as a camera) is used to capture the spectral reflection pattern.
A method according to claim 5, wherein the imaging device and the illumination delivery device are positioned in controlled/fixed relationship to one another and to the vehicle such that accurately repeatable images can be taken with the damaged and replacement windscreens fitted to the vehicle.
7 A method according to any preceding claim, wherein the image data is captured using an imaging device which is wavelength tuned to a wavelength (or wavelength band) of the illuminating radiation
8 A method according to any preceding claim, wherein the imaged spectral reflection data is processed to identify roll (pitch) and yaw components.
9 A method according to any preceding claim, wherein the spectral reflection profile from the lens of the DAC is imaged and compared to a datum spectral reflection profile (SRP).
10. A method for calibration of a DAC of a vehicle diver assistance system comprising: i) a first step comprising investigating the accuracy of fitment for a vehicle replacement windscreen having a mounted driver assistance camera (DAC); wherein an illuminating radiation delivery device is used and the spectral reflection profile from the DAC is imaged and compared to a datum spectral reflection profile (SRP); a subsequent step comprising calibration of the DAC.
11 A system for investigating the accuracy of fitment for a vehicle replacement windscreen having a mounted driver assistance camera (DAC); wherein an illuminating radiation delivery device (such as a light source) is used to direct light to impinge on the lens of the DAC and the spectral reflection profile from the DAC is imaged using an imaging device (such as a camera) and compared (for example using a processor) to a datum spectral reflection profile (SRP).
12 A system according to claim 11, wherein the system further includes means for assuring the spatial consistency of the illuminating radiation delivery device, the windscreen, and the imaging device for imaging the DAC on the windscreen before replacement and for imaging the DAC on the windscreen following replacement.
13. A system according to any of claims 11 or claim 12, wherein the system further comprises means for calibration of the DAC.