WO2020260854A1

WO2020260854A1 - Analysis of damage to vehicle glazing panels

Info

Publication number: WO2020260854A1
Application number: PCT/GB2020/051240
Authority: WO
Inventors: Devender VERMA
Original assignee: Belron International Limited
Priority date: 2019-06-24
Filing date: 2020-05-21
Publication date: 2020-12-30
Also published as: GB201908974D0; GB2585004A

Abstract

A computer implemented method of vehicle glazing panel damage assessment utilises a portable device having a camera to take an image/photograph of the windscreen to assess one or more of the location of the vehicle damage with respect to the perimeter of the windscreen; the size and /or nature of the damage to the windscreen; or the correct/appropriate mounting and device elements a replacement windscreen may require, in order to mount one or more electronic devices that may be fitted to the assessed windscreen. This enables an operative in the field, or a vehicle driver, to be able to make an assessment in the field, of whether windscreen damage (such as a break or crack) can be repaired or whether a replacement windscreen needs to be fitted instead.

Description

Analvsis of damage to vehicle glazing panels

The present invention relates to analysis of damage to vehicle glazing panels such as vehicle windscreens.

WO2017/194950 describes techniques for identifying the size of damage (such as a crack or break) present in a vehicle windscreen.

The present invention seeks to provide additional and enhanced techniques for analysing and assessing damage to vehicle glazing panels and making a determination of whether a vehicle windscreen can be repaired or should instead be replaced.

Vehicle windscreens play an important part in establishing the overall structural integrity of a vehicle, which is important for example if the vehicle is involved in a collision. Significant damage to a windscreen, if left unrepaired can impair the overall structural integrity of the vehicle and put occupants at risk. Furthermore, if cracks or breaks are too significant, then even if repaired the vehicle integrity will be compromised and in such circumstances, it is necessary to replace rather than repair the windscreen.

It is important to be able to make an assessment in the field, of whether windscreen damage (such as a break or crack) can be repaired or whether a replacement windscreen needs to be fitted instead. Historically this has been done by human judgement and experience. The present invention seeks to use an AI based system to make such assessment, in order to substitute for human judgement and experience.

Furthermore, certain governments, regulations and standards require replacement rather than repair if the windscreen damage is over a certain size or present at a certain location of the windscreen. The present invention seeks to build into an AI based assessment and decision-making system, information in making the assessment based on such las regulations and standards that may be in force or operating. Also, in circumstances where it is determined that a replacement rather than repair is needed, it may be that the replacement windscreen will need to be provided with brackets objects or devices already fitted to the windscreen. For example, vehicles may utilise advanced driver assistance systems (ADAS) which have components mounted to the windscreen. Forward facing cameras for ADAS systems are often windscreen mounted and provided with mounting brackets secured to the windscreen. When ordering a replacement windscreen, it is important to know whether and which mounting brackets, objects and devices need to be provided on the screen prior to dispatch.

It is also an object of the invention to provide a system which enables assessment to be made prior to a technician attendance at the vehicle, in order to streamline repair and/or replacement process.

An improved technique has now been devised.

According to the present invention, there is provided, a computer implemented method of vehicle glazing panel damage assessment, the method comprising:

using a portable device having a camera to take an image/photograph of the windscreen to assess one or more (or all) of:

i) location of the vehicle damage with respect to the perimeter of the windscreen; ii) the size and /or nature of the damage to the windscreen;

iii) the correct/appropriate mounting and device elements a replacement windscreen may require, in order to mount one or more electronic devices that may be fitted to the assessed windscreen.

It is preferred that the photograph/image is transmitted to a remote processor wherein the processor uses an AI or machine learning system or method to make the assessment in i) ii) or iii).

In certain embodiments, it is preferred that the size and /or nature of the damage to the windscreen is assessed by using a scaling object placed in the vicinity of the windscreen damage, which scaling object is captured in the image/photograph. A scaling object can also be used in a photograph/image taken of substantially the whole screen in order to assess the size and shape of the entire screen. For example, a credit card or drivers licence placed in the middle of the screen can serve this purpose since they are generally of consistent shape and dimensions.

Conveniently, coin may be used as the scaling object which is placed (on the windscreen) near the windscreen damage. The processor uses an AI or machine learning system or method to identify the coin (and thereby it’s size), for example identifying the

denomination of the coin.

In alternative an alternative scaling object may be used. For example, a credit card or drivers’ licence could be used. Alternatively, an object fixed to the vehicle in the vicinity of the windscreen could be used, for example a windscreen wiper present in an image/photograph could be used as the scaling object.

It is preferred that he AI makes an assessment, taking into account the geographical location or country of registration of the vehicle, in relation to one or more of:

i) the location of the vehicle damage with respect to the perimeter of the windscreen;

ii) the size and /or nature of the damage to the windscreen;

iii) the correct/appropriate mounting and device elements a replacement windscreen may require, in order to mount one or more electronic devices that may be fitted to the windscreen.

This is because certain governments, regulations and standards require replacement rather than repair if the windscreen damage is over a certain size or present at a certain location of the windscreen. The present invention seeks to build into an AI based assessment and decision-making system, information in making the assessment based on such las regulations and standards that may be in force or operating. By taking the geographical location of the vehicle into account in making the AI based assessment of the damage, the assessment can ensure that any laws, regulations or standards operating in the state in which the vehicle is present are complied with. This can be taken a step further by making the assessment take into account the country or state of registration of the vehicle.

In a preferred realisation of the computer implemented method, it is preferred that a plurality of photographs/images are used, a first image/photograph capturing substantially the entire windscreen and a second being a close-up image or photograph of the damage and including a scaling object positioned in the vicinity of the damage.

The‘whole screen’ photograph or image enables the AI based processing of that image to be used to identify the correct windscreen to be identified for replacement purposes. The ‘whole screen’ photograph or image also enables the AI based processing of that image to be used to identify the location on the screen of the damage and identify whether the damage area is in any prohibited repair regions of the windscreen. That is any location of the screen that applicable legislation or regulation may deem prohibited for repair being conducted.

The second,‘close up’ image or photograph captures the break/damage and the scaling object (e.g. the coin).

The plurality of images are preferably uploaded to a remote processor where the AI based assessment is conducted.

It is preferred that the image(s)/photograph(s) are electronically assessed using the AI to assess all of

iii) the correct/appropriate mounting and device elements a replacement windscreen may require, in order to mount one or more electronic devices that may be fitted to the assessed windscreen;

and,

as a result of the computer assessment a decision is made relating to whether the damage can be repaired, or a replacement windscreen must be fitted. If the assessment is made that the windscreen must be replaced an order may be created or processed for the required windscreen.

It is preferred that the AI based assessment for creating/processing the order includes an assessment of any mountings or devices that need to be present on the replacement windscreen.

It is preferred that the vehicle occupant uploads the photograph(s)/image(s) to a remote server from their own personal device.

It is preferred that the interaction of the vehicle occupant to upload the

photograph(s)/image(s) to the remote server is via the internet or equivalent.

The invention provides a globally applicable solution taking into account various international legislation in the field. The invention improves accuracy and reliability of assessment resulting in fewer unsafe repairs being conducted therefore improving vehicle safety. The invention provides increased automation of the repair process and enables customers to provide the information to make the assessment, without the attendance of a skilled technician for that part of the process.

The invention will now be further described, by way of example only with reference to figure 1 which is a representation of a close up image/photograph of a windscreen break using a coin as a scaling object to enable the AI based system to identify the coin based on AI and machine learning techniques and once identified enabling the size of the break to be accurately estimated knowing the dimensional parameters of the coin used for scaling.

Figure 2 is a representation of a‘whole screen image.

Figure 3 is a representation of an interactive image rendered during the method of the invention and enabling the user to click on the image to indicate the position of the break. The user accesses the web page via the internet using their own smartphone, computer or other device. The user is prompted to upload two photos. The first prompt is to upload a photo of the entire windscreen. The purpose of this is to determine what windscreen is already fitted and determine the correct windscreen for a replacement if needed. The photo can also be used to identify the location of the damage and determine if it positioned in a location of the screen that any applicable legislation deems as not allowed for repairs to be conducted. As an option the location can also be identified by using an interactive image on the website and requesting the user to click at the location of the damage.

Next the user is requested to upload a‘close-up’ photograph of the damage to the windscreen. Before doing this the user is requested to place a scaling object on the windscreen, in the vicinity of the damage. Typically, this will be a coin. This is chosen because the AI system has been trained to recognise different coins and the processing system, once the coin has been identified can use the recorded dimensions of the coin to estimate the size of the damage (the beak or crack in the windscreen).

When the relevant photographs have been uploaded, the processor uses the images and the AI module to make an assessment of various features as will be explained below, in order to make an assessment of whether the windscreen damage is susceptible for repair, or alternatively that windscreen replacement should be the required option.

If vehicle replacement is the preferred option, the process can also order the required windscreen together with any fitments, mounting brackets or devices that may be required to match those fitted to the existing windscreen.

The system can notify the user submitting the photographs of the results of the assessment, request confirmation of the appropriate repair or replacement outcome and if the user requests, arrange to send an appropriate technician to repair or replace the windscreen.

If the user requests the work to be done, the company can schedule the appropriate technician to attend. In the event of a replacement windscreen being required the system can also organise the delivery of the correct replacement windscreen, where necessary fitted with the relevant fitments, mounting brackets or devices corresponding to those mounted to the windscreen that is being replaced.

The AI module behind the processing step has been trained using known machine learning techniques to:

1. Estimate the overall dimensions of the windscreen from the‘whole windscreen’ photograph and from that identify the relevant windscreen for replacement purposes.

2. Recognise the object (e.g. the coin) used for scaling purposes and from that

estimate the distance of the coin from the damage and also the overall size of the break or crack forming the damage. In refined techniques the AI module can be used to enable more detailed investigation of the damage. This information is envisaged as being derived from the‘close-up’ photograph but in time it is envisaged that a single photograph/image will be sufficient to enable assessment for the overall screen dimensions and screen selection and also the size and nature of the break that would otherwise be derived from the‘close-up’ photograph.

3. Consequent on 2 above estimate the position (for example in x, y coordinates) of the damage on the windscreen and the processor is able to ascertain whether the position is in any locations that are prohibited from repair being carried out.

Prohibited locations can arise from, for example, any applicable legislation, regulations, best practice information or specific preferences of the

repair/replacement company.

4. Identify any fitments, brackets or devices attached to the windscreen being

photographed. For example, any ADAS cameras or LIDAR devices and their mounting brackets can be identified from the photographs and if a replacement windscreen is required it can be ensured that the correct fitments, brackets or devices (where necessary) are fitted to the replacement windscreen before dispatch to the replacement technician. AI and machine learning techniques known in the art may be used to train the AI to recognise windscreens, scaling objects (e.g. coins) and mounting brackets and devices mounted in the windscreen.

Machine learning algorithms build a mathematical model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to perform the task. Machine learning algorithms are widely used in many applications, such as computer vision, Machine learning, deep learning and data mining techniques are known which enable data analysis and refinement of predictions through unsupervised or autonomous learning.

As an example, the following technique may be used. The machine learning algorithms utilise object detection to identity the location and size of objects such as coins, damage or sensors. In particular, models using Regional Convolutional Neural Networks (Faster R- CNN) and Single Shot Multi-Box Detectors (Inception SSD) can be trained which enable identification of a) the boundaries of an object within an image, b) type of object (i.e. coin, damage, etc) and c) provides a confidence score of the algorithms' predictions.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be capable of designing many alternative embodiments without departing from the scope of the invention as defined by the appended claims. In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The word "comprising" and "comprises", and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. In the present specification,“comprises” means“includes or consists of’ and“comprising” means“including or consisting of’. The singular reference of an element does not exclude the plural reference of such elements and vice-versa. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A computer implemented method of vehicle glazing panel damage assessment, the method comprising:

i) location of the vehicle damage with respect to the perimeter of the

windscreen;

ii) the size and /or nature of the damage to the windscreen;

iii) the correct/appropriate mounting and device elements a replacement

windscreen may require, in order to mount one or more electronic devices that may be fitted to the assessed windscreen.

2. A method according to claim 1, wherein the photograph/image is transmitted to a remote processor wherein the processor uses an AI or machine learning system or module, or method to make the assessment in i) ii) or iii).

3. A method according to claim 1 or claim 2, wherein the size and /or nature of the damage to the windscreen is assessed by using a scaling object placed in the vicinity of the windscreen damage, which scaling object is captured in the image/photograph .

4. A method according to claim 3, wherein a coin is used as the scaling object which is placed in the vicinity of the windscreen damage.

5. A method according to claim 4, wherein the wherein the processor uses an AI or machine learning system or method to identify the coin, for example identifying the denomination of the coin.

6. A method according to any preceding claim, wherein the AI makes an assessment in i) ii) or iii) taking into account the geographical location of the vehicle.

7. A method according to claim 6, wherein the assessment relates to regulatory or legal standards applying at the location of the vehicle.

8. A method according to any preceding claim, wherein a plurality of

photographs/images are used, a first image/photograph capturing substantially the entire windscreen and a second being a close-up image or photograph of the damage and including a scaling object placed on the windscreen in the vicinity of the damage.

9. A method according to claim 8, wherein the plurality of images are uploaded to a remote processor where the AI based assessment is conducted.

10. A method according to any preceding claim, wherein the image(s)/photograph(s) are electronically assessed using the AI to assess all of

i) location of the vehicle damage with respect to the perimeter of the

windscreen;

ii) the size and /or nature of the damage to the windscreen;

and,

as a result of the computer assessment a decision is made relating to whether the damage can be repaired, or a replacement windscreen must be fitted.

11. A method according to claim 10, wherein if the assessment is made that the

windscreen must be replaced an order is created or processed for the required windscreen.

12. A method according to claim 11, wherein the AI based assessment for

creating/processing the order includes an assessment of any mountings or devices that need to be present on the replacement windscreen.

13. A method according to any preceding claim, wherein the vehicle occupant uploads the photograph(s)/image(s) to a remote server from their personal device.

14. A method according to claim 13, wherein the interaction of the vehicle occupant to upload the photograph(s)/image(s) to the remote server is via the internet or equivalent.