GB2579842A

GB2579842A - Vehicle Tire Leak Detection

Info

Publication number: GB2579842A
Application number: GB1820568.2A
Authority: GB
Inventors: Prabhakar Pavannag; Devrani Divya
Original assignee: Continental Automotive GmbH
Current assignee: Continental Automotive GmbH
Priority date: 2018-12-18
Filing date: 2018-12-18
Publication date: 2020-07-08
Anticipated expiration: 2038-12-18
Also published as: GB2579842B; GB201820568D0

Abstract

A method for detecting a leaking vehicle tire (e.g. from a puncture) by determining the vehicle is tilted. First and second image sensors on the vehicle respectively capture images of first and second reference features on opposite sides of the vehicle, for example indentations in the car body 372a, 372b or door handles (382a, 382b; fig. 3C). The reference points are located at substantially the same height on the vehicle body, meaning misalignment indicates a tire may be leaking. The images are aligned and compared to determine if the reference features are aligned. If misaligned, actions are taken, for example outputting a warning to the driver, modifying braking or steering control, or disabling an autonomous mode. Tire leaking may be verified by repeatedly comparing images of reference features. Tire pressures, height of the tires above the ground, height of a vehicle body part above the ground and/or a safe travel distance for the tires may also be calculated. The image sensors may be wide angle fish-eye cameras which are part of a 360 degree surround view system (see figures 2 and 3A), with the images transformed to a common perspective view for comparison (as in figures 3B, 3C).

Description

VEHICLE TIRE LEAK DETECTION

TECHNICAL FIELD

The present disclosure relates to methods and apparatus for 5 detecting if a tire of a vehicle is leaking.

BACKGROUND

There is a need for drivers to be notified of tire leakages and punctures so that they can have the defective tire(s) repaired or replaced before a tire blowout or burst occurs. The rapid loss of tire pressure associated with a tire burst can be dangerous for a driver as it may result in a loss of vehicular control and consequently accidents such as vehicles flipping over.

In view of the above, systems for detecting tire punctures have been developed. For example, a -known system uses the correlation between steering wheel angle, the signal values of runout rate sensors and transverse acceleration sensors to determine if any tires on a vehicle have burst. Runout rate sensors measure the angular deviation of a vehicle's direction of travel from a straight line while transverse acceleration sensors measure sideslip signals. However, such a system has its drawbacks. It is complex and may suffer from inaccuracies due to the nature of calculations involved.

There are also other systems which utilize changes in tire pressure as a means of detection. Air pressure inside pneumatic tires may be monitored either directly or indirectly using Tire pressure monitoring systems (TPMS). For Direct TPMS, tire pressure is physically measured using pressure sensors are mounted on each vehicle wheel. Indirect TPMS, on the other hand, determine tire pressure based on wheel rotational speed and other vehicle operating signals. However, some indirect TPMS systems are unable to identify the location of the tire experiencing a loss of pressure and as such inform a driver which tire should be repaired or replaced. Furthermore, tire pressure alone is not always a good indicator of tire punctures as other factors such as changes in ambient temperature may also cause a reduction in tire pressure.

In view of the foregoing, alternative methods and apparatus for detecting vehicle tire leakages and punctures are desirable.

SUMMARY

Aspects of this disclosure provide methods and apparatus for detecting if a tire of a vehicle is leaking.

A first aspect of this disclosure provides a method for detecting it a tire of a vehicle is leaking comprising receiving from a first image sensor located on the vehicle a first image comprising a first reference feature located externally on a first side of the vehicle and receiving from a second image sensor located on the vehicle, a second image comprising a second reference feature located externally on a second opposite side of the vehicle. By way of example, the first and second sides of the vehicle may correspond to the sides of the vehicle where the doors are located. The method further comprises aligning the first and second images to each other and determining if the images of the first and second reference features are aligned after aligning the first and second images to each other. The first and second reference features are located at substantially the same height on the vehicle body and misalignment between the images of the first and second reference features is indicative that at least one tire located on the first or second side of the vehicle may be leaking. The method also comprises the execution of one or more actions in response to the first reference feature image being misaligned with the second reference feature image.

In some implementations, the first and second images may be subjected to pre-processing before they are aligned to each other. The pre-processingmay comprise trans forming the first and second images to a common perspective view before aligning the first and second images to each other so that they maybe compared. Other pre-processing steps may be directed at removing any distortions in the images. For example, if the first and second image sensors are fisheye cameras the first and second image may be subjected to a geometric alignment and photometric alignment algorithm before the first and second images are aligned to each other. The geometric alignment algorithm may comprise correcting for fish-eye lens distortion in the first and second images while the geometric alignment algorithm reduces differences in colour and brightness. In some implementations, the one or more actions may comprise verifying if a tire located on the first or second side of the vehicle is leaking. In one variation, verifying if a tire is leaking may comprise receiving at least one additional image of a third reference feature from the first image sensor and at least one additional image of a fourth reference feature from the second image sensor, wherein the additional images are captured in a temporally later time compared to the first and second images reference feature image and determining if the third and fourth reference features in the additional images are aligned to each other. The first and second reference features may be re-used for third and fourth reference features respectively or different reference features maybe used. The steps of receiving the additional images and determining if the third and fourth reference features in the additional images are aligned may be repeated over a predetermined period of time.

In some implementations, the method may also further comprise the steps of determining at least one of: a tire pressure of at least one tire, a height of at least one tire or a height of a vehicle body part above a ground, and determining if the tire pressure of the at least one tire, the height of the at least one tire and/or the height of the vehicle body part above the ground are within predefined limit or limits. The height of at least one tire or a height of a vehicle body part above a ground may be determined at least in part using an image of the at least one tire and/or the vehicle body part captured by the first or second image sensor. The image of the at least one tire and/or the vehicle body part may, for example, be captured by causing the first or second image sensor to zoom in to the at least one tire and/or the vehicle body part. In another optional implementation of the disclosure, the step of determining if the images of the first and second reference features are aligned is implemented when one or more of the tire pressure of the at least one tire, the height of the at least one tire or the height of a vehicle body part are not within predefined limit or limits. Additionally or alterna-tively, the step of determining if the images of the first and second reference features are aligned may also be configured to be a recurring process. In one optional implementation, the one or more actions executed in response to the first reference feature image being misaligned with the second reference feature image may comprise modifying a braking and/or steering control mechanism. of the vehicle, disabling an autonomous or semi-autonomous mode of operating the vehicle, issuing a warning to a driver of the vehicle in response to the further verification indicating that a tire is leading or has burst, or a combination thereof. The one or more actions may also comprise calculating a safety travel distance for the vehicle. One aspect of this disclosure provides for a non-transitory computer-readable storage medium comprising computer-readable instructions for carrying out one or more of the above discussed methods.

Another aspect of the disclosure provides an apparatus for detecting a tire leak of a vehicle comprising a processor and at least one memory coupled to the processor and storing in-structions executable by the processor causing the processor to receive from a first image sensor located on the vehicle a first image comprising a first reference feature located externally on a first side of the vehicle and receive from a second image sensor located on the vehicle, a second image comprising a second reference feature located externally on a second opposite side of the vehicle. The processor is further cause to align the first and second images to each other and determine if the images of the first and second reference features are aligned after the first and second images are aligned to each other. The first and second reference features are located at substantially the same height on the vehicle body and misalignment between the images of the first and second reference features is indicative that a tire located on the first or second side of the vehicle may be leaking. The processor causes the execution of one or more actions in response to the first reference feature image being misaligned with the second reference feature image.

In some implementations, the one or more actions caused to be executed comprises verifying if a tire located on the first or second side of the vehicle is leaking. Verifying if a tire is leaking may comprise causing the processor to receive at least one additional image of a third reference feature from the first image sensor and at least one additional image of a fourth reference feature from the second image sensor, wherein the additional images are captured in a temporally later time compared to the first and second images reference feature image. The processor then determines if the third and fourth reference features in the additional images are aligned to each other. In one variation, the processor is further caused to transform the first and second images to a common perspective view before aligning the first and second images to each other. In another optional variation, the processor may be further caused to determine at least one of: a tire pressure of at least one tire, a height of at least one tire or a height of a vehicle body part above a ground, and determine if the tire pressure of the at least one tire, the height of the at least one tire and/or the height of the vehicle body part above the ground are within predefined limit or limits.

The details of one or more implementations of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a functional block diagram of a system 100 comprising 20 a tire information module for detecting a tire puncture of a vehicle in accordance with one implementation of this disclosure; FIG. 2 is a simplified top view of a car mounted with surround view camera system in accordance with one implementation of this 25 disclosure; FIG. 3A shows a car equipped with a surround view camera system and exemplary images of captured by side cameras in the system; FIG. 3B shows exemplary images of a first and second reference feature for monitoring whether there is a puncture in the rear tires of a car; FIG. 3C shows exemplary images of a third and fourth reference feature for monitoring whether there is a puncture in the front tires of a car; FIC. 4 illustrates exemplary images of left and right rear tires of a car which may be used to determine the height of the tires in accordance with one implementation of this disclosure; FIG. 5 is flow diagram illustrating an automated method for 10 detecting if a tire of a vehicle is leaking in accordance with one implementation of this disclosure; FIG. 6A is an exemplary top view image of a vehicle' s surroundings stitched together from images captured by a surround view system. 15 when there is no tire leakage; and FIC. 6B is an exemplary top view image of a vehicle' s surroundings stitched together from images captured by a surround view system when both tires on the right-hand side of the vehicle are leaking.

DETAIIJED DESCRIPTION

In the following detailed description, reference is made to the 25 accompanying figures. In the figures, similar symbols typically identify similar components, unless context dictates otherwise.

FIC. 1 is a functional block diagram of a system 100 associated with a vehicle comprising an environment module 110, a tire information module 130, a steering module 140, a braking module 150, an autonomous driving module 160 and a user interface 170 in accordance with one implementation of this disclosure. In the FIG. 1 implementation, the environment module 110 includes a surround view camera system 12C also known as around view camera system comprising four wide angle image sensors (122, 124, 126, 128) mounted externally around the vehicle and a processor 129 which may be operable to perform functions such as image processing and controlling the operation of the surround view camera system 120. It is to be appreciated that other types of image sensors may also be suitable and the environment module 110 may also comprise other sensor modules for monitoring an external environment of the vehicle such as radar, lidar and ultrasonic sensor modules. In some implementations, the four image sensors (122, 124, 126, 128) may be wide angle fisheye cameras mounted at the front bumper (122), rear bumper (124) and side mirrors (126,128) of a car like in the exemplary implementation shown in FIG. 2.

In FIG. 2, each of the cameras (122, 124, 126, 128) have a horizontal field of view that is more than 180 degrees such that images from the cameras may be stitched together to provide a driver of the car 210 with a 360-degree bird's-eye top view of the area around the vehicle. The horizontal field of view associated with the front, rear and side cameras are denoted by the reference numerals 122a, 124a, 126a and 128a respectively. A 360-degree top view image may be generated by a computing processor for the surround view system such as the one 129 shown in FIG. 1 by processing and stitching together images captured by the tour cameras (122, 124, 126, 128). In some implementations, a top view image may be generated by subjecting the input image frames from each camera to a geometric alignment and photometric alignment algorithm prior to stitching the four images together in a composite view synthesis step. The geometric alignment algorithm may comprise correcting for fish-eye lens distortion in the original incoming image frames and deriving for each camera, a perspective transformation matrix for converting lens distortion corrected images from their original perspective view to a common bird's-eye perspective. When deriving the perspective transformation matrices for each camera, matching features in overlapping regions of adjacent views may be located and the transformation matrix parameters chosen so as to minimise distances between matched features. Overlapping regions refer to portions of the image frames which come from the same region in the real world but are captured by two adjacent cameras. The perspective transformation matrices may be derived based on the extrinsic parameters of the cameras or the content of the image frames themselves. The lens distortion corrected images may be processed by a photometric alignment algorithm which corrects differences in colour and brightness between adjacent images before being stitched together in the composite view synthesis step. During the composite view synthesis step, the perspective transformation matrices derived during geometric alignment are used. The pixels lying in overlapping regions may either be blended or data from one of two images are used.

Referring back to FIG. 1, the system 100 further comprises a tire information module 130 in communication with the environment module 110. The tire information module 130 comprises a computing processor 132, a hardware memory 134 and a tire pressure management system. (TPMS) 136. The computing processor 132 maybe a microcontroller capable of accessing the memory 134 to store information and execute instructions stored therein. Alter-natively, the processor 132 and memory 134 may also be integrated on a single integrated circuit. The memory 134 stores information accessible by the processor 132 such as instructions executable by the processor 132 and data which may be stored, retrieved or otherwise used by the processor 132. For example, the processor 132 may execute a method for detecting a tire puncture on the vehicle based on images of at least a first and second reference feature captured by image sensors in the environment module 110 which are mounted on the vehicle. The first and second reference features being located externally on opposite sides of the vehicle. Although FIG. 1 functionally illustrates the processor 132 and memory 134 as being located within the same block, it will be appreciated by an ordinary person skilled in the art that the processor and memory may actually comprise multiple processors and/or memories located in different housing. Accordingly, references to a processor or memory will be understood to include references to a collection of processors and/or memories that operate to facilitate a method for detecting a tire puncture of a vehicle as described in this disclosure. Further, it will be appreciated by a person skilled in the art that the processor and memory need not exist as part of an independent tire information module but can also be a shared element or process of other modules, programs or hardware. For instance, the processor 132 and memory 134 exist as part of an Electronic Control Unit (ECU) that controls a steering or braking module.

By way off example, the tire information module 130 may use images from the first and second side cameras (126, 128) in the surround view camera system 120 for detecting whether there is any tire puncture on the vehicle. FIG. 3A shows exemplary images (326, 328) to-pen by a first and second fisheye camera (126, 128) mounted on the side mirrors of a car. In one implementation, tire punctures are detected by aligning images captured by the two side cameras (126, 128) and determining if images of a first and second reference feature located on opposite sides of the car and at substantially the same height on the car body are aligned. For example, in the FIG. 3A illustration, car handles (340a, 340b) for both front doors may be used as reference features for detecting front tire punctures while car handles (350a, 350b) for the rear doors may be used for detecting rear tire punctures. Other vehicle parts located on both the sides of the vehicle such as body side mouldings and running lights maybe used as reference features. Alternatively, body markers specifically designed to be used as reference features may also be added to a vehicle. The images 326, 328 taken by the side cameras (126, 128) also shows a side view of the 4 vehicle tires (360a, 360b, 362a, 362b). In some implementations, the height of the vehicle tires above the ground may be measured from the images shown in FIG. 3A or a magnified version thereof.

FIG. 33 is a further illustration of the FIG. 3A implementation where indentations in the car body are used as reference features. The close-up image 370 on the left-hand side of FIG. 33 is taken by the first side camera 126 and it comprises a first reference feature /indentation 372a located on the same side of the vehicle as the first side camera. The close-up image 374 on the right-hand side of FIG. 33 is taken by the second side camera 128 and it comprises a second reference feature /indentation 372b located on the same side of the vehicle as the second side camera (128). In the FIG. 33 example, the side camera zooms into the region of interest before taking the close-up images. It is also possible to magnify a region of interest instead for purposes of checking for alignment. In one variation of the FIG. 3A implementation, raw images from the side cameras (126, 128) are processed by the surround view processor 129 before they are sent to the tire information module 130. Such processing including transforming the images to a common perspective, removing fish-eye lens distortion from the raw images, removing any brightness and colour differences between the images or a combination thereof.

The processor 132 in the tire information module 130 then aligns the incoming images in the manner as shown in FIG. BE and determines if the first and second reference features (372a, 372b), that is, the indentations are aligned. Alignment of the incoming images may be based on pre-existing extrinsic parameter values of the respective cameras where the images are taken. Alternatively, a recalibration of some or all of the extrinsic parameters may be conducted, and the alignment based at least in part on the newly recalculated values. For instance, a re-calibration may be conducted only for the extrinsic parameters which define the rotational orientation of each camera. Alignment of images is then based on the newly recalibrated rotational related extrinsic parameters and pre-existing translational related extrinsic parameters. In the FIG. 3B image, the indentations (372a, 372b) are aligned which indicates that neither of the rear tires has a puncture with respect to the other. FIG. 3C which is another exemplary illustration of FIG. 3A, an edge on the oppositely situated front car handles is used as reference features. The picture 380 on the left-hand side of FIG. 3C is an image of the car handle 382a located on the same side of the car as the first side camera 126 while the picture on the right hand side 384 is an image of the car handle 382b located on the same side as the second camera 128. Since the edges of the two car handles are aligned, this is indicative that neither of the front tires has a puncture relative to the other. Although the cameras used for detecting tire puncture in the FIG. 3A example are mounted on the side mirrors, it will be appreciated by a person skilled in the art that cameras situated elsewhere on the vehicle may also be used for the same function so long as the reference features can be captured within their respective field of view Referring back to FIG. 1, the processor 132 in the tire information module 130 may trigger the process of checking for tire puncture and alignment between the reference features when a loss of pressure on one of the vehicle tires above a predetermined threshold is detected by the TPMS 136. In another variation, the tire information system may also be configured to run recurrent checks for misalignment and tire pressure readings from the TPMS 136 are used instead as a means of confirming if there is indeed a tire puncture as indicated by any misalignment of reference features. In yet another variation of this disclosure, the processor 132 may also detect or confirm the occurrence of tire punctures by measuring the height of a tire and/or clearance of a vehicle body part above the ground and checking if these measurements fall within predefined limits. For example, the clearance between the base of a vehicle carriage and the ground may be used as a means for detecting or confirming if there is a tire puncture. As discussed in relation to FIG. 3A, the height of a tire above the ground may be measured using images captured by the image sensors in the surround view system 120. The same may also be applied to determining the clearance of a vehicle body part above the ground. In some implementations, raw images captured by the surround view image sensors may be processed by the processor 129 in the surround view camera system (e.g. to remove fish-lens distortion) prior to being transmitted to the tire information module 130 for measurement of tire/vehicle part clearance. FIG. 4 shows exemplary images of rear wheel tires taken by respective side cameras mounted opposite sides of a car. As shown, these images may be used for measuring the heicht of the rear car tires above the ground. The image 420 on the left-hand side of FIG. 4 is an image of a first rear tire and it is captured by a first side camera located on a first side of the car. Based on the measurement from the ground to the top of the first rear tire, the first rear tire has a height of x above the ground. Image 440 is an image of a second rear tire captured by a second side camera located on the opposite side of the car. The second rear tire has a height of y based on the image of the second rear tire.

Referring back to FIG. 1, the system 100 also comprises a steering module 140 in communication with the tire information module 130.

The steering module 140 is responsible for measuring and controlling steering related functions and may include a steering sensor for measuring the steering angle at which the steering wheel is being turned and electronic stability program which controls, for example, the steering and suspension being applied to the vehicle. The tire information module 130 is also in communication with a braking module 150 such as an electronic braking system which may be responsible for the activation and control of all braking components in the vehicle such as the parking brake and service brakes, diagnostic and monitoring of the braking components. The braking module 150 may also include an anti-lock braking system (ABS) 152 which maintains tractive contact between the vehicle tires and a road surface by preventing the tires from locking up during braking. In some implemen-tations, the processor 132 in the tire information module 130 may provide information on the location of any tire puncture being detected to the braking module 140 and the ABS 152 may be configured to adjust its operation accordingly based on such information. In addition, the system 100 also comprises an autonomous driving module 160 and a user interface 170 in communication with the tire information module 130. The autonomous driving module 160 is responsible for executing semi-autonomous and autonomous driving functions such as adaptive cruise control (ACC), active lane assist, highly automated driving (HAD) and park assist. As for the user interface 170, it may be used for communicating audio and visual messages to a driver of the vehicle. The user interface 170 may comprise components such as an instrument panel, an electronic display and an audio system. The instrument panel may be a dashboard or a centre display which displays for example, a speedometer, tachometer and warning light indicators. The warning light indicators maybe light icons which alert a driver of the vehicle on the status of vehicle features such as the status of the tires including any tire punctures, electronic parking brake (EPB), and anti-lock brake system (ABS). The warning light indicators may display a yellow or orange colour to indicate that the associated vehicle feature needs to be serviced or replaced soon and a red light to indicate that the feature has a serious safety issue and the vehicle is unsafe to be driven. The user interface may also comprise an electronic display such as an infotainment or heads up display for communicating other visual messages to the driver and an audio system for playing audio messages, warning or music. It is to be appreciated that other vehicle systems not shown in FIG. 1 may also rely on tire puncture related information from the tire information module to control its operations and as such is in communication with the tire information module.

FIG. 5 is a flow chart illustrating an exemplary method 500 for detecting a tire of a vehicle is leaking in accordance with one implementation of this disclosure. The operations of method 500 will be described with reference to the system. in FIG. 1. However, it will be appreciated that other systems may also be suitable. The process starts at block 501 and may be initiated upon the ignition of the vehicle being switched on. In block 510, starting of the process in block 501 activates the TPMS 136 to start monitoring if there is a change in the pressure of any tires in the vehicle in block 510. If the TPMS 136 detects a loss of tire pressure for any of vehicle tires, the process goes on to block 520. Otherwise, the TPMS 136 continues to monitor the tire pressure for example either continuously or on a recurring basis. In some implementations, TPMS 136 may be configured to send a signal to the processor 132 in the tire information system. 130 when a loss of tire pressure is detected thereby triggering the processor to start the process for checking if any of the vehicle tires maybe punctured. Alternatively, the processor 132 may also query the TPMS 136 about any losses in tire pressure and trigger the detection process when it receives an affirmative answer. In some variations, the process of checking for punctured tires by the processor 132 may be activated only if the loss of tire pressure exceeds a predetermined amount in order to avoid unnecessary processing.

At block 520, in response to a loss of tire pressure being detected by the TPMS 136, the processor 132 receives from a first image sensor such as the surround view side camera 126 in FIG. 1, a first image comprising a first reference feature located externally on a first side of the vehicle. The processor also receives from a second image sensor (e.g. surround view side camera 128) a second image comprising a second reference feature located on the opposite second side of the vehicle. The first and second reference features being located at substantially the same height on the vehicle body and situated such that any misalignments between the first and second reference features is indicative that a tire located on one side of the vehicle may have punctured. The reasoning is that when a tire has a puncture, it reduces in height. Accordingly, a reference feature located on the same side of the vehicle as the punctured tire will be lowered and as such closer to the ground compared to a corresponding reference feature located on the opposite side of the vehicle where there is no tire puncture. Misalignments between the first and second reference features will be manifested more obviously, the closer the reference features are to the punctured tire. For instance, the body indentations (372a, 372b) in FIG. 3B are used as reference features for checking if there is a leakage on one the rear tires since they are located closer to the rear tires compared to the car front handles in FIG. 30. The car front door handles, on the other hand, are used for determining if one of the front tires is punctured. Other vehicle parts located externally on both sides of the vehicle such as body side mouldings and running lights may also be used as reference features. Alternatively, body markers specifically designed to be used as reference features may also be added to a vehicle. In some implementations, the processor may instruct the image sensors to specifically take a close-up view of reference features in order to check for misalignment.

In block 522, the processor 132 aligns the first and second images to each other. Examples of aligned first and second images each comprising a corresponding reference feature located on opposite sides of a car are shown in FIG. 3B and 3C. In some imple-mentations, images of the reference features taken by an image sensor may be processed prior to being aligned in block 522. For example, images from fisheye cameras may be corrected for fish-lens distortion and converted to a common perspective prior to alignment. These processing steps may be carried out by a processor in the image sensor module (e.g. surround view system. processor 129 in FIG. 1) or the processor 132 in the tire information module 130. After aligning the first and second images in block 522, the process continues to decision block 524 where the processor 132 determines if the first reference feature in the first image is aligned with the second reference feature in the second image. If the reference features are aligned, the process goes back to step 510 where the TPMS continues to monitor if there is still a loss of tire pressure.

On the other hand, if the first and second reference features are misaligned, the process goes onto block 530 where further steps are taken to confirm if there is indeed a tire puncture. In block 530, the processor 132 requests for additional images of the first and second reference features from the surround view camera module 120. These additional images being captured by the first and second image sensor (126,128) at a temporally later time compared to the first and second images discussed earlier. As with the first and second images, these additional images may be corrected for any image distortion and converted to a cormon perspective before the processor aligns the additional images and checks if the first and second reference features are aligned at block 532. If the reference features are aligned, the process goes back to block 510. However, if the reference features are not aligned, the process goes on to block 534 which determines if the checks for misalignment on additional images of reference features images has exceeded a predefined time frame. If the answer is no, the process goes back to block 530 where an additional pair of images containing the first and second reference images is retrieved and checked for alignment. If the answer is yes, the process goes on to block 540. Since misalignment between images of the first and second reference features may also be due to other causes such the vehicle travelling over potholes and other uneven terrain, verifying any detected misalignments over a period of time on additional images taken in a temporally later time helps to mitigate instances of tire puncture misdiagnosis. It is to be appreciated that the first and second reference features may be re-used for the additional misalignment checks in blocks 530 to 532, or other reference features associated with the same tire where a puncture is suspected may also be used. The additional checks may also be conducted for a predetermined number of additional images instead of over a predetermined period of time. It is also possible that only one additional image of the first reference feature from the first image sensor and one additional image of the second reference feature from the second image sensor is used to verify if there is any misalignment. In another variation not illustrated in FIG. 5, the processor 132 may also initiate other steps to rule out other potential causes for misalignment between the images of reference features such as changes in the extrinsic parameters of the first and second image sensors which maybe due to other causes. In the event that a drift in extrinsic parameters is detected, the affected image sensor(s) may be calibrated and images of reference features to be used for further alignment checks taken post-calibration.

Going back to decision block 534, if the checking of additional images has exceeded the predefined time frame, the process goes on to block 540 where the processor initiates a process for checking if the height of a tire where a puncture is suspected and/or the height of a vehicle part proximate to the suspected puncture tire above the ground is within a predefined limit or limits. As discussed earlier with reference to FIG. 4, the aforementioned heights may be measured from images of the relevant tire/vehicle part. If the height of the tire/vehicle part is within predefined limit (s) , the process goes back to block 510. However, if the height of the tire/vehicle part lies outside the predefined limit(s), the process goes on to block 550 where the processor causes the execution of one or more actions based on a tire puncture being detected. Byway of example, the actions being executed may include causing a tire replacement warning to be issued to the driver. Such warning could be visual and/or aural. In some implementations, the one or more actions may also include calculating a safety travel distance for the vehicle before tires must be changed and communicating this distance to the driver through a user interface such as the one 170 in FTC. 1. The safety travel distance may be computed based on one or more of the tire pressure, the amount of misalignment between the first and second reference features and deviations in the height of the tire/vehicle part from predefined limit(s). Other parameters tied to the operation of the vehicle such as the condition of the brakes, transmission and driver's driving style may also be considered when computing the safety travel distance. The safety travel distance is preferably constantly updated based on current values of parameters used for computation and communicated to the driver. The one or more actions being executed by the processor 132 in block 550 may also include transmitting the information on detected tire puncture or leakage including the location of the suspected puncture to a vehicle module which is at least partially responsible for the driving related operations of the vehicle. For instance, in the FIG. 1 implementation, the tire information module may transmit such information to the steering module 140, braking module 150 and/or autonomous driving module which it is in communication with. These respective modules may modify its braking/steering control operations and/or disable an autonomous or semi-autonomous mode of operating the vehicle based on the tire puncture related information being received. The processor 132 may also cause the transmission of other information which may assist the respective steering, braking and/or autonomous driving module (140-160) in making a decision on what changes to effect as a result of the tire puncture such as the amount of pressure loss associated with the tire, the amount of tire height deviation and the operating condition of other vehicular parts such as the brakes. It is to be appreciated that the FIG. 5 implementation is not intended to be limiting and variations may be made while still achieving a method for detecting if a tire of a vehicle is leaking. For instance, instead of relying on tire pressure variations, the height of each vehicle tire and/or clearance of vehicle body parts above ground may be monitored and deviations in dimensions used for triggering the actions starting from block 520. Similarly, tire pressure maybe used as a means of confirming any tire punctures instead of as a means for detection.

FIG. 6A and 6B are top view images which provide a 360-degree bird's eye view of a car's surroundings in two-dimensional perspective. The top view images are generated by stitching together images captured by surround view cameras mounted at the front, rear and sides of the car like in the FIG. 2 example. In some implementations, the too view images may be used for detecting situations where both front or both rear tires of a vehicle are leaking. In FIG. 6A the front wheels of the car 610 are not punctured. Therefore, image 620a captured by the front camera (e.g. 122 in FIG. 2) is alignedwith the images (622a, 624a) captured by the side cameras (126, 128). This is evident in the clear boundary lines 640, 650 between the front camera image (620) and those (622, 624) captured by the side cameras. The images of the same squares on the ground plane (642, 652) are also continuous and aligned between the front and side camera images. In contrast, when the front wheels of the car 610 are both leaking, the image 620b captured by the front camera (e.g. 122 in FIG. 2) is aligned with the images (622b, 624b) captured by the side cameras (126, 128). This is evident in the lack of a clear boundary between the front camera image and side camera images unlike the image in FIG. 6A. Furthermore, same squares on the ground plane (662, 682) are also not aligned between the front camera image (620b) and the side camera images (622b, 624b) . As discussed earlier, the extrinsic parameters of an image sensor or camera are used in the process of stitching together the images to form a top view. When there is a tire leakage, at least the z-translation extrinsic parameter which is related to the height of an image sensor above the ground plane is lowered as the tire height decreases. Due to the change in at least the z-translation extrinsic parameter, the images from the front and side cameras will be misaligned since the stitching process will then be using incorrect extrinsic parameters for alignment.

While various aspects and implementations have been disclosed herein, other aspects and implementations will be apparent to those skilled in the art. The various aspects and implementations disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims, along with the full scope of equivalents to which such claims are entitled. It is also to be understood that the terminology used herein is for the purpose of describing particular implementations only, and is not intended to be limiting.

Claims

CLAIMSWhat is claimed is: 1. A method for detecting if a tire of a vehicle is leaking comprising: receiving from a first image sensor located on the vehicle a first image comprising a first reference feature located externally on a first side of the vehicle; receiving from a second image sensor located on the vehicle, 10 a second image comprising a second reference feature located externally on a second opposite side of the vehicle; aligning the first and second images to each other; determining if the images of the first and second reference features are aligned after aligning the first and second images to each other, wherein the first and second reference features are located at substantially the same height on the vehicle body and misalignment between the images of the first and second reference features is indicative that a tire located on the first or second side of the vehicle may be leaking; and causing the execution of one or more actions in response to the first reference feature image being misaligned with the second reference feature image.
2. The method of claim. 1, wherein the one or more actions comprises further verifying if a tire located on the first or second side of the vehicle is leaking.
3. The method according to claim 2, wherein further verifying if a tire is leaking comprises: receiving at least one additional image of a third reference feature from the first image sensor and at least one additional image of a fourth reference feature from the second image sensor, wherein the additional images are captured in a temporally later time compared to the first and second images reference feature image; and determining if the third and fourth reference features in the additional images are aligned to each other.
4. The method of claim. 3, wherein the steps of receiving the additional images and determining if the third and fourth reference features in the additional images are aligned is repeated over a predetermined period of time.
5. The method according to any of the preceding claims further comprising: determining at least one of: a tire pressure of at least one tire, a height of at least one tire or a height of a vehicle body 15 part above a ground; and determining if the tire pressure of the at least one tire, the height of the at least one tire and/or the height of the vehicle body part above the ground are within predefined limit or limits.
6. The method of claim 5 wherein the height of at lease one tire and/or the height of the vehicle body part above the ground is determined at least in part using an image of the at least one tire and/or the vehicle body part captured by the first or second image sensor.
7. The method according to any of the preceding claims, wherein the step of determining if the images of the first and second reference features are aligned is implemented when one or more of the tire pressure of the at least one tire, the height of the at least one tire or the height of a vehicle body part are not within predefined limit or limits.
8. The method according to any of the preceding claims, the one or more actions further comprises modifying a braking and/or steering control mechanism. of the vehicle, disabling an autonomous or semi-autonomous mode of operating the vehicle, issuing a warning to a driver of the vehicle in response to the further verification indicating that a tire is leaking or has burst, or a combination thereof.
9. The method according to any of the preceding claims wherein the one or more actions further comprises calculating a safety travel distance for the vehicle.
10. The method according to any of the preceding claims further comprising transforming the first and second images to a common perspective view before aligning the first and second images to each other.
11. An apparatus for detecting a tire leak of a vehicle comprising: a processor; at least one memory coupled to the processor and storing 20 instructions executable by the processor causing the processor to: receive from a first image sensor located on the vehicle a first image comprising a first reference feature located externally on a first side of the vehicle; receive from a second image sensor located on the vehicle, a second image comprising a second reference feature located externally on a second opposite side of the vehicle; align the first and second images to each other; determine if the images of the first and second reference features are aligned after the first and second images are aligned to each other, wherein the first and second reference features are located at substantially the same height on the vehicle body and misalignment between the images of the first and second reference features is indicative that a tire located on the first or second side of the vehicle may be leaking; and cause the execution of one or more actions in response to the first reference feature image being misaligned with the second reference feature image.
12. The apparatus of claim 11 wherein the one or more actions caused to be executed comprises verifying if a tire located on the first or second side of the vehicle is leaking.
13. The apparatus of claim 13, wherein verifying if a tire is leaking comprises causing the processor to: receive at least one additional image of a third reference feature from the first image sensor and at least one additional image of a fourth reference feature from the second image sensor, wherein the additional images are captured in a temporally later time compared to the first and second images reference feature image; and determine if the third and fourth reference features in the 20 additional images are aligned to each other.
14. The apparatus according to any of claims 11-13, wherein the processor is further caused to: determine at least one of: a tire pressure of at least one 25 tire, a height of at least one tire or a height of a vehicle body part above a ground; and determine if the tire pressure of the at least one tire, the height of the at least one tire and/or the height of the vehicle body part above the ground are within predefined limit or limits. 30
15. The apparatus according to any of claims 11-14, wherein the processor is further caused to transform the first and second images to a common perspective view before aligning the first and second images to each other.
16. A non-transitory computer-readable storage medium comprising computer-readable instructions for carrying out the methods according to any of claims 1-10.