EP4511803A1

EP4511803A1 - Computer-implemented method of determining and adjusting camera pitch angle error of a vehicle camera

Info

Publication number: EP4511803A1
Application number: EP23718305.8A
Authority: EP
Inventors: Florin ISTOC
Original assignee: Continental Autonomous Mobility Germany GmbH
Current assignee: Aumovio Autonomous Mobility Germany GmbH
Priority date: 2022-04-21
Filing date: 2023-04-19
Publication date: 2025-02-26
Also published as: GB202206215D0; WO2023203092A1; GB2618127A; GB2618127B

Abstract

The invention refers to a computer-implemented method of determining and adjusting a camera pitch angle error of a vehicle camera acquiring images of a road plane. The method comprises: acquiring from the camera of a current image of the road plane, projecting the total road plane pixels of the current image, selecting the longitudinal lane marking pixels corresponding to a longitudinal lane marking, measuring the measured lane marking widths at predetermined distances in respect to camera position, computing for said current image a width difference between each measured lane marking width and the precedent measured lane marking, computing for each width difference a width difference per meter, computing a mean of the width differences per meter, comparing the mean of the width differences with a predetermined mean width threshold, correcting the initial pitch angle with a predefined correction coefficient parameter, outputting a corrected pitch angle, computing the mean of the width differences per meter corresponding to the corrected pitch angle, determining the camera pitch angle error by computing the difference between the mean of the width differences per meter corresponding to the corrected pitch angle and the mean of the width differences per meter corresponding the initial pitch angle, and finally adjusting the camera pitch angle corresponding to the error.

Description

202102044 1 Computer-implemented method of determining and adjusting camera pitch angle error of a vehicle camera Description The invention is related to cameras used in advanced driver assistance systems ADAS processing chain, particularly in the field of determining camera pitch angle error of a camera of a vehicle. The cameras used in the advanced driver assistance systems ADAS processing chain are mounted on the vehicles and adjusted vertically and horizontally. The field of view of each camera has a vertical component given by a pitch angle of the orientation of the camera in respect a horizontal line, and a horizontal component. The invention deals only with the pitch angle. The value of the pitch angle is computed statically based on optical flow and based on some assumptions in respect to the environment. For example road plane is assumed to be horizontal and the camera is mounted at a known height from the road plane. The raw images acquired by the camera are subject to usual deformation based on the fact that the objects that are closer appear to be bigger than the objects that are placed farther from the camera. During its functioning, the pitch angle of the camera suffers alterations, because of the continuous movement of the camera when the vehicle is in motion and/or because the mounting itself of the camera allows movement of the camera to a higher or lower position in respect to the initial position. The alteration of the pitch angle leads to pitch angle errors. In prior art, camera pitch angle errors are detected while the vehicle is in motion. The bird ‘s eye projection is sensitive to the camera pitch angle errors. Internal 202102044 2 In order to avoid the disadvantages of the prior art, the present invention has the objective of determining camera pitch angle error of a vehicle camera applicable to a wider range of situations, irrespective of whether the vehicle is in motion or is stopped. In order to overcome the disadvantages of prior art, in a first aspect of the invention it is presented a computer-implemented method of determining and adjusting a camera pitch angle error of a vehicle camera acquiring images of a road plane, the camera having an initial pitch angle, the method comprising the following steps: Step 1: acquiring from the camera of a current image of the road plane and representing the acquired image as total road plane pixels, Step 2: projecting the total road plane pixels of the current image from the road plane to a bird’s eye perspective of the road plane, outputting projected total road pixels, Step 3: selecting from the projected total road plane pixels the longitudinal lane marking pixels corresponding to a longitudinal lane marking, the longitudinal lane marking having a real lane marking width W expressed in pixels, Step 4: measuring the measured lane marking widths W1, W2, ... Wn at predetermined distances D1, D2, ..., Dn in respect to camera position along the selected lane marking and storing the measurements of the measured lane marking widths W1, W2, ... Wn corresponding to said predetermined distances D1, D2, ..., Dn, Step 5: computing for said current image a width difference between each measured lane marking width W_n and the precedent measured lane marking width Wn-1, computing for each width difference a width difference per meter Step 6: computing a mean of the width differences per meter corresponding to said image, comparing the mean of the width differences per meter with a predetermined mean width threshold, Step 7: if the absolute value of the mean of the width differences per meter is greater than the predetermined mean width threshold, correcting the initial pitch angle with a predefined correction coefficient parameter, outputting a corrected pitch angle and computing the mean of the width differences per meter corresponding to the corrected pitch angle, Internal 202102044 3 Step 8: determining the camera pitch angle error by computing the difference between the mean of the width differences per meter corresponding to the corrected pitch angle of step 7 and the mean of the width differences per meter corresponding the initial pitch angle of step 6, adjusting the camera pitch angle corresponding to the error and storing the camera pitch angle error for further use. In a second aspect of the invention, it is presented a hardware processing unit comprising one or more processors, at least one non-volatile memory and a non-transitory computer-readable storage medium, the hardware processing unit being configured to perform operations of the computer-implemented method according to any preferred embodiment. In a third aspect of the invention, it is presented a non-transitory computer-readable storage medium encoded with a computer program, the computer program comprising instructions executable by one or more processors, the one or more processors being configured to perform operations of the computer-implemented method according to any preferred embodiment. The advantages of the invention are the following: ^ The method of the invention is capable of determining camera pitch angle errors irrespective of whether the vehicle is in motion or is stopped, ^ The method of the invention is capable of determining camera pitch angle from a single image acquired from the camera. ^ The method of the invention is not computational -intensive. Further special features and advantages of the present invention can be taken from the following description of an advantageous embodiment by way of the accompanying drawing. Figure Fig.1 illustrates the principle behind the method of the invention. Internal 202102044 4 Detailed description The principle of the invention is based on the inventor’s first observation that the dimensions of the objects pictured in the images that are acquired in a bird’s eye perspective substantially reflect their real dimensions, in a totally contradiction with the same objects pictured in the images that are acquired in a camera perspective, when the camera is mounted on the vehicle. Based on the height at which the camera is mounted and based on the pitch angle of the camera, the raw images are processed by the camera by projecting them from a camera perspective to a bird's eye perspective. The bird's eye perspective, being essentially parallel to the road plane, is supposed to provide more accurate information about the distances between the objects placed in the field of view of the camera. The principle of the invention is equally based on the inventor’s second observation that, in order to check if the images that are acquired in the bird’s eye perspective indeed substantially reflect their real dimension, it’s convenient to use as benchmark objects from the road whose dimension do not change. The benchmark objects are marking lanes that are painted along the lanes on either side or on both sides, each marking lane having a real lane marking width W. The real lane marking width W is standardized, thus, it will always have the same width. It is not important for the invention on which side in respect to the ego vehicle the marking lanes are placed, but it is important that the same marking lane be used for the method of the invention. Starting from the inventor’s observations, the solving of the invention’s problem is made by comparing the width of the marking lanes in the bird’s eye perspective and by determining differences between them, as schematically shown in Fig.1. With reference to Fig.1, the inventors conceived in the first aspect of the invention a computer-implemented method of determining and adjusting a camera pitch angle Internal 202102044 5 error of the vehicle camera acquiring images of a road plane. The camera has an initial pitch angle, the value of which is known. The method comprises eight steps and is carried out by a hardware processing unit. In the first step, the camera acquires a current image of the road plane. The image acquired is undergoing the usual pre-processing operations of representing the image as total road plane pixels. The way the image acquired is represented as total road plane pixels is according to prior art. In the second step, the total road plane pixels of the current image are projected from the road plane to the bird’s eye perspective of the road plane, outputting projected total road pixels. In the third step, the longitudinal lane marking pixels corresponding to a longitudinal lane marking are selected from the projected total road plane pixels, the longitudinal lane marking having a real lane marking width W expressed in pixels. In the fourth step, the measured lane marking widths W1, W_2, ..., W_n are measured at predetermined distances D1, D2, ..., Dn in respect to camera position along the selected lane marking and the measurements of the measured lane marking widths W1, W_2, ..., W_n corresponding to said predetermined distances (D1, D2, ..., Dn) are stored. Fig.1 shows exemplification of three such predetermined distances D1, Dn-1 and Dn, corresponding to the points A, B and C on the image. The number of predetermined distances D1, Dn-1 and Dn depends on various factors such as but not limited to the level of accuracy of the camera and the speed of the vehicle. In the real life, since the real lane marking width W is substantially the same, there should be no differences between each measured lane marking width W_n and the Internal 202102044 6 precedent measured lane marking width Wn-1. If such differences exist, they are related to the camera pitch angle error. In the fifth step, for said current image, a width difference is computed between each measured lane marking width Wn and the precedent measured lane marking width W_n-1. At the same time, in the fifth step, a width difference per each meter of the real lane marking width W is computed for each width difference. In the sixth step it is computed a mean of the width differences per meter corresponding to said image, because the mean is more reliable than each of the width differences per each meter. The mean of the width differences per meter computed in the sixth step corresponds to the initial pitch angle. Then, the mean of the width differences per meter is compared with a predetermined mean width threshold. In the seventh step, if the absolute value of the mean of the width differences per meter is greater than the predetermined mean width threshold: the initial pitch angle is corrected with a predefined correction coefficient parameter, outputting a corrected pitch angle and the mean of the width differences per meter is computed corresponding to the corrected pitch angle, In the eighth step, the camera pitch angle error is determined by computing the difference between the mean of the width differences per meter corresponding to the corrected pitch angle of step 7 and the mean of the width differences per meter corresponding the initial pitch angle of step 6. Then, at the end of the eighth step, the camera pitch angle is adjusted corresponding to the error and the camera pitch angle error is stored in a memory for further use. Internal 202102044 7 The width difference per each meter depends linearly on two factors: on the camera pitch angle error on one hand and on the real lane marking width W on the other hand, the latter being presumed to be constant. Hereafter a non-limiting example of calculation in the eighth step: The predefined correction coefficient parameter is noted with Corr and the camera pitch angle error is noted with Err. The predefined correction coefficient parameter Corr is typically within the range from 0.1° to 1.0°. The two means of the width differences per meter are noted as follows: MWDi is the mean of the width differences per meter corresponding the initial pitch angle computed in step 6. MWDc is the mean of the width differences per meter corresponding to the corrected pitch angle computed in step 7. MWDc ~ (Err + Corr)*W this corresponds to the width difference per each meter after adjusting said camera pitch angle with the correction coefficient parameter Corr. MWDi ~ (Err)*W this corresponds to the width difference per each meter before adjusting said camera pitch angle with the correction coefficient: ε = MWDc / MWDi = (Err + Corr)*W/(Err)*W where ε is the ratio of the two means of the width differences per meter, ε ^{= MWDc / MWDi = (Err + Corr)/Err} ε x ^{Err = (Err + Corr)} Err = Corr/( ε -1) [1] Internal 202102044 8 The camera pitch angle error Err is computed using the equation 1 depending on the ^{mean of the width differences per meter} ε ^{and on the predefined correction} coefficient parameter Corr. The method as described above can be applied using a single image and can be applied irrespective of whether the vehicle is stopped or is in motion. In a first preferred embodiment, in order to improve the accuracy of the determination of the camera pitch angle error, a predetermined number k of successive images is acquired in step 1, where k ≥2, comprising at least one pair of images comprising a current image k+1 and a precedent image k. In the preferred embodiment, the step 5 further comprises computing the differences between the first measured lane marking width W₁ ^k+1 of the current image k+1 and the last measured lane marking width W_n ^k of the precedent image k, and the step 6 is further comprising computing the mean of the width differences per meter corresponding to the predetermined number k of successive images. The preferred embodiment has the advantage of further improving the accuracy of the method because the mean of the width differences per meter is computed based on a larger number of width differences per meter, In a second preferred embodiment, independently from the number of the images acquired from the camera, the predetermined width threshold is expressed as a percentage. For example, the predetermined mean width threshold is 5 %. If the mean width difference is greater than 5% for at least 11 times, then the warning message is sent. If the mean width difference is less than 5% for at least 9 times, then the warning message is not sent. In a third preferred embodiment, the result of the method of step 7 is used to correct the pitch angle of the camera, as follows: Internal 202102044 9 if the mean of the width differences has a positive value, the camera pitch angle has to be re-adjusted by increasing the camera pitch angle, or if the mean of the width differences has a negative value, the camera pitch angle has to be re-adjusted by decreasing the camera pitch angle. In a second aspect of the invention, it is presented a hardware processing unit comprising one or more processors, at least one non-volatile memory and a non-transitory computer-readable storage medium, the hardware processing unit being configured to perform operations of the computer-implemented method according to any preferred embodiment. In a third aspect of the invention, it is presented a non-transitory computer-readable storage medium encoded with a computer program, the computer program comprising instructions executable by one or more processors, the one or more processors being configured to perform operations of the computer-implemented method according to any preferred embodiment. While certain embodiments of the present invention have been described in detail, the person skilled in the art will recognize various alternative designs and embodiments for practicing the invention as defined by the following claims. Internal 202102044 10 List of reference signs W real lane marking width Dn predetermined distance W1, W2, ... Wn measured lane marking widths D1, D2, ..., Dn predetermined distances k ordinal number of the current image Internal

Claims

202102044 11 Patent claims 1. Computer-implemented method of determining and adjusting a camera pitch angle error of a vehicle camera acquiring images of a road plane, the camera having an initial pitch angle, c h a r a c t e r i z e d i n that the method comprises the following steps: Step1: acquiring from the camera of a current image of the road plane and representing the acquired image as total road plane pixels, Step 2: projecting the total road plane pixels of the current image from the road plane to a bird’s eye perspective of the road plane, outputting projected total road pixels, Step 3: selecting from the projected total road plane pixels the longitudinal lane marking pixels corresponding to a longitudinal lane marking, the longitudinal lane marking having a real lane marking width W expressed in pixels, Step 4: measuring the measured lane marking widths W1, W2, ... Wn at predetermined distances D1, D2, ..., Dn in respect to camera position along the selected lane marking and storing the measurements of the measured lane marking widths W1, W2, ... Wn corresponding to said predetermined distances D1, D2, ..., Dn, Step 5: computing for said current image a width difference between each measured lane marking width Wn and the precedent measured lane marking width Wn-1, computing for each width difference a width difference per meter Step 6: computing a mean of the width differences per meter corresponding to said image, comparing the mean of the width differences per meter with a predetermined mean width threshold, Step 7: if the absolute value of the mean of the width differences per meter is greater than the predetermined mean width threshold: correcting the initial pitch angle with a predefined correction coefficient parameter, outputting a corrected pitch angle, computing the mean of the width differences per meter corresponding to the corrected pitch angle, Step 8: determining the camera pitch angle error by computing the difference between the mean of the width differences per meter corresponding to the corrected pitch angle of step 7 and the mean of the width differences per meter corresponding the initial pitch angle of step 6, Internal 202102044 12 adjusting the camera pitch angle corresponding to the error and storing the camera pitch angle error for further use. 2. The method of claim 1 c h a r a c t e r i z e d i n that a predetermined number k of successive images is acquired in step 1, where k ≥2, comprising at least one pair of images comprising a current image k+1 and a precedent image k, the step 5 is further comprising computing the differences between the first measured lane marking width W₁ ^k+1 of the current image k+1 and the last measured lane marking width W_n ^k of the precedent image k the step 6 is further comprising computing the mean of the width differences per meter corresponding to the predetermined number k of successive images, 3. The method of claim 1 or 2, c h a r a c t e r i z e d i n that the predetermined width threshold is expressed as a percentage. 4. The method of any of the precedent claims, c h a r a c t e r i z e d i n that if the mean of the width differences per meter has a positive value, the camera pitch angle has to be re-adjusted by increasing the camera pitch angle, or if the mean of the width differences per meter has a negative value, the camera pitch angle has to be re-adjusted by decreasing the camera pitch angle. 5. A hardware processing unit comprising one or more processors, at least one non-volatile memory and a non-transitory computer-readable storage medium, the hardware processing unit being configured to perform operations of the computer-implemented method according to any of the preceding claims 6. A non-transitory computer-readable storage medium encoded with a computer program, the computer program comprising instructions executable by one or more processors, the one or more processors being configured to perform operations of the computer-implemented method according to any of the claims 1 to 4. Internal