JP7048426B2 - How to perform sensor calibration and system - Google Patents

Description

The present invention relates to a method of performing calibration of a vehicle sensor. Another aspect of the invention relates to a sensor calibration performing system comprising at least one sensor, control unit and memory, in which case the system is adapted for performing the method.

Modern vehicles have multiple sensors through which information about the surroundings of the vehicle can be obtained. It must be ensured that the vehicle sensors are accurately calibrated to ensure reliable operation of the vehicle system that relies on the acquired data.

European Patent Application Publication No. 2793045 describes a method for inspecting a vehicle's peripheral detection system. Objects are detected by the first peripheral sensor type and are classified as static and dynamic objects. In addition, the relative position of the detected static object with respect to the vehicle is determined. This position is subsequently compared to the relative position detected using the second perimeter sensor type. Since the proper motion of the vehicle is also taken into account in the method, it is not necessary for the first peripheral sensor type sensor to detect the object at the same time as the second peripheral sensor type sensor.

German Patent Application Publication No. 102015009869 describes a method for estimating the self-position of a vehicle. In the method, a continuous image is detected and the actual position of the vehicle is determined by the position recorded and assigned to the image. In addition, suitable image features for self-position estimation are extracted in the image, such as lane markings or drain manhole covers. The image may be detected, for example, by a stereo camera.

International Publication No. 2012/05/2308 describes a method by which the base width of a stereo detection system can be determined thereby. In this case, it is designed to detect objects in the vicinity of the vehicle and determine the dimensions of the object and the distance to the vehicle via a stereo detection system. The determined dimension is compared with the reference dimension of the object determined at the reference distance. For comparison, objects with standard dimensions such as traffic signs and license plates with known heights are particularly suitable.

Disadvantages of known methods in the prior art are that the calibration of the sensor requires another sensor system unrelated to it, or that the system relies on the presence of standard known markings.

European Patent Application Publication No. 2793045 German Patent Application Publication No. 102015009869 International Publication No. 2012/05/2308

A method of performing calibration of the vehicle sensor is proposed. In step 1 a) of the method, sensors are designed to detect first data about the perimeter of the vehicle. Subsequently, in the second step b), the detected first data is used to determine the stationary structure and the parkable position, in which case the position of the stationary structure is also determined. In the next step c), a stationary structure is selected for calibration after the sensor. Further, in step d), the parkable position is selected, in which case the selected parking position is the stationary structure selected for sensor calibration after the vehicle has parked in the selected parking position. Selected to be in the field of view. In a subsequent step e), the vehicle is parked in the selected parking position and the position of the stationary structure selected for sensor calibration is stored. The parked vehicle is stopped at the parking position. When continuous running is intended, the vehicle is started by step f) of the method. In the subsequent step g), the sensor is used to detect the second data about the perimeter of the vehicle, and in the next step h) of the method, the detected second data is used to actually select the stationary structure. The position is determined. Step i) is followed by, in step i), the actual position is compared to the stored position and the deviation between the actual position and the stored position is determined. Finally, the sensor is calibrated using the determined deviation.

The method is applicable to only one sensor, as no data from other sensors is required to calibrate this sensor. However, the method is also applicable to use multiple sensors, in which case the sensors may be of the same type or of different types. When applied to multiple sensors, in particular steps a) to c) and g) to j) are designed to be performed separately for each sensor and independently of the other sensors, in this case. For the determination of the parkable position in step b), unlike that, a list of data of all related sensors may be referred to. Further, in this case, in the selection of the parkable position in step d), all the sensors are commonly considered so that the selected stationary structure exists in the respective field of view of all the sensors at the selected parking position. Is preferable.

Sensors calibrated by the method may be, for example, stereo cameras, video cameras, radar sensors, rider sensors or ultrasonic sensors. Further, by the method, a plurality of such sensors and a combination of the plurality of the sensors can be calibrated.

In the determination of the stationary structure according to step b), it is preferable that the object is recognized in the first data and the object is classified into a stationary object and a movable object. In this case, the stationary structure is a stationary object or part of a stationary object.

Algorithms generally known to those of skill in the art, tuned to the sensor type used, may be used for the recognition of objects in the first data. For example, in a stereo camera or a video camera, an image processing algorithm may be used. In the use of rider sensors, algorithms that identify objects by their characteristic shape may be used, and in the case of radar or ultrasonic sensors, classification is performed using the known size of the object and the distance of the object. You may be broken. It is preferred that a method tuned to each sensor type be used to perform object classification based on contour information. The contour may be determined directly depending on the sensor type, or may be determined from the sensor data via an evaluation method. Instead or in addition, mechanical engineering techniques may be used to recognize and classify objects within the sensor primitive data. For the recognition of an object, a collection containing known objects may be provided, each recording whether it is a stationary object or a movable object.

Examples for objects or parts of objects classified as stationary are houses, traffic signs, curb edges, walls, stanchions, trees, roof stanchions, drainage gutters, drainage grids, ground, walls, roofs, lighting covers, lighting, Windows, doors and other building openings. Examples for objects classified as moving objects are vehicles, baby carriages, people, animals, plants, foliage, and garbage containers.

The stationary structure may be the same as the recognized object or may be part of the object. In particular, for a relatively small object that can be detected as a whole by a sensor, the entire object can be selected as a stationary structure. This is especially the case for objects such as traffic signs. For relatively large objects, such as the case of a house, which are not entirely detectable by the sensor at once, it is preferred that characteristic portions such as edges or windows be selected as the stationary structure. Other examples for characteristic parts of objects are structural differences in ground, walls and roofs and contour information for windows, doors and other building openings.

The sensor to be calibrated has a recognizable field of view of this object inside. The field of view may be represented, for example, through a horizontal or vertical opening angle, in which case, for certain sensor types such as ultrasonic sensors and radar sensors, the maximum distance also limits the field of view.

It is preferable that the selection criteria are taken into consideration in the selection of the stationary structure according to step c). The selection criteria are, for example, the accuracy of the determined position of each stationary structure, the recognizability of the stationary structure under fluctuating peripheral conditions, the relative position of one stationary structure to other stationary structures, and at least two of these criteria. Selected from combinations.

Examples for fluctuating ambient conditions are, for example, meteorological and lighting conditions. For example, a stationary structure that is well recognized not only at night but also in daylight is preferable. Stationary objects such as trees and stanchions that sway in the wind, for example, are not preferred.

The more accurately the position of the stationary structure can be determined, the more appropriate this is for later calibration of the sensor, and this structure will be considered earlier in the selection.
For different sensor calibrations, such as stereo cameras, it is advantageous when multiple stationary structures are arranged relative to each other so that sensor calibration is possible for two axes that are orthogonal to each other. Is. That is, it is advantageous when at least three structures are placed together such that they are not placed on one common line.

In the selection of the selected parking position in step d), the parkable position is evaluated, and in this case, the parkable position is preferably selected by the best evaluation. In this case, the parkable position is better evaluated when the vehicle is parked in this parkable position, the more the selected stationary structure is located in the field of view of the sensor, respectively, in view of the parkable position. It is preferably designed to be. In other words, it is preferred to maximize the number of selected stationary structures visible to the sensor from the parkable position.

In the evaluation of the parkable position, the selected stationary structure may be shielded by the movable object when viewed from the parkable position, and may not be shielded by the movable object when viewed from the parkable position. It is preferably rated lower than the selected stationary structure. For this purpose, in particular, it may be designed to analyze whether another parkable position is located between the parkable position just inspected and the corresponding stationary structure. In such cases, vehicles parked in other parkable positions may block the sensor's view of this stationary structure. This stationary structure is evaluated lower because the corresponding stationary structure is not available at this time for calibration after the sensor.

The vehicle is preferably designed to be adapted for self-driving maneuvers, especially in lateral maneuvers or vehicle steering as well as vertical maneuvers or vehicle acceleration and braking. It will also be executed automatically. Within the scope of the method, the vehicle parked in step e) at this time may be designed to automatically arrive at the selected parking position.

In step e), it is further designed to memorize the position of the selected stationary structure. For this purpose, it is specifically designed to memorize the relative position of each stationary structure with respect to the vehicle. Optionally, for this purpose, the position of each stationary structure is determined again using sensors after reaching the selected parking position, or is calculated by known proper motion of the vehicle.

The vehicle shall be parked in the selected parking position for a long time until continuous driving is intended. After the vehicle has started for continuous running, the remaining steps g) to j) are executed. Following that, continuous running is possible with the calibrated sensor.

Another aspect of the invention is to provide a sensor calibration execution system. The system includes at least one sensor, control unit and memory. In this case, one of the methods described herein is designed to form and install an execution system.

Since the execution system of the described method is installed, the features described within the scope of the method are correspondingly applied to the system and vice versa. Applies to the method accordingly.

At least one sensor in the system is in this case calibrated using the method described.
The at least one sensor is, for example, a stereo camera, a video camera, a radar sensor, a rider sensor, an ultrasonic sensor, or a combination of at least two of these sensors.

After the vehicle is parked, the proposed method and proposed system allow for simple sensor recalibration, which in particular is based on temperature fluctuations while the vehicle is stopped, which causes sensor calibration. Defects can be compensated.

It is advantageous that the proposed sensor calibration method does not require a separate sensor from the sensor to be calibrated. Moreover, similarly, knowledge of a particular standard or pre-measured by other means is not required.

Before heading to the final stop position, a stationary structure suitable for later calibration is recognized in the perimeter of the vehicle, and the final stop position is from this stop position as many suitable stationary structures exist, ie. It is advantageous to be selected to be within the field of view of the sensor to be calibrated. In this case, this stationary structure selected is remembered so that it can be used in the restart of the vehicle.

In a preferred modification of the invention, the parking position selected as the final stop position is automatically reached so that the vehicle stops exactly at the optimum stop position.
In addition, it is advantageous to be designed to take into account the possibility of shielding of the selected stationary structure in determining the parking position selected. Thus, the probability of obstructing the view of the selected stationary structure of the sensor calibrated by, for example, another parked vehicle is reduced.

Examples of the present invention are described in the drawings and described in detail below.

FIG. 1 shows a schematic view of a vehicle having the system according to the present invention. FIG. 2 shows the selection of a parking position for parking a vehicle.

In the following description of an embodiment of the invention, the same parts and elements are designated by the same reference numerals, in which case repeated description of these parts or elements in individual cases is omitted. The drawings show the subject matter of the present invention only in schematic drawings.

FIG. 1 shows a vehicle 1 including a sensor calibration execution system 10. The system 10 includes a sensor 12, which is formed as a stereo camera in the embodiment shown in FIG. Further, the system 10 includes a control unit 14, which is coupled to the sensor 12 and the memory 16. Here, in particular, the control unit 14 and the memory 16 are designed to be formed in the form of only one control device.

The sensor 12 formed as a stereo camera has a field of view 13, which is directed forward in the embodiment shown in FIG.
The control unit 14 is coupled to other parts of the vehicle 1 to enable automatic driving of the vehicle 1. This is shown in the schematic of FIG. 1 by coupling to the handle 18 and coupling to the pedal 20.

FIG. 2 shows a vehicle 1 equipped with a system 10 according to the invention including a sensor 12. Vehicle 1 is on road 22 and is about to be parked. There is a first parkable position 24 and a second parkable position 26 at the right lane edge of the road 22. Since the parked vehicle 28 is already located at the other parking position 29, the other parking position 29 is not available.

Within the perimeter of the vehicle 1, there are various objects 40, 42 located within the field of view 13 of the sensor 12, and thus detectable by the sensor 12. The objects 40 and 42 can be classified into a stationary object 40 and a movable object 42. The stationary object 40 is immovable and therefore its position does not change. The stationary object 40 is usually fixedly coupled to the ground.

On the other hand, the movable object 42 can basically change its position even if they are stationary in the middle. An example of a movable object 42 is a parked vehicle 28 present on another parking position 29.

Examples for the stationary object 40 are the curb edge 30, the traffic sign 32, and the house 36 present in the field of view of the sensor 12.
When the vehicle 1 is parked on any of the parkable positions 24 and 26 for a long time, there is a possibility that the sensor 12 may be calibrated poorly due to, for example, temperature fluctuation. Therefore, for calibration of the sensor 12 performed when the vehicle restarts, there are as many stationary structures as possible in the field of view 13 of the sensor 12 whose position is known to the system 10 of the vehicle 1. When present, it is preferable to calibrate. That is, it allows the sensor 12 to be calibrated based on the deviations recognized at the positions of these known stationary structures determined by the sensor 12 after the restart.

To ensure the presence of as many such stationary structures as possible prior to parking, the present invention is designed to use the sensor 12 to detect first data about the perimeter of vehicle 1. The first data detected by the sensor 12 includes information about the objects 40, 42 located in the field of view 13 of the sensor 12. Subsequent to it, it is designed to determine a stationary structure that indicates a stationary object 40 or a portion of the stationary object 40. For this purpose, first, it is executed to classify the objects 40 and 42 detected via the sensor 12 into the stationary object 40 and the movable object 42. In this example of FIG. 2, the curb edge 30, the traffic sign 32, and the house 36 are recognized as the stationary object 40. Subsequently, in this example, a stationary structure is determined that includes the edge 38 of the house 36, the stanchions 34 of the traffic sign 32, and the curb edge 30. All these stationary structures are suitable for calibration and are therefore selected.

Further, it is advantageous that the detected first data is designed to identify the parkable positions 24, 26 in the periphery of the vehicle 1.
Subsequently, from the parkable positions 24 and 26, on the other hand, a parking position is selected from which as many stationary structures as possible are visible to the sensor 12, i.e. located within the field of view 13 of the sensor 12. Further, in this selection, it is preferable to consider whether the view of the sensor 12 to the corresponding stationary structure may be obstructed.

In the example of FIG. 2, when the first parkable position 24 is selected, first, not only the curb edge 30 and the traffic sign 32 but also the house 36 can be seen from the first parkable position 24 with respect to the sensor 12. Will. However, if another vehicle is parked on the second parkable position 26 after the vehicle 1 is parked on the first parkable position 24, the visibility of the sensor 12 to the traffic sign 32 and the house 36 is obstructed. There will be times. Therefore, in the selection, the second parkable position 26 is more highly valued and is determined as the correspondingly selected parking position 27. At this time, the vehicle 1 steers toward the parking position 27 selected by automatic driving, and information including the position of the stationary structure is stored in the memory 16 of the system 10. After reaching the selected parking position 27, the vehicle 1 is stopped.

After the vehicle 1 is restarted, the sensor 12 detects the second data about the periphery of the vehicle 1 and the position of the preselected stationary structure is determined in the second data. Subsequently, a comparison is made between the position of the stationary structure stored in the memory 16 and the actual position of the stationary structure determined from the second data. Since the stationary structure is immovable, it does not change its relative position with respect to vehicle 1. Therefore, if there is a deviation in position, it must be due to a poor calibration of the sensor 12. Therefore, it is subsequently designed to calibrate the sensor 12 with the perceived deviation. Following that, the vehicle 1 can start its travel.

The present invention is not limited to the examples described herein and the embodiments discussed therein. Rather, within the scope of the claims, a number of changes within the scope of the activities of those skilled in the art are possible.

1 Vehicle 10 System 12 Sensor 13 View 14 Control Unit 16 Memory 18 Handle 20 Pedal 22 Road 24 1st Parking Position 26 2nd Parking Position 27 Selected Parking Position 28 Parked Vehicle 29 Other Parking Position 30 Curb Edge Edge 32 Traffic signs 34 Pillars 36 Houses 38 Curbs 40 Stationary objects 42 Movable objects

Claims (11)

a) A step of detecting the first data about the periphery of the vehicle (1) using the sensor (12), and
b) A step of determining the stationary structure and the parkable position (24, 26) using the detected first data, and a step of determining the position of the stationary structure.
c) Steps to select a stationary structure for calibration after sensor (12),
d) In the step of selecting from the parkable positions (24, 26), the selected parking position (27) is the sensor (12) after the vehicle (1) is parked at the selected parking position (27). The step in which the stationary structure selected for calibration of is selected to be within the field of view (13) of the sensor (12), and
e) A step of parking the vehicle (1) in the selected parking position (27) and storing the position of the stationary structure selected for calibration of the sensor (12).
f) Steps to start the vehicle (1) and
g) A step of detecting the second data about the periphery of the vehicle (1) using the sensor (12), and
h) Using the detected second data, the step of determining the actual position of the selected stationary structure, and
i) The step of comparing the actual position of the selected stationary structure with the stored position of the selected stationary structure, where the actual position of the selected stationary structure and the selected stationary structure are stored. The steps that determine the deviation between the positions and
j) The step of calibrating the sensor (12) using the determined deviation,
A method of performing calibration of the sensor (12) of the vehicle (1), including. In the determination of the stationary structure by step b), the object (40, 42) is recognized in the first data and classified into the stationary object (40) and the movable object (42), in which case the stationary object is the stationary object (40). ) Or the method according to claim 1, wherein it is a part of a stationary object (40). House (36), traffic sign (32), rim stone edge (30), fence, stanchion, tree, roof stanchion, drainage gutter, drainage grid, ground, wall, roof, lighting cover, recognized in the first data. The method of claim 2, wherein the lights, windows, doors and other building openings are classified as stationary objects (40). The method according to claim 2 or 3, wherein the vehicle (28), the baby carriage, the person, the animal, the plant, the group of leaves, and the garbage container recognized in the first data are classified as a movable object (42). In the selection of stationary structures according to step c), the accuracy of the determined position of each stationary structure, the recognizability of the stationary structure under fluctuating peripheral conditions, the relative position of one stationary structure with respect to other stationary structures, and their criteria. The method according to any one of claims 1 to 4, wherein a selection criterion selected from at least two combinations of the above is taken into consideration. In the selection of the selected parking position (27) in step d), the parkable position (24,26) is evaluated, and in this case, the parkable position (24,26) is selected by the best evaluation. The method according to any one of claims 1 to 5. The parkable position (24,26) is better the more the selected stationary structure is located within the field of view (13) of the sensor (12), as viewed from the parkable position (24,26), respectively. 6. The method of claim 6, characterized in that it is evaluated in. In the evaluation of the parkable position (24,26), the selected stationary structure that may be shielded by the movable object (42) as seen from the parkable position (24,26) is the parkable position (24,26). 26), the method of claim 6 or 7, characterized in that it is rated lower than the selected stationary structure, which is unlikely to be shielded by the movable object (42). The method according to any one of claims 1 to 8, wherein the vehicle (1) automatically arrives at the selected parking position (27) in order to park in step e). At least one sensor (12), control unit (14) and memory (16), characterized in that the execution system (10) of the method according to any one of claims 1-9 is formed and installed. ), A sensor calibration execution system (10). The system (10) according to claim 10, wherein the at least one sensor (12) is a stereo camera, a video camera, a radar sensor, a rider sensor, an ultrasonic sensor, or a combination of at least two of these sensors.

Images