JP2020076580A - Axial deviation estimation device - Google Patents

Abstract

To increase estimation accuracy of an axial deviation amount of a sensor mounted in a vehicle.SOLUTION: A first sensor and a second sensor mounted in a vehicle recognize a situation around the vehicle, and their visual fields at least partially overlap. An axial deviation estimation device estimates an axial deviation amount between the first sensor and the second sensor by contrasting a result of object recognition by the first sensor with a result of object recognition by the second sensor. At that time, the axial deviation estimation device sets as a mask area an area in which there is a high likelihood of presence of an object which causes decrease in estimation accuracy of the axial deviation amount. The mask area includes an area in a lane, for example. Also, the axial deviation estimation device extracts an object present in an area other than the mask area. The axial deviation estimation device estimates the axial deviation amount on the basis of the result of recognition of the object thus extracted.SELECTED DRAWING: Figure 8

Description

  The present invention relates to an axis deviation estimating device for estimating an axis deviation amount of a sensor mounted on a vehicle.

  Patent Document 1 discloses an obstacle recognition device that recognizes an obstacle by combining a plurality of pieces of sensor information. More specifically, the obstacle recognition device includes a camera, a radar, and a correction unit. The camera images the obstacle and acquires first parameter information indicating the imaging result. The radar measures an obstacle and acquires second parameter information indicating the measurement result. The correction means calculates the axial shift amount of the azimuth angle of the camera or the radar based on the first parameter information and the second parameter information. Then, the correction unit corrects the axis deviation of the camera or the radar based on the calculated axis deviation amount.

JP, 2010-249613, A

  According to Patent Document 1, the obstacle is recognized by each of the camera and the radar. However, when the recognition accuracy of the obstacle is low, the estimation accuracy of the axis deviation amount is also low. If the estimated axis deviation amount is not stable, it takes a long time to determine a highly reliable axis deviation amount.

  An object of the present invention is to provide a technique capable of improving the estimation accuracy of the amount of axial deviation of a sensor mounted on a vehicle.

One aspect of the present invention relates to an axis deviation estimating device.
The axis deviation estimation device estimates the axis deviation amount between the first sensor and the second sensor mounted on the vehicle.
The first sensor and the second sensor recognize a situation around the vehicle, and their respective visual fields at least partially overlap.
The axis deviation estimation device is
A storage device that stores first recognition information indicating a recognition result by the first sensor, second recognition information indicating a recognition result by the second sensor, and lane information indicating an arrangement of lanes around the vehicle,
A processor that estimates the amount of axis deviation between the first sensor and the second sensor.
The processor is
Based on the lane information, set at least one of the area within the lane and the area derived from the direction of the lane as the mask area,
An object existing in a region other than the mask region is extracted based on the first recognition information and the second recognition information,
The amount of axis deviation is estimated by comparing the recognition result of the object by the first sensor and the recognition result of the object by the second sensor.

  According to the present invention, the axis deviation amount is estimated by comparing the object recognition result by the first sensor and the object recognition result by the second sensor. However, not all objects are used unconditionally. A region in which there is a high possibility that there is an object that causes a decrease in the accuracy of estimating the amount of axis deviation is set as the mask region. Then, the object existing in the area other than the mask area is used for estimating the amount of misalignment. As a result, the accuracy of estimating the amount of misalignment is improved. Since the estimated axis deviation amount is stable, it is possible to quickly determine the axis deviation amount with high reliability.

It is a conceptual diagram for explaining the sensor mounted on the vehicle according to the embodiment of the present invention. It is a conceptual diagram for demonstrating the outline of the axis shift estimation process which concerns on a comparative example. It is a conceptual diagram for demonstrating the outline of the axis shift estimation process which concerns on a comparative example. It is a conceptual diagram for demonstrating the outline of the axis shift estimation process which concerns on a comparative example. It is a conceptual diagram for explaining the 1st example of the mask area | region which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating the 2nd example of the mask area | region which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating the 2nd example of the mask area | region which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating the axis shift estimation process which concerns on embodiment of this invention. It is a conceptual diagram for demonstrating the axis shift estimation process which concerns on embodiment of this invention. It is a block diagram showing an example of composition of an axis gap estimating device concerning an embodiment of the invention. It is a block diagram which shows the example of driving environment acquisition apparatus and driving environment information which concern on embodiment of this invention. It is a flowchart which shows the process by the processor of the axis shift estimation apparatus which concerns on embodiment of this invention.

  Embodiments of the present invention will be described with reference to the accompanying drawings.

1. Sensor Mounted on Vehicle FIG. 1 is a conceptual diagram for explaining a sensor mounted on the vehicle 1 according to the present embodiment. The vehicle 1 has a function of performing automatic driving. In order to perform automatic driving, it is necessary to recognize the situation around the vehicle 1. Therefore, the vehicle 1 is provided with a sensor for recognizing (detecting) the situation around the vehicle 1. More specifically, the sensor includes a first sensor 10 and a second sensor 20.

  The first sensor 10 is one of a camera, a lidar (LIDAR: Laser Imaging Detection and Ranging), and a radar. The camera images the surroundings of the vehicle 1. The camera is, for example, a stereo camera. The lidar and the radar are active sensors that emit electromagnetic waves (laser light, radio waves) and detect an object based on the reflected waves. For example, the rider can sequentially output (scan) the laser light in a plurality of directions and calculate the distance and direction of the reflection point based on the light receiving state of the reflected light.

  The second sensor 20 is also one of a camera, a rider, and a radar. However, the first sensor 10 and the second sensor 20 are not integrated but installed separately.

  The respective fields of view of the first sensor 10 and the second sensor 20 at least partially overlap. Therefore, the same object may be recognized by the first sensor 10 and the second sensor 20. When the two objects recognized by the first sensor 10 and the second sensor 20 satisfy a predetermined condition, the two objects are integrated as the same object. Such processing is called "fusion processing". The fusion process is used in automatic driving control.

  Here, the coordinate system will be defined. The “first coordinate system” is a sensor coordinate system fixed to the first sensor 10. The first coordinate system is represented by the X1, Y1, and Z1 axes that are orthogonal to each other. For example, the X1 axis is parallel to the front-back direction of the vehicle 1. Typically, the X1 axis coincides with its optical axis in the case of a camera and the central axis of the scanning range in the case of a lidar or radar. The Y1 axis is parallel to the lateral direction of the vehicle 1. The Z1 axis is parallel to the vertical direction.

  The “second coordinate system” is a sensor coordinate system fixed to the second sensor 20. The second coordinate system is represented by an X2 axis, a Y2 axis, and a Z2 axis that are orthogonal to each other. For example, the X2 axis is parallel to the front-back direction of the vehicle 1. Typically, the X2 axis coincides with its optical axis for cameras and the central axis of the scanning range for lidar and radar. The Y2 axis is parallel to the lateral direction of the vehicle 1. The Z2 axis is parallel to the vertical direction.

  When the attitude of the first sensor 10 changes, the direction of the first coordinate system changes. When the attitude of the second sensor 20 changes, the direction of the second coordinate system changes. Such a change is hereinafter referred to as "axis deviation". The axis misalignment may occur due to aging, contact between the sensor and an object, or the like. The amount of misalignment is the amount of misalignment. In the present embodiment, the axis deviation amount is represented by the magnitude of the deviation in the direction between the first coordinate system and the second coordinate system. In order to carry out the object recognition processing and the fusion processing with high accuracy, it is necessary to know the amount of axis deviation.

  The present embodiment proposes a method for accurately estimating the amount of axis deviation between the first sensor 10 (first coordinate system) and the second sensor 20 (second coordinate system). The process of estimating the amount of axis deviation will be referred to as "axis deviation estimation process" below. Hereinafter, the axis deviation estimation processing will be described in detail.

2. Axis deviation estimation process 2-1. Comparative Example First, a comparative example will be described. FIG. 2 is a conceptual diagram for explaining the outline of the axis deviation estimation process according to the comparative example. The vehicle 1 is in the lane LA. The lane LA is a track on which the vehicle travels. The lane boundary LB is a boundary between the lane LA and the road shoulder. The lane LA is an area sandwiched between the left and right lane boundaries LB.

  The results of object recognition (object detection) by the first sensor 10 and the second sensor 20 are used for the axis deviation estimation process. In FIG. 2, the other vehicle 2 and the road structure 3 are illustrated as the recognized objects. The other vehicle 2 is traveling in the lane LA. Examples of the road structure 3 include guardrails and walls. Typically, the road structure 3 is arranged along the lane LA (lane boundary LB) in a region outside the lane LA. In the example shown in FIG. 3, road structures 3L and 3R are present on the left side and the right side of the vehicle 1, respectively.

  FIG. 3 conceptually shows an example of the result of object recognition by each of the first sensor 10 and the second sensor 20. The first point group PC1 is a point group on the surface of the object detected by the first sensor 10. The position of the first point group PC1 is defined in the first coordinate system. The second point group PC2 is a point group on the surface of the object detected by the second sensor 20. The position of the second point group PC2 is defined in the second coordinate system. Then, coordinate conversion of at least one of the first coordinate system and the second coordinate system is performed so that the positional error between the first point group PC1 and the second point group PC2 is minimized. The deviation in the direction between the first coordinate system and the second coordinate system when the position error is the minimum is represented by an angle δ. This angle δ is estimated as the axis deviation amount δ.

  However, some objects are unsuitable for use in the axis deviation estimation process. For example, when the moving object such as the other vehicle 2 shown in FIG. 2 is used, the estimation accuracy of the axis deviation amount δ may decrease. This is because when the moving object moves, the detection position of the moving object shifts by the difference in the measurement timing between the first sensor 10 and the second sensor 20.

  As another example, consider a case where at least one of the first sensor 10 and the second sensor 20 is an active sensor (rider, radar). In the case of the active sensor, the weaker the intensity of the reflected wave, the lower the position accuracy of the detected object. That is, the positional accuracy of the reflection point having a low reflectance is low. For example, consider the point PX on the right road structure 3R in FIG. The incident angle of the electromagnetic wave emitted from the active sensor with respect to the point PX is small, and the reflectance at the point PX is low. Therefore, the position accuracy of the point PX is low. When such a point PX is used in the axis deviation estimation process, the accuracy of estimating the axis deviation amount δ is reduced.

  In this way, when an inappropriate object is used in the axis deviation estimation process, the accuracy of estimating the axis deviation amount δ is reduced. If the estimated axis deviation amount δ is not stable, it takes a long time to determine the highly reliable axis deviation amount δ.

  FIG. 4 shows a case where the estimation accuracy of the axis deviation amount δ is low. The horizontal axis represents time, and the vertical axis represents the estimated axis deviation amount δ. In order to determine the highly reliable axis deviation amount δ, the axis deviation amount δ is repeatedly estimated and accumulated. FIG. 4 also shows the average value μ and standard deviation σ of the plurality of axis deviation amounts δ estimated in a certain period.

In the example shown in FIG. 4, the estimated axis deviation amount δ is not stable and its variation is large. In a situation where the estimated axis deviation amount δ is not stable, the axis deviation amount δ cannot be fixed as one. For example, it is assumed that the determination condition for determining the amount of axis deviation δ is “the variance σ ² in a certain period is less than a threshold value”. In the situation as shown in FIG. 4, the time until the definite condition is established becomes long. In the worst case, the definite condition is not established forever, and as a result, the axis deviation amount δ is not established forever. If the axis deviation amount δ is not fixed, the automatic operation of the vehicle 1 cannot be started.

2-2. Therefore, according to the present embodiment, an object causing deterioration in estimation accuracy is excluded from the axial deviation estimation process according to the present embodiment. Therefore, the concept of "mask area MSK" is introduced. The mask region MSK is a region in which there is a high possibility that an object that causes a decrease in estimation accuracy will exist. By executing the axial deviation estimation process without using the object included in the mask region MSK, it is possible to prevent the estimation accuracy of the axial deviation amount δ from decreasing.

  FIG. 5 is a conceptual diagram for explaining the first example of the mask region MSK. A first example of the mask area MSK is an area in the lane LA. There is a high possibility that a moving object such as another vehicle 2 exists in the lane LA. Therefore, the moving object can be masked by setting the area in the lane LA as the mask area MSK. As a result, it is possible to prevent the estimation accuracy from deteriorating due to the moving object. The area in the lane LA is hereinafter referred to as “first mask area MSK1” for convenience.

  FIG. 6 is a conceptual diagram for explaining the second example of the mask region MSK. Hereinafter, the second example of the mask region MSK will be referred to as a “second mask region MSK2” for convenience. The second mask area MSK2 is also associated with the lane LA and is particularly derived from the direction of the lane LA.

  More specifically, the direction of the lane LA is the direction of the lane boundary LB which is the boundary between the lane LA and the road shoulder. As shown in FIG. 6, the angle θ is defined for each point PL on the lane boundary LB. The angle θ is an angle formed by the line-of-sight direction and the tangential direction. The line-of-sight direction is the direction from the first sensor 10 or the second sensor 20 to the point PL. The tangent direction is the tangent direction of the lane boundary LB at the point PL. The mask section MS is a section in which the angle θ is less than the threshold value θth. The second mask area MSK2 is an area along the lane boundary LB of the mask section MS and is an area located on the road shoulder side as viewed from the lane boundary LB.

  As can be seen from FIG. 6, the second mask region MSK2 defined as described above may be able to mask the road structure 3 having a low reflectance (for example, see the point PX in FIG. 2). high. When the road structure 3 having a low reflectance is masked, deterioration of estimation accuracy is prevented.

  FIG. 7 shows a setting example of the second mask area MSK2 in the case of a straight road. In the example shown in FIG. 7, the second mask region MSK2 masks the road structure 3 located far from the vehicle 1. In this case as well, the road structure 3 having a low reflectance is masked and the estimation accuracy is prevented from being lowered.

  The second mask region MSK2 may be different between the first sensor 10 and the second sensor 20. For example, the second mask region MSK2 regarding the first sensor 10 is defined by using the direction from the first sensor 10 to the point PL as the line-of-sight direction. The second mask region MSK2 regarding the second sensor 20 is defined by using the direction from the second sensor 20 to the point PL as the line-of-sight direction. Further, the first sensor 10 and the second sensor 20 may have different threshold values θth.

  FIG. 8 shows a case where the first mask region MSK1 and the second mask region MSK2 are applied to the situation shown in FIG. The moving object in the lane LA is masked by the first mask area MSK1. Further, the road structure 3 having a low reflectance is masked by the second mask area MSK2. That is, an object that causes a decrease in estimation accuracy is excluded from the axis deviation estimation process. As a result, the estimation accuracy of the axis deviation amount δ is improved.

  According to the present embodiment, at least one of first mask region MSK1 and second mask region MSK2 is set as mask region MSK. Even when only one of the first mask region MSK1 and the second mask region MSK2 is set as the mask region MSK, the effect of improving the estimation accuracy of the axis deviation amount δ can be obtained.

  Further, the area outside the visual field of the first sensor 10 and the area outside the visual field of the second sensor 20 may be set as the mask area MSK.

  FIG. 9 is a diagram for explaining the effect of the present embodiment, and its format is the same as that of FIG. 4 already described. According to the present embodiment, the estimation accuracy of the axis deviation amount δ is improved. Therefore, the estimated axis deviation amount δ is stable and its variation is small. Since the estimated axis deviation amount δ is stable, it is possible to quickly determine the axis deviation amount δ with high reliability. As a result, the time until the automatic driving of the vehicle 1 can be started is shortened.

  Hereinafter, an axis deviation estimation device that realizes the axis deviation estimation process according to the present embodiment will be described.

3. Axis deviation estimation device 3-1. Configuration Example FIG. 10 is a block diagram showing a configuration example of the axis deviation estimating device 100 according to the present embodiment. The axis deviation estimating device 100 is mounted in, for example, the vehicle 1. The axis deviation estimating device 100 receives, from the first sensor 10, the first recognition information D10 indicating the recognition result by the first sensor 10. Further, the axis deviation estimating device 100 receives, from the second sensor 20, the second recognition information D20 indicating the recognition result by the second sensor 20.

  The axis deviation estimation device 100 performs the axis deviation estimation process based on the first recognition information D10 and the second recognition information D20. More specifically, the axis deviation estimation device 100 includes a processor 110, a storage device 120, and a driving environment acquisition device 130.

  The processor 110 executes various processes including an axis deviation estimation process. For example, the processor 110 is included in a controller (ECU: Electronic Control Unit) that controls automatic driving of the vehicle 1. Various processes by the processor 110 are realized by the processor 110 executing the control programs stored in the storage device 120.

  The storage device 120 stores the first recognition information D10, the second recognition information D20, the lane information D30, and the driving environment information D130. The first recognition information D10 indicates the recognition result by the first sensor 10. The second recognition information D20 indicates the recognition result by the second sensor 20. The lane information D30 indicates the arrangement of the lane LA around the vehicle 1. The arrangement of the lane LA is a concept including the position, shape, width, etc. of the lane LA. The driving environment information D130 indicates the driving environment of the vehicle 1.

  The driving environment acquisition device 130 performs a driving environment acquisition process for acquiring the driving environment information D130. The processor that performs the driving environment acquisition process may be the same as or different from the processor 110 described above.

  FIG. 11 shows an example of the driving environment acquisition device 130 and the driving environment information D130. The driving environment acquisition device 130 includes a vehicle position sensor 131, a surrounding condition sensor 132, a map database 133, and a communication device 134. The driving environment information D130 includes vehicle position information D131, surrounding situation information D132, and map information D133.

  The vehicle position sensor 131 detects the position and direction of the vehicle 1 and acquires vehicle position information D131 indicating the position and direction of the vehicle 1. For example, the vehicle position sensor 131 includes a GPS (Global Positioning System) sensor. The GPS sensor receives signals transmitted from a plurality of GPS satellites and calculates the position and azimuth of the vehicle 1 based on the received signals.

  The surrounding situation sensor 132 detects the surrounding situation of the vehicle 1 and acquires the surrounding situation information D132 indicating the surrounding situation of the vehicle 1. For example, the ambient condition sensor 132 includes at least one of a camera, a rider, and a radar. The ambient condition sensor 132 may be the same as the first sensor 10 and the second sensor 20. The peripheral situation information D132 includes at least one of image pickup information indicating the image pickup result by the camera and measurement information indicating the measurement result by the rider or radar.

  The map information 133 is recorded in the map database 133. The map information D133 includes information on the arrangement of the lane LA. The driving environment acquisition device 130 acquires the necessary range of map information D133 from the map database 133. When the map database 133 is stored in the management server outside the vehicle 1, the driving environment acquisition device 130 communicates with the management server via the communication device 134 and acquires the map information D133 of the necessary range from the management server. To do.

  In addition, when the map information D133 includes the position of the characteristic object (eg, white line, signboard, pole) on the road, the driving environment acquisition device 130 performs a known self-position estimation process (localization) based on the driving environment information D130. ) May be performed. The self-position estimation process improves the accuracy of the vehicle position information D131.

3-2. Processing Flow FIG. 12 is a flowchart showing processing by the processor 110 of the axis deviation estimating device 100 according to this embodiment. The process shown in FIG. 12 is repeatedly executed every fixed cycle.

  In step S10, the processor 110 acquires the first recognition information D10 from the first sensor 10. Further, the processor 110 acquires the second recognition information D20 from the second sensor 20. Further, the processor 110 acquires the driving environment information D130 from the driving environment acquisition device 130. The processor 110 stores the acquired information in the storage device 120.

  In step S20, the processor 110 acquires the lane information D30 indicating the arrangement of the lane LA around the vehicle 1. More specifically, the processor 110 acquires the lane information D30 based on the driving environment information D130.

  For example, the map information D133 includes information on the arrangement of the lane LA. Therefore, the processor 110 can acquire the map information D133 around the vehicle 1 based on the vehicle position information D131, and can acquire the lane information D30 from the map information D133.

  As another example, the processor 110 may recognize the white line around the vehicle 1 based on the surrounding situation information D132 (imaging information and measurement information). In this case, the processor 110 estimates the arrangement of the lane LA from the recognized position of the white line, and acquires the lane information D30.

  In step S30, the processor 110 sets the mask area MSK related to the lane LA based on the lane information D30. The mask region MSK includes at least one of the first mask region MSK1 (see FIG. 5) and the second mask region MSK2 (see FIGS. 6 and 7).

  In step S40, the processor 110 estimates the axis deviation amount δ based on the first recognition information D10 and the second recognition information D20 other than the mask area MSK.

  More specifically, the processor 110 extracts an object existing in a region other than the mask region MSK based on the first recognition information D10 and the second recognition information D20. That is, the processor 110 limits the object extraction range by excluding the mask area MSK from the object extraction range.

  Then, the processor 110 estimates the axis deviation amount δ by comparing the object recognition result by the first sensor 10 and the object recognition result by the second sensor 20. For example, as shown in FIG. 3, coordinate conversion of at least one of the first coordinate system and the second coordinate system is performed so that the positional error between the first point group PC1 and the second point group PC2 is minimized. Is done. The angle between the first coordinate system and the second coordinate system when the position error is the minimum is estimated as the axis deviation amount δ.

In step S50, the processor 110 determines whether or not a predetermined confirmation condition is satisfied. For example, the definite condition is that the variance σ ^{2 of} the plurality of axis deviation amounts δ estimated in a certain period is less than the threshold value. When the definite condition is not satisfied (step S50; No), the process in this cycle ends. When the definite condition is satisfied (step S50; Yes), the process proceeds to step S60.

  In step S60, the processor 110 determines the axis deviation amount δ. For example, the processor 110 determines the average value μ of the plurality of axis deviation amounts δ estimated in a certain period as the axis deviation amount δ.

1 vehicle 2 other vehicle 3, 3L, 3R road structure 10 first sensor 20 second sensor 100 axis deviation estimation device 110 processor 120 storage device 130 driving environment acquisition device 131 vehicle position sensor 132 peripheral condition sensor 133 map database 134 communication device D10 1st recognition information D20 2nd recognition information D30 Lane information D130 Driving environment information D131 Vehicle position information D132 Surrounding situation information D133 Map information LA lane (runway)
LB lane boundary MSK Mask area MSK1 1st mask area MSK2 2nd mask area

Claims (1)

An axis deviation estimating device for estimating an axis deviation amount between a first sensor and a second sensor mounted on a vehicle,
The first sensor and the second sensor recognize a situation around the vehicle, and their respective visual fields at least partially overlap,
The axis deviation estimating device,
A storage device that stores first recognition information indicating a recognition result by the first sensor, second recognition information indicating a recognition result by the second sensor, and lane information indicating an arrangement of lanes around the vehicle. When,
And a processor that estimates the amount of axis deviation,
The processor is
Based on the lane information, at least one of the area in the lane and the area derived from the direction of the lane is set as a mask area,
An object existing in a region other than the mask region is extracted based on the first recognition information and the second recognition information,
An axis deviation estimating device for estimating the axis deviation amount by comparing a recognition result of the object by the first sensor and a recognition result of the object by the second sensor.

Images