JP2021117176A - Ground feature position estimating device, vehicle position estimating device, and vehicle and ground feature position estimating method - Google Patents

Abstract

To provide a ground feature position estimating device, with which it is possible to appropriately estimate the position of a ground feature necessary for estimating the position of a vehicle while easily reducing costs.SOLUTION: A ground feature position estimating device 31 comprises: a ground feature position estimation unit 312 for estimating the position of a ground feature relative to a vehicle on the basis of an imaged image acquired from an imaging unit 15 fixed to the vehicle and including a ground feature; an attitude estimation unit 313 which, when it is determined that the vehicle was doing accelerated motion when the imaged image was imaged, estimates the attitude at accelerated motion time of the vehicle when the imaged image was imaged, on the basis of the reference attitude of the imaging unit 15 when the vehicle is stopped and the position of the ground feature in the imaged image; a correction amount calculation unit 314 for calculating the amount of correction of the estimated position of the ground feature on the basis of the attitude at accelerated motion time; and a correction processing unit 315 for correcting the position of the ground feature estimated by the ground feature position estimation unit 312, on the basis of the correction amount calculated by the correction amount calculation unit 314.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a feature position estimation device, a vehicle position estimation device, a vehicle, and a feature position estimation method.

In recent years, with the introduction of automatic driving technology for vehicles, it has become important to accurately detect the position of the vehicle. Therefore, a technique for accurately detecting the position of a vehicle is being studied (see, for example, Patent Document 1).

Patent Document 1 discloses a repetitive arithmetic control method in Kalman filter processing that estimates a state quantity including a vehicle position, a speed, and a sensor posture. In this method, the state quantity including the position, speed and attitude of the sensor is calculated. Next, among the calculated state quantities, the difference between the previous time and the current time is calculated for the sensor attitude value calculated from the attitude pitch angle of the acceleration sensor and the attitude yaw angle of the gyro sensor. Then, when the difference becomes larger than the previous difference, the difference is regarded as converged, the repetitive calculation of the state quantity is stopped, and the calculated state quantity is output. When the difference becomes equal to or less than the set value, the repeated calculation of the state quantity is stopped and the calculated state quantity is output.

Japanese Unexamined Patent Publication No. 2009-250778

However, in the technology as in Patent Document 1, a gyro sensor used only for estimating the posture of the sensor, that is, the posture of the vehicle is used, and a technology capable of reducing the cost has been desired.

The present disclosure provides a feature position estimation device, an own vehicle position estimation device, a vehicle and a feature position estimation method that can appropriately estimate the position of a feature for estimating the position of a vehicle while easily reducing costs. The purpose is to provide.

The feature position estimation device of the present disclosure includes a feature position estimation unit that estimates the position of the feature with respect to the vehicle based on an image captured including a feature acquired from an image pickup unit fixed to the vehicle, and the imaging unit. When it is determined that the vehicle is accelerating when the image is captured, it is based on the reference posture of the imaging unit when the vehicle is stopped and the position of the feature in the captured image. The feature position is estimated based on the posture estimation unit that estimates the posture of the vehicle during acceleration motion when the captured image is captured and the posture during acceleration motion estimated by the posture estimation unit. The ground estimated by the feature position estimation unit based on the correction amount calculation unit that calculates the correction amount of the estimated position of the feature estimated by the unit and the correction amount calculated by the correction amount calculation unit. It is provided with a correction processing unit that corrects the position of an object.

The own vehicle position estimation device of the present disclosure estimates the position of the vehicle based on the position of the feature obtained by the processing of the above-mentioned feature position estimation device and the correction processing unit of the feature position estimation device. It is equipped with a vehicle position estimation unit and a vehicle position estimation unit.

The vehicle of the present disclosure includes the above-mentioned own vehicle position estimation device.

The feature position estimation method of the present disclosure includes a feature position estimation step in which a computer estimates the position of the feature with respect to the vehicle based on an captured image including a feature acquired from an imaging unit fixed to the vehicle. When it is determined that the vehicle is accelerating when the captured image is captured, the reference posture of the imaging unit when the vehicle is stopped and the position of the feature in the captured image. Based on the posture estimation step for estimating the posture of the vehicle during acceleration motion when the captured image is captured, and the ground during the acceleration motion estimated by the posture estimation step. It is estimated in the feature position estimation step based on the correction amount calculation step for calculating the correction amount of the estimated position of the feature estimated in the object position estimation step and the correction amount calculated in the correction amount calculation step. The correction processing step of correcting the position of the feature is executed.

According to the feature position estimation device, the own vehicle position estimation device, the vehicle and the feature position estimation method of the present disclosure, the position of the feature for estimating the position of the vehicle is appropriately determined while easily reducing the cost. Can be estimated.

It is a block diagram which shows the schematic structure of the automatic operation control system which concerns on 1st and 2nd Embodiment of this disclosure. It is a flowchart of the own vehicle position estimation process in the own vehicle position estimation apparatus which concerns on 1st Embodiment of this disclosure. It is explanatory drawing of the feature position estimation model which concerns on 1st Embodiment of this disclosure. It is explanatory drawing of the feature position estimation model with respect to the feature on the right side of the vehicle which concerns on 2nd Embodiment of this disclosure. It is explanatory drawing of the feature position estimation model with respect to the feature on the left side of the vehicle which concerns on 2nd Embodiment of this disclosure.

[Embodiment]
<First Embodiment>
Hereinafter, the first embodiment of the present disclosure will be described with reference to the drawings.

[Configuration of automatic driving control system]
First, the configuration of the automatic driving control system according to the first embodiment of the present disclosure will be described. FIG. 1 is a block diagram showing a schematic configuration of an automatic driving control system.

The automatic driving control system 1 shown in FIG. 1 controls the automatic driving of the vehicle. The automatic driving control system 1 includes a GNSS (Global Navigation Satellite System) 11, an acceleration detection unit 12, a speed detection unit 13, a steering angle detection unit 14, an imaging unit 15, an input unit 16, and a map information storage unit. 17, a storage unit 18, and an automatic operation control device 20 are provided.

The acceleration detection unit 12 detects the acceleration or deceleration of the vehicle and outputs the detection result to the automatic driving control device 20.

The speed detection unit 13 detects the speed of the vehicle and outputs the detection result to the automatic driving control device 20.

The steering angle detection unit 14 detects the steering angle of the steering wheel and outputs the detection result to the automatic driving control device 20.

The imaging unit 15 includes a front camera 151, a right camera 152, a left camera 153, and a rear camera 154. The front camera 151, the right camera 152, the left camera 153, and the rear camera 154 are fixed so that their optical axes face the front, right, left, and rear of the vehicle, respectively. In the following, the front camera 151, the right camera 152, the left camera 153, and the rear camera 154 may be collectively referred to as "each camera 151, 152, 153, 154". Each camera 151, 152, 153, 154 images a feature based on the control of the automatic driving control device 20, and outputs the captured image to the automatic driving control device 20.

The input unit 16 is, for example, a touch panel arranged on the vehicle, and outputs information based on the input operation of the occupant to the automatic driving control device 20.

The map information storage unit 17 is composed of, for example, a non-volatile memory. The map information storage unit 17 stores map information. The map information includes information on the position and shape of features and the like. As information representing the position and shape of the feature, global coordinates that specify the actual position of the feature can be exemplified. Examples of features for estimating the position of a vehicle include buildings and other structures, white lines on roads, curbs, medians, traffic lights, signs, and bus stops.

The storage unit 18 is composed of, for example, a non-volatile memory. The storage unit 18 stores various information necessary for automatic driving of the vehicle. Examples of the information stored in the storage unit 18 include information acquired or generated by the automatic driving control device 20, fixed positions of the cameras 151, 152, 153, and 154, postures, and the like.

The automatic driving control device 20 controls the automatic driving of the vehicle. The automatic driving control device 20 includes a vehicle position estimation device 30 and a driving control device 40.

The own vehicle position estimation device 30 estimates the position of the vehicle (own vehicle position). The own vehicle position estimation device 30 includes a feature position estimation device 31 and an own vehicle position estimation unit 32.

The feature position estimation device 31 includes a processor and a memory. The feature position estimation device 31 executes the feature position estimation program stored in the memory by the processor. The feature position estimation device 31 includes a reference attitude parameter calculation unit 311, a feature position estimation unit 312, an attitude estimation unit 313, a correction amount calculation unit 314, and a correction processing unit 315.

The reference posture parameter calculation unit 311 calculates the reference posture of each camera 151, 152, 153, 154 when the vehicle is stopped as the reference posture parameter. The feature position estimation unit 312 estimates the position of the feature with respect to the vehicle based on the captured image including the feature acquired from each camera 151, 152, 153, 154. When the posture estimation unit 313 determines that the vehicle is accelerating when the captured image is captured, the posture estimation unit 313 determines the reference posture of each camera 151, 152, 153, 154 and the position of the feature in the captured image. Based on this, the acceleration posture of the vehicle when the captured image is captured is estimated. The acceleration motion means acceleration, deceleration, and turning. The correction amount calculation unit 314 calculates the correction amount of the estimated position of the feature based on the posture during acceleration. The correction processing unit 315 corrects the position of the feature estimated by the feature position estimation unit 312 based on the correction amount calculated by the correction amount calculation unit 314. The detailed processing of the reference attitude parameter calculation unit 311, the feature position estimation unit 312, the attitude estimation unit 313, the correction amount calculation unit 314, and the correction processing unit 315 will be described later.

When the vehicle is accelerating, the own vehicle position estimation unit 32 estimates the own vehicle position based on the position of the feature obtained by the processing of the correction processing unit 315 of the feature position estimation device 31. Further, the own vehicle position estimation unit 32 estimates the own vehicle position based on the position of the feature estimated by the feature position estimation unit 312 when the vehicle is not accelerating.

The driving control device 40 controls the driving of the vehicle. For example, the operation control device 40 is based on information on the destination point based on the operation of the passenger input unit 16, information on the current position input from the GNSS 11, map information stored in the map information storage unit 17, and the like. , Set the travel route. The driving control device 40 controls the driving of the vehicle so that the vehicle travels on the traveling route based on the own vehicle position estimated by the own vehicle position estimation device 30.

[Operation of automatic operation control system]
Next, as the operation of the automatic driving control system 1, the estimation process of the own vehicle position in the own vehicle position estimation device 30 will be described. In the following, the tilt angle when the vehicle is rolling may be referred to as a "roll angle", and the tilt angle when the vehicle is pitched may be referred to as a "pitch angle".

First, the outline of the own vehicle position estimation process will be described.

As the vehicle accelerates, the vehicle may pitch and tilt backwards. On the other hand, when the vehicle is decelerated, the vehicle may pitch and tilt so that the front is lowered. When the vehicle tilts, the optical axes of the cameras 151, 152, 153, and 154 also tilt. As a result, when a feature existing at the same position with respect to the vehicle is photographed, the position of the feature in the captured image when pitching may be significantly deviated from the position when not pitching. The own vehicle position estimation device 30 estimates the own vehicle position based on the position of the feature in the captured image, but when the vehicle is pitching, the own vehicle position may not be estimated appropriately. Therefore, the own vehicle position estimation device 30 determines that the vehicle is accelerating when the absolute value of the acceleration of the vehicle is equal to or greater than the acceleration threshold, that is, when the pitch angle of the vehicle is equal to or greater than the pitch angle threshold, and the captured image is displayed. After correcting the position of the feature estimated based on it, the position of the own vehicle is estimated.

Also, when the vehicle turns to the left (when turning), the vehicle may roll and tilt to the right. On the other hand, when the vehicle turns to the right (when turning), the vehicle may roll and tilt to the left. Similar to the case where the vehicle is pitching, when a feature existing at the same position with respect to the vehicle is photographed, the position of the feature in the captured image when rolling is changed from the position when not rolling. There is a risk of a large deviation. Therefore, the own vehicle position estimation device 30 may not be able to appropriately estimate the own vehicle position when the vehicle is rolling. Therefore, in the own vehicle position estimation device 30, when the absolute value of the steering angle of the steering wheel is equal to or greater than the steering angle threshold and the speed of the vehicle is equal to or greater than the speed threshold, that is, when the roll angle of the vehicle is equal to or greater than the roll angle threshold. It is determined that the vehicle is accelerating, the position of the feature estimated based on the captured image is corrected, and then the position of the own vehicle is estimated.

Further, when the pitch angle of the vehicle is less than the pitch angle threshold value and the roll angle is less than the roll angle threshold value, the own vehicle position estimation device 30 determines that the vehicle is not accelerating and estimates based on the captured image. Estimate the position of the own vehicle without correcting the position of the feature to be made.

Next, the details of the own vehicle position estimation process will be described. In the following, when the vehicle position does not deviate from the target position of the vehicle in the lateral direction (horizontal direction) (when it does not deviate laterally) and when it does not deviate in the front-rear direction (when it does not deviate vertically). The process of estimating the position of the own vehicle will be described. FIG. 2 is a flowchart of the own vehicle position estimation process in the own vehicle position estimation device. FIG. 3 is an explanatory diagram of the feature position estimation model.

As shown in FIG. 2, the reference posture parameter calculation unit 311 of the own vehicle position estimation device 30 calculates the reference posture parameters of the cameras 151, 152, 153, and 154 constituting the imaging unit 15 (step S1). The reference posture parameter represents the reference posture of each camera 151, 152, 153, 154 with respect to the vehicle. As the reference posture parameter, a parameter representing the direction of the optical axis of each camera 151, 152, 153, 154 can be exemplified.

The reference posture parameter calculation unit 311 causes each camera 151, 152, 153, 154 to take an image when the vehicle is stopped, and acquires an captured image from each camera 151, 152, 153, 154, respectively. Further, the reference posture parameter calculation unit 311 recognizes the current position of the vehicle based on the information input from the GNSS 11, and acquires the map information including the information of the current position from the map information storage unit 17. Next, the reference posture parameter calculation unit 311 uses, for example, the current position of the vehicle, the position of a feature (for example, a white line on the road) in the captured image acquired from each camera 151, 152, 153, 154, and map information. Reference orientations for each camera 151,152,153,154 using well-known methods (see, eg, republished patent WO 00/64175) for calculating camera orientations based on the global coordinates of the feature included. Calculate the parameters. Then, the reference posture parameter calculation unit 311 stores the calculated reference posture parameter in the storage unit 18. This enables coordinate conversion between the camera coordinate system based on the position of the camera and the local coordinate system C described later with the rotation center of the vehicle as the origin when the vehicle pitches or rolls.

Then, for example, after the vehicle starts traveling, the feature position estimation unit 312 controls each of the cameras 151, 152, 153, and 154 to acquire an captured image (step S2). Then, the feature position estimation unit 312 estimates the position of the feature (for example, the white line of the road) with respect to the vehicle (step S3). The feature position estimation unit 312 uses a feature position estimation model as shown in FIG. 3 when estimating the position of the feature. The feature position estimation model includes a vehicle-based local coordinate system C. Before the description of the process in step S3, the feature position estimation model will be described.

The origin O, X axis (not shown), Y axis, and Z axis of the local coordinate system C are defined as follows.
-When estimating the position of features in front of or behind the vehicle-Origin O: Center of rotation of the vehicle when the vehicle pitches-X-axis: Axis extending to the left and right of the vehicle-Y-axis: Axis extending to the front and back of the vehicle- Z-axis: Axis extending up and down the vehicle-When estimating the position of features on the right or left side of the vehicle-Origin O: Center of rotation of the vehicle when the vehicle rolls-X-axis: Axis extending forward and backward of the vehicle・ Y-axis: Axis that extends to the left and right of the vehicle ・ Z-axis: Axis that extends to the top and bottom of the vehicle

The position pointed to by the reference numeral K is the center of the lens of the camera. The solid line pointed to by the reference numeral F is the road surface. The alternate long and short dash line indicated by the symbol J is the optical axis of the camera when the vehicle is not tilted, that is, the optical axis represented by the reference attitude parameter. ΔD is the distance along the horizontal plane (XY plane of the local coordinate system C) from the center of rotation of the pitching or rolling of the vehicle to the center K of the lens of the camera. The vector M is a vector whose start point is the origin O and whose end point is the point F0 of the road surface F located below the origin O in the vertical direction. The vector L1 is a vector starting from the point F0 and ending at the center K of the lens. The vector L2 is a vector whose starting point is the origin O and whose end point is the center K of the lens.

Next, the process of step S3 will be described. Here, the estimation process of the position of the feature located in front of the vehicle will be described, but the estimation process of the feature located on the right, left, and rear of the vehicle is also performed in the same manner. The feature position estimation unit 312 recognizes the imaging position of the captured image based on the information input from the GNSS 11, and acquires the map information including the imaging position information from the map information storage unit 17. Then, the feature position estimation unit 312 uses a well-known technique for converting the position coordinates of the feature in the image captured by the camera into the position coordinates in the local coordinate system C, which is the feature in front of the vehicle included in the map information. By using it, the position coordinate W of the feature in the local coordinate system C is estimated. In this case, since the relative position of the center K of the camera lens with respect to the origin O of the local coordinate system is known, a vector N having the center K of the lens as the start point and the position coordinate W of the feature as the end point can also be estimated.

Here, when the vector N when the vehicle is not tilted is the vector N1, the vector N when the vehicle is tilted is the vector N2 tilted with respect to the vector N1. That is, the estimated position of the feature (position coordinate W2) when the vehicle is tilted is a position deviated from the estimated position (position coordinate W1) of the feature when the vehicle is not tilted. In reality, the center of rotation of the camera when the vehicle is tilted is a position deviated from the center K of the lens, but in the feature position estimation model of the first embodiment, the center K of the lens is The position of the feature is estimated assuming that it is the center of rotation of the camera.

Next, the posture estimation unit 313 acquires the detection result of the acceleration or deceleration of the vehicle from the acceleration detection unit 12, and determines whether or not the absolute value of the acceleration is equal to or greater than the acceleration threshold value (step S4). The acceleration threshold value is set to, for example, a value such that the pitch angle of the vehicle becomes equal to or higher than a predetermined pitch angle threshold value.

When the posture estimation unit 313 determines that the absolute value of the acceleration is equal to or greater than the acceleration threshold value (step S4: YES), the posture estimation unit 313 estimates the pitch angle of the vehicle (step S5). That is, the posture estimation unit 313 estimates the posture during acceleration of the present disclosure. For example, the posture estimation unit 313 estimates the pitch angle of the vehicle by calculating the pitch angle of the vehicle in which the position of the feature obtained from the map information and the position of the feature in the captured image captured by the camera match. The posture estimation unit 313 may estimate the pitch angle based on the features behind the vehicle, or may estimate the pitch angle based on the features in front of and behind the vehicle.

On the other hand, when the attitude estimation unit 313 determines that the absolute value of the acceleration is not equal to or greater than the acceleration threshold value (step S4: NO), the attitude estimation unit 313 acquires the detection result of the steering angle of the steering wheel from the steering angle detection unit 14, and the absolute steering angle is absolute. It is determined whether or not the value is equal to or greater than the steering angle threshold value (step S6). The steering angle when the vehicle is moving forward or backward is 0 °.

When the attitude estimation unit 313 determines that the absolute value of the steering angle is equal to or greater than the steering angle threshold value (step S6: YES), the attitude estimation unit 313 acquires the speed detection result from the speed detection unit 13 and determines whether the speed is equal to or greater than the speed threshold value. (Step S7). The steering angle threshold value and the speed threshold value are set to values such that the roll angle of the vehicle becomes equal to or higher than a predetermined roll angle threshold value, for example.

When the attitude estimation unit 313 determines that the speed is equal to or higher than the speed threshold value (step S7: YES), the attitude estimation unit 313 estimates the roll angle of the vehicle (step S8). That is, the posture estimation unit 313 estimates the posture during acceleration of the present disclosure. For example, as in the case of estimating the pitch angle, the attitude estimation unit 313 calculates the roll angle of the vehicle in which the position of the feature obtained from the map information and the position of the feature in the captured image captured by the camera match. Estimates the roll angle of the vehicle. The posture estimation unit 313 may estimate the roll angle based on the feature on the left side of the vehicle, or may estimate the roll angle based on the feature on the right side and the left side of the vehicle.

When the processing of step S5 or step S8 is performed, the correction amount calculation unit 314 calculates the correction amount ΔY of the estimated position of the feature (step S9). For example, the correction amount calculation unit 314 calculates the direction of the vector N1 obtained by rotating the vector N2 obtained in the process of step S3 when the vehicle is tilted by the pitch angle θp estimated in step S5. When the vehicle is tilted in the counterclockwise direction indicated by the arrow T in FIG. 3, the correction amount calculation unit 314 rotates a vector in the same direction as the vector N2 in the direction opposite to the tilt direction (clockwise direction). , Calculate the direction of the vector N1. Next, the correction amount calculation unit 314 calculates the magnitude of the vector N1 by utilizing the fact that its Z component is H. Here, H is the height of the center K of the lens from the road surface F. Then, the correction amount calculation unit 314 calculates the distance between the position coordinate W1 and the position coordinate W2 as the correction amount ΔY by using the following equation (1).
ΔY = | N1-N2 | ... (1)

Then, the correction processing unit 315 corrects the estimated position of the feature based on the captured image when the vehicle is tilted (step S10). For example, the correction processing unit 315 calculates a position moved from the position coordinate W2 along the road surface F by a correction amount ΔY, that is, the position coordinate W1 which is the end point of the vector N1. When the vehicle is tilted in the counterclockwise direction indicated by the arrow T in FIG. 3, the correction processing unit 315 calculates a position moved from the position coordinate W2 in the direction closer to the center of the vehicle by the correction amount ΔY.

Next, the own vehicle position estimation unit 32 estimates the own vehicle position (step S11). For example, the own vehicle position estimation unit 32 specifies the position coordinate W1 calculated in the process of step S10 as the estimated position of the feature obtained based on the captured image when the vehicle is not tilted, and is based on the estimated position. Estimate the position of the own vehicle.

The difference between the processes of steps S9 and S10 performed following the process of step S8, that is, the processes performed when the vehicle is rolling and the processes of steps S9 and S10 performed following the process of step S5. Since the point uses the roll angle θr instead of the pitch angle θp, the description thereof will be omitted.

After that, the feature position estimation unit 312 determines whether or not the operation is completed (step S12). When the feature position estimation unit 312 determines that the operation has been completed (step S12: YES), the feature estimation process ends the own vehicle position estimation process, and determines that the operation has not been completed (step S12: NO). The process of step S2 is performed.

On the other hand, the vehicle position estimation unit 32 determines that the absolute value of the steering angle is not equal to or greater than the steering angle threshold (step S6: NO), or that the speed is not equal to or greater than the speed threshold. In the case (step S7: NO), the position coordinate W1 calculated in the process of step S3 is specified as the estimated position of the feature, and the own vehicle position is estimated based on the estimated position.

[Action and effect of the first embodiment]
The feature position estimation device 31 estimates the position of the feature with respect to the vehicle based on the captured image acquired from the imaging unit 15. Next, when the feature position estimation device 31 determines that the vehicle is accelerating at the time of imaging, the feature position estimation device 31 sets the vehicle as the acceleration posture based on the reference posture parameter and the position of the feature in the captured image. Estimate the pitch or roll angle. Then, the feature position estimation device 31 calculates a correction amount ΔY of the estimated position of the feature based on the pitch angle or the roll angle, and based on the correction amount ΔY, the feature estimated based on the captured image. Correct the position. Therefore, the feature position estimation device 31 estimates the acceleration posture of the vehicle by using an imaging unit that can also be used for observing the surroundings of the vehicle without using the gyro sensor used only for estimating the posture of the vehicle. The position of the feature with respect to the vehicle can be appropriately estimated based on the attitude during acceleration. Then, the own vehicle position estimation unit 32 can appropriately estimate the own vehicle position based on the position of the feature with respect to the vehicle estimated by the feature position estimation device 31. Therefore, it is possible to provide the feature position estimation device 31 capable of appropriately estimating the position of the feature for estimating the position of the vehicle while easily reducing the cost.

When the vehicle position estimation device 30 determines that the vehicle is not accelerating at the time of imaging, the vehicle position estimation device 30 does not estimate the acceleration posture, but estimates the vehicle position based on the position of the feature estimated based on the captured image. do. As described above, when the vehicle is not accelerating at the time of imaging, the processing load of the own vehicle position estimation device 30 can be reduced by not performing the acceleration processing for estimating the posture during acceleration.

<Second embodiment>
Hereinafter, the second embodiment of the present disclosure will be described with reference to the drawings. In the first embodiment, the position of the own vehicle when the vehicle is not laterally displaced and vertically displaced is estimated, but in the second embodiment, the position of the own vehicle when rolling in the laterally displaced state is estimated. do. Since the configuration of the automatic driving control system 1 is the same as that of the first embodiment, the operation of the automatic driving control system 1 will be described. The state in which the vehicle is not rolling and is not laterally displaced is referred to as a "reference state", and the state in which the vehicle is rolling but not laterally displaced is referred to as a "first displacement state". Is rolling and laterally displaced is called a "second displacement state". FIG. 4 is an explanatory diagram of a feature position estimation model for a feature on the right side of the vehicle. FIG. 5 is an explanatory diagram of a feature position estimation model for a feature on the left side of the vehicle.

[Operation of automatic operation control system]
The own vehicle position estimation device 30 estimates the position of the feature based on the captured image after performing the processes of steps S1 and S2 shown in FIG. The feature position estimation unit 312 uses a feature position estimation model as shown in FIGS. 4 and 5 when estimating the position of the feature. The feature position estimation model includes a vehicle-based local coordinate system Cr. First, the local coordinate system Cr will be described with reference to FIGS. 4 and 5.

In the local coordinate system Cr, the center of rotation of rolling when the vehicle is viewed from the front side is the origin Or, the axis extending vertically from the origin Or is the Z axis, and the axis extending from the origin Or to the left and right of the vehicle is the Y axis. This is a coordinate system in which the axis extending from the origin Or to the front and rear of the vehicle is the X axis (not shown). The position pointed to by the symbol Km is the center of the lens of the right camera 152. The position pointed to by the reference numeral Kh is the center of the lens of the left camera 153. ΔDm is the distance along the horizontal plane (XY plane of the local coordinate system Cr) from the rotation center of the rolling of the vehicle to the center Km of the lens of the right camera 152. ΔDh is the distance along the horizontal plane from the center of rotation of the rolling of the vehicle to the center Kh of the lens of the left camera 153. The alternate long and short dash line indicated by the symbol Jm is the optical axis of the right camera 152 when the vehicle is in the reference state, that is, the optical axis represented by the reference attitude parameter. The alternate long and short dash line pointed to by the symbol Jh is the optical axis of the left camera 153 when the vehicle is in the reference state. The angle formed by the optical axis of the right camera 152 and the horizontal plane may be the same as or different from the angle formed by the optical axis of the left camera 153 and the horizontal plane.

Next, the process of estimating the position of the feature will be described. The feature position estimation unit 312 performs the same processing as in steps S1 to S3 of the first embodiment, and performs the same processing as the reference posture parameter of the right camera 152, the center of the lens of the right camera 152, and the feature on the right. The vector Nm connecting the positions of is estimated.

Here, assuming that the vector Nm when the vehicle is in the reference state is the vector Nm1, the vector Nm in the second displacement state is the vector Nm3 tilted with respect to the vector Nm1. That is, the estimated position (position coordinate Wm3) of the feature on the right side of the vehicle in the second displacement state is a position deviated from the estimated position (position coordinate Wm1) in the reference state.

Further, the feature position estimation unit 312 performs the same processing as the processing for the feature on the right side of the vehicle, calculates the vector Nh (Nh1, Nh3) corresponding to the feature on the left side, and takes the distance ΔDh into consideration. The position coordinates Wh (Wh1, Wh3) are estimated as the positions of the features on the left side with respect to the vehicle. Actually, the center of rotation of the camera in the second displacement state is a position deviated from the centers Km and Kh of the lens, but in the feature position estimation model of the second embodiment, the lens The position of the feature is estimated by assuming that the centers Km and Kh of the camera are the centers of rotation of the camera.

Next, the posture estimation unit 313 performs the processing of steps S6 and S7, and then performs the same processing as in step S8 to estimate the roll angle θr2 of the vector Nm3 in the second displacement state with respect to the vector Nm1 in the reference state. do. The roll angle of the vector Nh3 in the second displacement state with respect to the vector Nh1 in the reference state is also θr2.

After that, the correction amount calculation unit 314 calculates the correction amount ΔYm of the estimated position of the feature on the right side of the vehicle and the correction amount ΔYh of the estimated position of the feature on the left side of the vehicle. First, the correction amount calculation unit 314 performs the same processing as in step S3 of the first embodiment, and obtains vectors Nm1 and Nh1 corresponding to the right and left features of the vehicle when the vehicle is in the reference state. calculate.

Here, assuming that the vectors corresponding to the right and left features of the vehicle when the vehicle is in the first displacement state are Nm2 and Nh2, and the roll angle of the vectors Nm2 and Nh2 is θr1, the vector Nm2 is It can be said that the vector Nm1 is a vector obtained by rotating θr1 and multiplying by α, and the vector Nh2 is a vector obtained by rotating the vector Nh1 by θr1 and multiplying by β. That is, the following equations (2) and (3) hold.
Nm2 = R (θr1) ・ α ・ Nm1… (2)
Nh2 = R (θr1) ・ β ・ Nh1… (3)

Further, it is assumed that the estimated position of the feature in the first displacement state estimated based on the vectors Nm1 and Nh1 is the position coordinate Wm2, Wh2, the vector whose start point is the position coordinate Wm2 and whose end point is the position coordinate Wm3 is A. In this case, the following equation (4) holds. Further, since the vector having the position coordinate Wh2 as the start point and the position coordinate Wh3 as the end point is also A, the following equation (5) holds.
Nm3-A = Nm2 ... (4)
Nh3-A = Nh2 ... (5)

Substituting the equation (2) into the equation (4) gives the following equation (6), and substituting the equation (3) into the equation (5) gives the following equation (7).
Nm3-A = R (θr1) ・ α ・ Nm1… (6)
Nh3-A = R (θr1) ・ β ・ Nh1… (7)

Further, by eliminating the vector A from the equations (6) and (7), the following equation (8) is obtained.
Nm3-Nh3 = R (θr1) ・ (α ・ Nm1-β ・ Nh1)… (8)

The magnitude and orientation of the vectors Nm1, Nh1, Nm3, Nh3 are based on the roll angle θr2 of the vector Nm3 in the second displacement state, the height H of the center Km and Kh of the lens from the road surface F, the reference attitude parameter, and the like. , Can be calculated. Therefore, the correction amount calculation unit 314 can calculate θr1, α, and β based on the equation (8).

Further, the following equation (9) can be obtained from the equation (6).
A = Nm3-R (θr1) ・ α ・ Nm1… (9)

The correction amount calculation unit 314 calculates the magnitude ΔYa of the vector A, which is the amount of strike-slip, based on θr1 and α calculated based on the equation (8) and the equation (9).

Further, the correction amount calculation unit 314 is based on the θr1, α, β calculated based on the equation (8) and the equations (2), (3), and the vectors Nm2, Nh2 in the first displacement state. Is calculated. Then, the correction amount calculation unit 314 calculates the distance ΔYm1 between the position coordinate Wm1 and the position coordinate Wm2 using the following equation (10), and uses the equation (11) to obtain the position coordinate Wh1 and the position coordinate Wh2. The distance ΔYh1 of is calculated.
ΔYm1 = | Nm1-Nm2 | ... (10)
ΔYh1 = | Nh1-Nh2 | ... (11)

After that, the correction amount calculation unit 314 calculates the correction amount ΔYm of the estimated position of the feature on the right side of the vehicle using the following formula (12), and uses the formula (13) to calculate the ground on the left side of the vehicle. The correction amount ΔYh of the estimated position of the object is calculated.
ΔYm = ΔYm1 + ΔYa… (12)
ΔYh = ΔYh1 + ΔYa… (13)

Further, the correction amount calculation unit 314 calculates the correction amount ΔYm of the estimated position of the feature on the right side of the vehicle using the following formula (14), and uses the formula (15) to calculate the feature on the left side of the vehicle. It is also possible to calculate the correction amount ΔYh of the estimated position of.
ΔYm = | Nm1-Nm3 |… (14)
ΔYh = | Nh1-Nh3 |… (15)

After that, the correction processing unit 315 corrects the estimated position of the feature based on the captured image when the vehicle is in the second displacement state. For example, the correction processing unit 315 performs the same processing as in step S10 of the first embodiment, and moves the correction amount ΔYm from the position coordinate Wm3 along the road surface F, that is, the position coordinate Wm1 which is the end point of the vector Nm1. Is calculated, and the position moved by the correction amount ΔYh from the position coordinate Wh3, that is, the position coordinate Wh1 which is the end point of the vector Nh1 is calculated. At this time, when the vehicle is tilted in the counterclockwise direction indicated by the arrow T in FIGS. 4 and 5, the correction processing unit 315 moves the correction amount ΔYm from the position coordinate Wm3 toward the center of the vehicle. The position where the correction amount ΔYh is moved in the direction away from the center of the vehicle from the position coordinate Wh3 is calculated.

Then, the own vehicle position estimation unit 32 performs the same processing as in step S11 of the first embodiment, and the position coordinates Wm1 and Wh1 are obtained on the right side of the vehicle based on the captured image when the vehicle is in the reference state. It is specified as the estimated position of each feature on the left side, and the position of the own vehicle is estimated based on the estimated position.

[Action and effect of the second embodiment]
The feature position estimation device 31 calculates the correction amounts ΔYm and ΔYh in consideration of the lateral displacement, and corrects the position of the feature estimated based on the captured image based on the correction amounts ΔYm and ΔYh. Therefore, the feature position estimation device 31 can appropriately estimate the position of the feature with respect to the vehicle even if the vehicle is laterally displaced.

[Modification example]
Needless to say, the present disclosure is not limited to that shown in the embodiments described above, and various modifications can be made without departing from the spirit of the present disclosure.

For example, the reference attitude parameter may be input by the vehicle manufacturer or passenger. With such a configuration, it is not necessary to provide the reference attitude parameter calculation unit 311 in the feature position estimation device 31.

The own vehicle position estimation device 30 corrects the position of the feature in consideration of only the pitch angle or the roll angle, but the position of the feature may be corrected in consideration of both the pitch angle and the roll angle. Further, the own vehicle position estimation device 30 does not have to have one of the function of estimating the pitch angle and the function of estimating the roll angle. Further, the own vehicle position estimation device 30 of the second embodiment estimates the own vehicle position when the vehicle is rolled in a laterally displaced state, but the vehicle is vertically displaced by using the same method. You may estimate the position of the own vehicle when pitching while the vehicle is in the position. Further, the own vehicle position estimation device 30 has both a own vehicle position estimation function when the vehicle is rolling in a laterally displaced state and a own vehicle position estimation function when the vehicle is pitched in a vertically displaced state. You may have it.

The feature whose position is estimated by the feature position estimation device 31 may be a building such as a building, a white line on the road, a curb, a median strip, a signal, a sign, a bus stop, or the like, instead of the white line on the road. Further, the feature position estimation device 31 may estimate the positions of a plurality of features different from each other based on one captured image. The own vehicle position estimation unit 32 may estimate the own vehicle position based on the estimated positions of a plurality of features. The vehicle of the present disclosure may be a passenger car, a bus, a truck, a heavy machine, an agricultural machine, a motorcycle, a bicycle, a wheelchair, or the like.

Although the embodiments according to the present disclosure have been described in detail with reference to the drawings, the functions of the above-described devices can be realized by a computer program.

Computers that programmatically realize the functions of each of the above devices include input devices such as keyboards, mice, and touch pads, output devices such as displays and speakers, CPUs (Central Processing Units), ROMs (Read Only Memory), and RAMs (Random). Access Memory), storage devices such as hard disk devices and SSDs (Solid State Drives), readers that read information from recording media such as DVD-ROMs (Digital Versatile Disk Read Only Memory) and USB (Universal Serial Bus) memories, and networks. It is equipped with a network card that communicates via a bus, and each part is connected by a bus.

Then, the reading device reads the program from the recording medium on which the program for realizing the function of each of the above devices is recorded, and stores the program in the storage device. Alternatively, the network card communicates with the server device connected to the network, and stores the program downloaded from the server device for realizing the function of each device in the storage device.

Then, the CPU copies the program stored in the storage device to the RAM, sequentially reads the instructions included in the program from the RAM, and executes the program, thereby realizing the functions of the respective devices.

The present disclosure can be applied to a feature position estimation device, a vehicle position estimation device, a vehicle and a feature position estimation method.

1 Automatic operation control system 11 GNSS
12 Acceleration detection unit 13 Speed detection unit 14 Steering angle detection unit 15 Imaging unit 16 Input unit 17 Map information storage unit 18 Storage unit 20 Automatic operation control device 30 Vehicle position estimation device 31 Feature position estimation device 32 Vehicle position estimation unit 40 Operation control device 151 Front camera 152 Right camera 153 Left camera 154 Rear camera 311 Reference posture parameter calculation unit 312 Feature position estimation unit 313 Attitude estimation unit 314 Correction amount calculation unit 315 Correction processing unit

Claims (9)

A feature position estimation unit that estimates the position of the feature with respect to the vehicle based on an captured image including a feature acquired from an image pickup unit fixed to the vehicle.
When it is determined that the vehicle is accelerating when the captured image is captured, the reference posture of the imaging unit when the vehicle is stopped and the position of the feature in the captured image. Based on, a posture estimation unit that estimates the posture of the vehicle during acceleration motion when the captured image is captured, and a posture estimation unit.
A correction amount calculation unit that calculates a correction amount of the estimated position of the feature estimated by the feature position estimation unit based on the posture at the time of the acceleration motion estimated by the posture estimation unit.
A feature position estimation device including a correction processing unit that corrects the position of the feature estimated by the feature position estimation unit based on the correction amount calculated by the correction amount calculation unit. The feature position estimation device according to claim 1, wherein the acceleration motion of the vehicle is acceleration or deceleration. The ground according to claim 2, wherein when the attitude estimation unit determines that the absolute value of the acceleration of the vehicle detected by the acceleration detection unit is equal to or greater than the acceleration threshold value, the attitude estimation unit determines that the vehicle is accelerating. Object position estimation device. The feature position estimation device according to claim 1, wherein the acceleration motion of the vehicle is a turning motion. In the attitude estimation unit, the absolute value of the steering angle of the steering wheel of the vehicle detected by the steering angle detection unit is equal to or greater than the steering angle threshold value, and the speed of the vehicle detected by the speed detection unit is equal to or greater than the speed threshold value. The feature position estimation device according to claim 4, wherein if it is determined to be present, it is determined that the vehicle is accelerating. The feature position estimation device according to any one of claims 1 to 5,
A vehicle position estimation device including a vehicle position estimation unit that estimates the position of the vehicle based on the position of the feature obtained by the processing of the correction processing unit of the feature position estimation device. The vehicle position estimation unit estimates the position of the vehicle based on the position of the feature estimated by the feature position estimation unit when the vehicle is not accelerating. Own vehicle position estimation device. A vehicle comprising the own vehicle position estimation device according to claim 6 or 7. The computer
A feature position estimation step for estimating the position of the feature with respect to the vehicle based on an captured image including a feature acquired from an imaging unit fixed to the vehicle.
When it is determined that the vehicle is accelerating when the captured image is captured, the reference posture of the imaging unit when the vehicle is stopped and the position of the feature in the captured image. A posture estimation step for estimating the posture of the vehicle during acceleration motion when the captured image is captured, and a posture estimation step based on the above.
A correction amount calculation step for calculating the correction amount of the estimated position of the feature estimated by the feature position estimation step based on the posture at the time of the acceleration motion estimated by the posture estimation step, and a correction amount calculation step.
A feature position estimation method for executing a correction processing step for correcting the position of the feature estimated in the feature position estimation step based on the correction amount calculated in the correction amount calculation step.

Images