JP2019007739A - Self position estimation method and self position estimation device - Google Patents

Abstract

To improve estimation accuracy of the self position of a vehicle by estimating the amount of movement of the vehicle stably and accurately.SOLUTION: A first image is acquired by capturing an image of the periphery of the vehicle in the first direction by using a first camera, a second image is acquired by capturing an image of the periphery of the vehicle in the second direction different from the first direction by using a second camera, a first movement amount of the vehicle is calculated by using the first image, a second movement amount of the vehicle is calculated by using the second image, a third movement amount of the vehicle is calculated by using a vehicle signal output from the vehicle, the first movement amount is compared with the third movement amount, the second movement amount is compared with the third movement amount, one of the first movement amount, the second movement amount and the third movement amount is selected based on the result of the comparison and a position of the vehicle is estimated by using the selected movement amount.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a self-position estimation method and a self-position estimation apparatus.

  2. Description of the Related Art Conventionally, there is known a three-dimensional position measurement device that measures a three-dimensional position of an object based on a difference in position on an image of an object photographed by a camera at different times and a moving amount of the camera (Patent Document). 1). In Patent Document 1, the movement amount of the camera or the moving body is calculated from the change in the position of the feature point extracted from the image.

JP 2014-106092 A

  However, in Patent Document 1, although the amount of movement of the moving object is calculated using a camera, in order to improve the measurement accuracy of the three-dimensional position of the moving object among information obtained from a plurality of cameras, it is selectively used. Information is not used.

  The present invention has been made in view of such a conventional problem, and an object of the present invention is to improve the estimation accuracy of the vehicle's own position by stably and accurately calculating the movement amount of the vehicle. is there.

  In the self-position estimation method according to one aspect of the present invention, the first camera is used to capture an image of the surroundings of the vehicle in the first direction, the first image is obtained, and the second camera is used to obtain the first image. A second image is obtained by imaging the surroundings of the vehicle in a second direction different from the direction, the first movement amount of the vehicle is calculated using the first image, and the vehicle using the second image The second movement amount is calculated, the third movement amount of the vehicle is calculated using the vehicle signal output from the vehicle, the first movement amount is compared with the third movement amount, and the second movement amount is compared. The movement amount is compared with the third movement amount, and one of the first movement amount, the second movement amount, and the third movement amount is selected and selected based on the comparison result. The position of the vehicle is estimated using the travel amount.

  According to one aspect of the present invention, it is possible to improve the estimation accuracy of the vehicle's own position by stably and accurately calculating the amount of movement of the vehicle.

FIG. 1 is a functional block diagram showing the overall configuration of the self-position estimation apparatus according to the embodiment. FIG. 2 is a flowchart showing an example of the operation of the self-position estimation apparatus of FIG. 1 during learning (offline). FIG. 3 is a flowchart showing an example of the operation of the self-position estimation apparatus of FIG. 1 at the time of self-position estimation (on-line). FIG. 4A is a diagram illustrating temporal changes in the orientation of the front camera 12a and the rear camera 12b with respect to the sun 40. FIG. FIG. 4B is a diagram illustrating temporal changes of the front camera movement amount 41, the rear camera movement amount 42, and the odometry movement amount 43 when the vehicle 39 travels along the travel locus 38 of FIG. 4A.

  Next, embodiments will be described in detail with reference to the drawings.

<Self-position estimation device>
With reference to FIG. 1, the overall configuration of the self-position estimation apparatus according to the embodiment will be described. The self-position estimation device includes a front camera 12a (first camera) and a rear camera 12b (second camera) mounted on a vehicle, a vehicle sensor 13, and a feature point recording device 16 that records a feature point and a travel locus. And a controller 11 that learns checkpoints and travel trajectories and estimates the position of the vehicle.

  The front camera 12a captures the surroundings of the vehicle in a first direction (here, “front of the vehicle”) determined with reference to the vehicle, and generates image data (front image: first image). The rear camera 12b picks up an image of the periphery of the vehicle in a second direction (here, “rear of the vehicle”) that is different from the first direction determined with reference to the vehicle, and outputs image data (rear image: second image). Image). The cameras (12a, 12b) can simultaneously photograph around the vehicle in different directions. Here, the front-rear direction of the vehicle is illustrated as a different direction, but the left-right direction of the vehicle or a combination of other directions may be used. The angle difference between the first direction and the second direction does not need to be 180 °, and the imaging direction of the cameras (12a, 12b) is determined so as to have a predetermined angle difference that is not zero. .

  Each of the front camera 12a and the rear camera 12b may be a stereo camera or a monocular camera. The depth information of the object (feature point) can be acquired from the image data generated by each of the front camera 12a and the rear camera 12b. That is, it is possible to measure the three-dimensional position of an object around the vehicle from each image data.

  The vehicle sensor 13 is mounted on the vehicle and detects various information (vehicle signals) obtained from the vehicle. The vehicle sensor 13 includes, for example, a vehicle speed sensor that detects the traveling speed (vehicle speed) of the vehicle, a wheel speed sensor that detects the rotational speed of each tire included in the vehicle, and acceleration (including deceleration) in the three-axis directions of the vehicle. A triaxial acceleration sensor (G sensor) for detecting, a steering angle sensor for detecting a steering angle (including a turning angle), a gyro sensor for detecting an angular velocity generated in the vehicle, and a yaw rate sensor for detecting a yaw rate are included.

  The controller 11 learns the vehicle's own position and travel locus based on the data obtained from the front camera 12a, the rear camera 12b, and the vehicle sensor 13, and estimates the vehicle's own position based on the learning result.

  The controller 11 can be realized by using a general-purpose microcomputer including a CPU (Central Processing Unit), a memory, and an input / output unit. A computer program (self-position estimation program) for causing the controller 11 to function as a self-position estimation apparatus is installed and stored in a memory. By executing the computer program, the controller 11 functions as a plurality of information processing circuits (21 to 28) included in the self-position estimation apparatus. In addition, although embodiment demonstrates the example which implement | achieves several information processing circuits (21-28) with which a self-position estimation apparatus is provided with software, of course, the hardware for exclusive use for performing each information processing shown below is shown. It is also possible to prepare and configure the information processing circuits (21 to 28). Further, the plurality of information processing circuits (21 to 28) may be configured by individual hardware. Further, the information processing circuits (21 to 28) may also be used as an electronic control unit (ECU) used for other control related to the vehicle.

  The controller 11 includes, as a plurality of information processing circuits, an image acquisition unit 21 (image acquisition circuit), a front camera movement amount calculation unit 22 (first movement amount calculation circuit), and a rear camera movement amount calculation unit 23 (second Movement amount calculation circuit), vehicle odometry calculation unit 24 (third movement amount calculation circuit), movement amount difference calculation unit 25 (movement amount comparison circuit), movement amount selection unit 26 (movement amount selection circuit), , A feature point learning unit 27 and a self-position estimation unit 28 (self-position estimation circuit).

  A learning result by the feature point learning unit 27 is recorded in the feature point recording device 16.

  The image acquisition unit 21 acquires image data obtained by photographing each of the front camera 12a and the rear camera 12b. Specifically, the image acquisition unit 21 simultaneously acquires a time stamp indicating the photographing timing together with the image data.

  The front camera movement amount calculation unit 22 calculates the movement amount of the vehicle (front camera movement amount: first movement amount) using the front image obtained by the front camera 12a. The calculation method of the front camera movement amount is not particularly limited, and an existing method can be used. For example, first, feature points are extracted from the front image, and the three-dimensional positions of the feature points are specified based on the stereo matching processing of the feature points and the parallax information. Then, imaging is repeated while the vehicle travels, and feature point matching processing is performed between frames (images). From the change in the three-dimensional position of the feature point, the amount of movement of the vehicle (the amount of movement of the front camera) can be calculated. The rear camera movement amount calculation unit 23 calculates the movement amount of the vehicle (rear camera movement amount: second movement amount) using the rear image obtained by the rear camera 12b. Similarly to the front camera movement amount calculation unit 22, the rear camera movement amount calculation unit 23 also specifies the rear camera movement amount using an existing method.

  The vehicle odometry calculation unit 24 calculates the movement amount of the vehicle (odometry movement amount: third movement amount) based on the detection result from the vehicle sensor 13. First, the vehicle odometry calculation unit 24 acquires the detection result from the vehicle sensor 13 as a vehicle signal obtained from the vehicle. Specifically, the vehicle odometry calculation unit 24 simultaneously acquires a time stamp indicating the detection timing along with the vehicle signal. By receiving a time stamp indicating each of the imaging timing and the detection timing together with the image data and the vehicle signal, the image data and the vehicle signal can be associated with each other on the time axis.

  The vehicle odometry calculation unit 24 calculates the travel locus of the vehicle based on the vehicle signal detected by the vehicle sensor 13. The method for calculating the travel locus is not particularly limited, and an existing method can be used. For example, the yaw rate may be calculated from the difference between the rotation angles of the left and right tires, and the yaw angle of the vehicle (including the attitude of the vehicle and the traveling direction) may be calculated by integrating the yaw rate. Of course, a yaw rate sensor may be used. Further, the movement amount of the vehicle can be calculated from the rotation amount of the tire provided in the vehicle. The travel locus of the vehicle calculated based on the vehicle signal is referred to as “vehicle odometry”. The vehicle odometry calculation unit 24 calculates the amount of vehicle movement (odometry movement amount: third movement amount) in a predetermined time from the vehicle odometry.

  The “vehicle movement amount” including the front camera movement amount, the rear camera movement amount, and the odometry movement amount is a concept including the movement distance of the vehicle and the movement direction of the vehicle.

  The movement amount difference calculation unit 25 compares the front camera movement amount and the odometry movement amount, and compares the rear camera movement amount and the odometry movement amount. Specifically, the difference (forward difference) between the front camera movement amount and the odometry movement amount is calculated, and the difference (rear difference) between the rear camera movement amount and the odometry movement amount is calculated.

  The movement amount selection unit 26 selects one of the front camera movement amount, rear camera movement amount, and odometry movement amount as the vehicle movement amount based on the comparison result by the movement amount difference calculation unit 25. For example, a movement amount having a small difference from the odometry movement amount is selected from the front camera movement amount and the rear camera movement amount. In other words, the movement amount selection unit 26 compares the front difference calculated by the movement amount difference calculation unit 25 with the rear difference, and selects the smaller movement amount among the front difference and the rear difference. When the front difference is small, the front camera movement amount is selected, and when the rear difference is small, the rear camera movement amount is selected. When the front difference and the rear difference are the same, either the front camera movement amount or the rear camera movement amount may be selected. When both the front difference and the rear difference are outside the predetermined reference range, the odometry movement amount is selected without selecting the front camera movement amount and the rear camera movement amount.

  The amount of movement of the front camera and the amount of movement of the rear camera that are independently calculated based on the front image or the rear image are usually accurate with little error. However, the number of feature points extracted from the image can be reduced and the distance to the feature point can be accurately obtained due to environmental effects such as sunlight, the flatness of the landscape, and vehicle behavior due to turning and steps. It can be difficult. Furthermore, the number of feature points that succeed in matching decreases, or even if feature point matching succeeds, a subtle change occurs in the appearance of feature points, and the likelihood of matching decreases. For this reason, although the calculation accuracy of the amount of movement of the front camera and the amount of movement of the rear camera is usually high, the robustness is low, and it is significantly lowered due to the influence of disturbance.

  In addition, if the calculation accuracy of the camera movement amount is judged based only on the number of extracted feature points, the decrease in the number of extracted feature points and the decrease in the likelihood of matching are caused by sunlight. It is impossible to determine whether it is caused by a flat landscape with little change in luminance.

  Therefore, in order to stably obtain high accuracy with high robustness, first, by capturing the surroundings of the vehicle in different directions (front and rear) and acquiring the front image and the rear image, at least one of them is acquired. Make the image less susceptible to sunlight.

  Next, vehicle odometry calculated based on vehicle signals (for example, wheel side sensors) is generally known to accumulate errors and is not accurate in the long run, but when viewed locally The accuracy of vehicle odometry is stable and high. Therefore, each error of the front camera movement amount and the rear camera movement amount is evaluated using the odometry movement amount based on the vehicle odometry as a reference value (reference). That is, the movement amount difference calculation unit 25 calculates the front difference between the front camera movement amount and the odometry movement amount, and calculates the rear difference between the rear camera movement amount and the odometry movement amount.

  In principle, the movement amount selection unit 26 determines that the camera movement amount having the smaller difference from the odometry movement amount (reference) is the movement amount of the vehicle with higher accuracy, and selects this. However, if both the front camera movement amount and the rear camera movement amount have a large error, it is determined that the odometry movement amount used as a reference is a vehicle movement amount with higher accuracy than any camera movement amount. And select this.

  Thereby, the movement amount selection unit 26 can stably and accurately calculate the movement amount of the vehicle. In other words, it is possible to improve the robustness against the influence of the surroundings of the vehicle such as sunlight and a flat landscape with little luminance change.

  The feature point learning unit 27 calculates the map, the vehicle's own position on the map, and the traveling locus of the vehicle based on the front image or the rear image. The calculation method is not particularly limited, and an existing method can be used. For example, each of the front camera 12a and the rear camera 12b repeatedly captures images while the vehicle travels, and acquires a plurality of frames (images). Each of the front camera movement amount calculation unit 22 and the rear camera movement amount calculation unit 23 extracts feature points from the front image or the rear image, specifies the three-dimensional positions of the feature points, and features between frames (images). The amount of movement of the vehicle (the amount of movement of the front camera and the amount of movement of the rear camera) is specified from the change in the three-dimensional position of the point. On the other hand, the vehicle odometry calculation unit 24 calculates an odometry movement amount between frames (images) using a time stamp.

  The feature point learning unit 27 performs a feature point matching process between frames (images). Then, the movement amount selection unit 26 selects the movement amount of the vehicle between frames (images). The feature point learning unit 27 moves the three-dimensional position of the feature point according to the movement amount of the vehicle selected by the movement amount selection unit 26. By repeating this, the feature point learning unit 27 can represent the three-dimensional positions of the plurality of feature points extracted from the plurality of frames on one map (including the environment map). At the same time, the vehicle's own position on the map and the traveling locus of the vehicle can be specified. In addition, the traveling locus of the vehicle calculated based on the images of the cameras (12a, 12b) is referred to as “visual odometry (VO)”.

  As described above, when the feature point learning unit 27 learns the map (including the environment map), the vehicle's own position, and the travel locus of the vehicle based on the feature points, Since the amount of movement can be used, the accuracy of learning is improved.

  Further, the feature point learning unit 27 learns a point (check point) where the vehicle travels based on the images of the cameras (12a, 12b). Specifically, the check point is learned based on an image taken when the vehicle passes the check point. For example, the feature point learning unit 27 may learn a check point based on the position of the feature point extracted from the image or the three-dimensional position of the feature point. It should be noted that the feature point learning unit 27 may learn the travel locus (visual odometry) by dividing each check point, or may learn the travel locus without being divided. When dividing, it may be every predetermined time or every scene (for example, straight line, curve, highway, general road, intersection), and the dividing method is not limited. Further, the feature point learning unit 27 may learn a vehicle travel locus (vehicle odometry) calculated based on the vehicle signal instead of the travel locus visual odometry. The method for calculating the travel locus (vehicle odometry) is not particularly limited, and an existing method can be used. For example, the yaw rate may be calculated from the difference between the rotation angles of the left and right tires, and the yaw angle of the vehicle (including the attitude of the vehicle and the traveling direction) may be calculated by integrating the yaw rate. Of course, a yaw rate sensor may be used. Further, the movement amount of the vehicle can be calculated from the rotation amount of the tire provided in the vehicle.

  The learning result of the feature point learning unit 27 including the map, the vehicle's own position and travel locus, and the check point is recorded in the feature point recording device 16.

  The self-position estimating unit 28 estimates the self-position of the vehicle based on the map learned by the feature point learning unit 27, the self-position and traveling locus of the vehicle, and the check point. Specifically, it is confirmed whether or not the vehicle has passed the checkpoint, and the current position of the current vehicle is estimated based on the amount of movement of the vehicle from the checkpoint that passed last. At this time, the self-position estimation unit 28 uses the movement amount of the vehicle selected by the movement amount selection unit 26 as the movement amount of the vehicle from the check point that has passed last. As a result, the self-position estimation accuracy is improved.

  A method for confirming whether or not the vehicle has passed the checkpoint is, for example, as follows. The self-position estimation unit 28 reads the image (recorded image) when passing the checkpoint and the position information of the feature point in the recorded image from the feature point recording device 16. Then, the feature points in the image (target image) taken at the time of self-position estimation are associated with the feature points in the recorded image. When the feature points that have been successfully matched satisfy a predetermined condition, it can be determined that the position and orientation of the camera at the check point can be reproduced during travel control. That is, it can be determined that the target image has been acquired at the same camera position and orientation as when the recorded image was acquired. Therefore, in this case, the passage confirmation unit 24 determines that the vehicle has passed the check point.

  Note that whether or not a feature point that has been successfully matched satisfies a predetermined condition can be determined using, for example, the following method. First, let Fs be the number of feature points extracted from the recorded image. Of the feature points extracted from the target image, let Fn be the number of feature points successfully matched with the feature points extracted from the recorded image. For example, if the deviation of the relative position of the feature point with respect to the vehicle between the recorded image and the target image is less than a predetermined value, it is determined that the feature point has been successfully matched. The image passing certainty (Z) is defined as Z = Fn / Fs. The image passing certainty indicates the possibility that the vehicle has passed the checkpoint. When the image passing certainty is less than 0.8, the self-position estimating unit 28 has a low image passing certainty, so that the feature points that are successfully matched do not satisfy the predetermined condition. Is determined not to pass. When the image passing certainty factor is 0.8 or more, the self-position estimating unit 28 determines that the feature points that are successfully matched satisfy a predetermined condition, that is, the vehicle has passed the check point.

  In the self-position estimation process, each of the front camera 12a and the rear camera 12b repeatedly captures images while the vehicle travels, and acquires a plurality of frames (images). Each of the front camera movement amount calculation unit 22 and the rear camera movement amount calculation unit 23 extracts feature points from the front image or the rear image, specifies the three-dimensional positions of the feature points, and features between the frames (images). Performs a point matching process. Then, the moving amount of the vehicle (the moving amount of the front camera and the moving amount of the rear camera) is specified from the change in the three-dimensional position of the feature point between the frames (images). On the other hand, the vehicle odometry calculation unit 24 calculates an odometry movement amount between frames (images) using a time stamp. Then, the movement amount selection unit 26 selects one of the front camera movement amount, the rear camera movement amount, and the odometry movement amount based on the comparison result by the movement amount difference calculation unit 25. The self-position estimation unit 28 calculates the movement amount of the vehicle from the check point that has passed last based on the movement amount of the vehicle selected by the movement amount selection unit 26, and estimates the current position of the vehicle.

<Self-position estimation method>
Next, with reference to FIG.2 and FIG.3, the operation example of a self-position estimation apparatus is demonstrated. First, the operation of the self-position estimation apparatus during learning (offline) will be described with reference to FIG.

  In step S51, the image acquisition unit 21 acquires the front image and the rear image by photographing the surroundings of the vehicle using the cameras (12a, 12b). The image acquisition unit 21 synchronizes the shooting timing of the cameras (12a, 12b) and repeatedly shoots at a predetermined frame rate (1 to 5 FPS).

  In step S53, the front camera movement amount calculation unit 22 calculates the front camera movement amount using the front image obtained by the front camera 12a. In step S55, the rear camera movement amount calculation unit 23 calculates the rear camera movement amount using the rear image obtained by the rear camera 12b. Proceeding to step S57, the vehicle odometry calculation unit 24 calculates the odometry movement amount based on the detection result from the vehicle sensor 13.

  In step S59, the movement amount difference calculation unit 25 compares the front camera movement amount with the odometry movement amount. Specifically, the difference (forward difference) between the front camera movement amount and the odometry movement amount is calculated. In step S61, the movement amount difference calculation unit 25 compares the rear camera movement amount with the odometry movement amount. Specifically, the difference (rear difference) between the rear camera movement amount and the odometry movement amount is calculated. In step S63, the movement amount selection unit 26 determines whether both the front difference and the rear difference are out of the reference range. When both the front difference and the rear difference are outside the reference range (YES in S63), it can be estimated that both the front camera movement amount and the rear camera movement amount include a large error. Therefore, it progresses to step S67 and the movement amount selection part 26 selects the odometry movement amount.

  On the other hand, when at least one of the front difference and the rear difference is within the reference range (NO in S63), it can be estimated that at least one of the front camera movement amount and the rear camera movement amount does not include a large error. Therefore, it progresses to step S65 and the movement amount selection part 26 selects the movement amount of the smaller one of a front difference and a back difference. Thereby, the error of the front camera movement amount and the rear camera movement amount based on the odometry movement amount can be compared, and either one of the camera movement amounts having a small error can be selected. As described above, in S63 to S67, the movement amount selection unit 26 is based on the comparison result by the movement amount difference calculation unit 25, and is any one of the front camera movement amount, the rear camera movement amount, and the odometry movement amount. Can be selected.

  The feature point learning unit 27 learns the map, the vehicle's own position, the vehicle's travel locus, and the check point based on the feature point using the movement amount of the vehicle selected in step S65 or S67. The travel locus is recorded in the feature point recording device 16.

  Next, the operation of the self-position estimation apparatus at the time of self-position estimation (on-line) will be described with reference to FIG.

  In step S <b> 01, the self-position estimating unit 28 reads out the feature points and the travel locus from the feature point recording device 16. The read feature points are referred to as “recording feature points”. In step S03, the front camera 12a captures the surroundings of the vehicle, and the image acquisition unit 21 acquires a front image.

  In step S05, the front camera movement amount calculation unit 22 extracts feature points from the front image. The extracted feature points are called “extracted feature points”. In step S07, the front camera movement amount calculation unit 22 associates the recorded feature points with the extracted feature points. When the deviation between the relative position of the recorded feature point and the relative position of the extracted feature point is less than a predetermined value, it is determined that the recorded feature point and the extracted feature point are successfully associated with each other.

  In step S09, the front camera movement amount calculation unit 22 calculates the front camera movement amount using the extracted feature points that have been successfully matched.

  In step S11, the rear camera 12b captures the surroundings of the vehicle, and the image acquisition unit 21 acquires a rear image. Proceeding to step S13, the rear camera movement amount calculation unit 23 extracts feature points from the rear image obtained by the rear camera 12b. In step S15, the rear camera movement amount calculation unit 23 associates the recorded feature points with the extracted feature points.

  Proceeding to step S <b> 17, the rear camera movement amount calculation unit 23 calculates the rear camera movement amount using the extracted feature points that have been successfully matched.

  Proceeding to step S <b> 19, the vehicle odometry calculation unit 24 calculates the odometry movement amount based on the detection result from the vehicle sensor 13.

  In step S21, the movement amount difference calculation unit 25 compares the front camera movement amount with the odometry movement amount. Specifically, the difference (forward difference) between the front camera movement amount and the odometry movement amount is calculated. In step S23, the movement amount difference calculation unit 25 compares the rear camera movement amount with the odometry movement amount. Specifically, the difference (rear difference) between the rear camera movement amount and the odometry movement amount is calculated. Proceeding to step S25, the movement amount selecting unit 26 determines whether or not both the forward difference and the backward difference are outside the predetermined reference range. When both the front difference and the rear difference are outside the reference range (YES in S25), it can be estimated that both the front camera movement amount and the rear camera movement amount include a large error. Therefore, it progresses to step S29 and the movement amount selection part 26 selects the odometry movement amount.

  On the other hand, when at least one of the front difference and the rear difference is within the reference range (NO in S25), it can be estimated that at least one of the front camera movement amount and the rear camera movement amount does not include a large error. Therefore, it progresses to step S27 and the movement amount selection part 26 selects the movement amount of the smaller one among a front difference and a back difference. Thereby, the error of the front camera movement amount and the rear camera movement amount based on the odometry movement amount can be compared, and either one of the camera movement amounts having a small error can be selected. As described above, in S25 to S29, the movement amount selection unit 26 selects one of the front camera movement amount, the rear camera movement amount, and the odometry movement amount based on the comparison result by the movement amount difference calculation unit 25. Can be selected.

  Proceeding to step S31, the self-position estimating unit 28 confirms whether or not the vehicle has passed the check point based on the feature point read in step S01. Then, the current position of the current vehicle is estimated based on the amount of movement of the vehicle from the check point that passed last. At this time, the self-position estimating unit 28 uses the movement amount of the vehicle selected in S25 to S29 as the movement amount of the vehicle from the check point that passed last.

  As described above, at the time of learning (offline) and at the time of self-position estimation (on-line), the self-position estimation device performs feature point matching processing between frames (images), thereby moving the front camera movement amount and the rear The amount of camera movement is calculated. Therefore, the accuracy of the camera movement amount increases as the number of feature points extracted from the frame (image) or the number of feature points that succeed in the matching process increases.

  However, the number of feature points to be extracted, the number of feature points successfully matched, and the likelihood of matching are strongly influenced by the sun. As shown in FIG. 4A, when the shooting direction of the camera (12a, 12b) faces the sun 40, the luminance of the light incident on the imaging element of the camera exceeds the dynamic range of the imaging element, and the luminance distribution There will be fewer images. For this reason, the number of feature points extracted from an image having a small luminance distribution and the number of feature points that succeed in the matching process are greatly reduced, and the calculation accuracy of the camera movement amount is lowered.

  Furthermore, even if not affected by sunlight, if the luminance distribution is small in the original landscape, the image has a small luminance distribution. For this reason, when the number of feature points is used as a reference, it is not possible to distinguish between sunlight and a flat landscape.

  Therefore, as shown in FIG. 4B, the front camera movement amount 41 and the rear camera movement amount 42 are calculated independently, and at the same time, the odometry movement amount 43 is calculated. The front camera movement amount 41 and the rear camera movement amount 42 are fitted to the odometry movement amount 43 (function). A camera movement amount (41, 42) having a high degree of coincidence with the odometry movement amount 43 is selected.

  As shown in FIG. 4A, consider a scene in which a vehicle 39 travels along a travel locus 38. At the departure point, the front camera 12 a captures an image around the vehicle 39 in the direction opposite to the sun 40. On the other hand, the rear camera 12 b images the surroundings of the vehicle 39 toward the sun 40. The vehicle 39 turns left by 180 ° along the travel locus 38. At the arrival point, the front camera 12 a images the surroundings of the vehicle 39 toward the sun 40, and the rear camera 12 b images the surroundings of the vehicle 39 in the direction opposite to the sun 40.

  Therefore, as shown in FIG. 4B, at the departure point, the difference between the front camera movement amount 41 and the odometry movement amount 43 is smaller than the difference between the rear camera movement amount 42 and the odometry movement amount 43. Therefore, it is possible to select the front camera movement amount 41 with higher accuracy at the departure point.

  Thereafter, when turning left by 90 °, both the front camera 12a and the rear camera 12b are less affected by the sun 40, and both the front camera movement amount 41 and the rear camera movement amount 42 are different from the odometry movement amount 43. Get smaller.

  Then, at the arrival point that has further turned left by 90 °, the difference between the front camera movement amount 41 and the odometry movement amount 43 is larger than the difference between the rear camera movement amount 42 and the odometry movement amount 43. Therefore, the rear camera movement amount 42 with higher accuracy can be selected at the arrival point.

  As described above, according to the embodiment, the following operational effects can be obtained.

  The self-position estimation device compares the amount of movement of the front camera and the amount of odometry, compares the amount of movement of the rear camera and the amount of odometry, and based on the result of the comparison, the amount of movement of the front camera and the amount of movement of the rear camera Then, one of the odometry movement amounts is selected, and the position of the vehicle is estimated using the selected movement amount. Therefore, the self-position estimation apparatus can improve the self-position estimation accuracy of the vehicle by calculating the amount of movement of the vehicle stably and accurately. In other words, it is possible to improve the robustness against the influence of the surroundings of the vehicle such as sunlight and a flat landscape with little luminance change.

  The self-position estimation device calculates a difference between the front camera movement amount and the odometry movement amount, calculates a difference between the rear camera movement amount and the odometry movement amount, and among the front camera movement amount and the rear camera movement amount, the odometry movement is calculated. Select a movement amount with a small difference from the amount. Using the odometry movement amount as a reference value (reference), the magnitude of each error of the front camera movement amount and the rear camera movement amount can be evaluated. When viewed locally, the accuracy of the odometry movement amount calculated based on the vehicle signal is stable and high. Therefore, it is possible to select a camera movement amount with a small error.

  As shown in FIG. 4B, the self-position estimation apparatus calculates a front camera movement amount 41, a rear camera movement amount 42, and an odometry movement amount 43 in a predetermined local region, and the front camera movement amount 41 and the rear camera movement amount 42. Are fitted to the function 43 including the odometry movement amount, and the movement amount having a high match with the function 43 is selected from the front camera movement amount 41 and the rear camera movement amount 42. The degree of coincidence between the front camera movement amount 41 and the rear camera movement amount 42 can be evaluated using the odometry movement amount as the function 43. The accuracy of the odometry movement amount in the local region is stable and high. Therefore, it is possible to select a camera movement amount with a small error.

  The self-position estimation apparatus selects the odometry movement amount when the difference between the front camera movement amount and the odometry movement amount and the difference between the rear camera movement amount and the odometry movement amount are not within the predetermined range. When both the front camera movement amount and the rear camera movement amount have a large error, it can be determined that the odometry movement amount used as a reference is a vehicle movement amount with higher accuracy than any camera movement amount. Therefore, the movement amount of the vehicle with a small error can be selected.

  Although the contents of the present invention have been described according to the embodiments, the present invention is not limited to these descriptions, and it is obvious to those skilled in the art that various modifications and improvements can be made.

  The movement distance of the vehicle and the movement direction of the vehicle are collectively described as the movement amount of the vehicle. Not limited to this, the front camera movement amount, the rear camera movement amount, and the odometry movement amount are compared for each of the vehicle movement distance and the vehicle movement direction, and each of the vehicle movement distance and the vehicle movement direction is individually determined. You can choose. Alternatively, the movement amount of the vehicle may be replaced with one of the movement distance of the vehicle and the movement direction of the vehicle.

  In the flowcharts of FIGS. 2 and 3, the order may be changed as much as possible. For example, in FIG. 3, steps S03 to S09, steps S11 to S17, and step S19 may be performed with the order reversed. Or you may implement simultaneously.

  In addition, the self-position estimation apparatus in this embodiment can be used for automatic driving for driving the vehicle without driver intervention and manual driving for driving the vehicle based on driver intervention. When used for automatic driving, a travel locus of the vehicle is set using the self-position obtained by the self-position estimation apparatus in the present embodiment, and the vehicle is controlled based on the travel locus. When used for manual operation, it is possible to notify the occupant of the self position using the self position obtained by the self position estimating apparatus in the present embodiment, or to set a travel plan using a so-called navigation device. Moreover, the self-position estimation apparatus in this embodiment can be used when learning a target object (for example, a target, a white line, a road shape) and generating a map.

12a 1st camera 12b 2nd camera 21 Image acquisition part (image acquisition circuit)
22 Front camera movement amount calculation unit (first movement amount calculation circuit)
23 Rear camera movement amount calculation unit (second movement amount calculation circuit)
24 vehicle odometry calculation unit (third movement amount calculation circuit)
25 Movement amount difference calculation unit (movement amount comparison circuit)
26 Movement amount selection section (movement amount selection circuit)
28 Self-position estimation unit (self-position estimation circuit)
39 Vehicle 41 Front camera movement amount (first movement amount)
42 Rear camera travel (second travel)
43 Odometry travel (third travel)

Claims (5)

A self-position estimation method of a self-position estimation apparatus that estimates the position of the vehicle using a first camera and a second camera mounted on the vehicle,
Using the first camera to capture a first image by imaging the surroundings of the vehicle in a first direction;
Using the second camera to obtain a second image by imaging the periphery of the vehicle in a second direction different from the first direction;
A first movement amount of the vehicle is calculated using the first image;
A second movement amount of the vehicle is calculated using the second image;
A third movement amount of the vehicle is calculated using a vehicle signal output from the vehicle;
Comparing the first movement amount and the third movement amount;
Comparing the second movement amount and the third movement amount;
Based on the result of the comparison, select one of the first movement amount, the second movement amount, and the third movement amount,
A self-position estimation method for estimating the position of the vehicle using the selected amount of movement. The self-position estimation method according to claim 1,
Calculating a difference between the first movement amount and the third movement amount;
Calculating a difference between the second movement amount and the third movement amount;
A self-position estimation method for selecting a movement amount having a small difference from the third movement amount from the first movement amount and the second movement amount. The self-position estimation method according to claim 1,
Calculating the first movement amount, the second movement amount and the third movement amount in a predetermined local region;
Fitting each of the first movement amount and the second movement amount to a function consisting of a third movement amount;
A self-position estimation method for selecting a movement amount having a high match with the function from the first movement amount and the second movement amount. The self-position estimation method according to claim 2,
If neither the difference between the first movement amount and the third movement amount nor the difference between the second movement amount and the third movement amount falls within a predetermined reference range, A self-position estimation method for selecting a movement amount of 3. A self-position estimation device that estimates the position of the vehicle using a first camera and a second camera mounted on the vehicle,
The first camera is used to capture an image of the periphery of the vehicle in a first direction to obtain a first image, and the second camera is used to obtain a second direction different from the first direction. An image acquisition circuit that captures an image of the periphery of the vehicle to acquire a second image;
A first movement amount calculation circuit for calculating a first movement amount of the vehicle using the first image;
A second movement amount calculation circuit for calculating a second movement amount of the vehicle using the second image;
A third movement amount calculation circuit for calculating a third movement amount of the vehicle using a vehicle signal output from the vehicle;
A movement amount comparison circuit that compares the first movement amount and the third movement amount and compares the second movement amount and the third movement amount;
A movement amount selection circuit that selects any one of the first movement amount, the second movement amount, and the third movement amount based on a comparison result of the movement amount comparison circuit;
A self-position estimation apparatus comprising a self-position estimation circuit that estimates the position of the vehicle using the selected amount of movement.

Images