JP5962331B2 - Speed estimation apparatus and speed estimation method - Google Patents

Description

  The present invention relates to a speed estimation device and a speed estimation method for estimating the speed of a vehicle based on, for example, a plurality of images taken while the vehicle is traveling with a camera mounted on the vehicle.

  In recent years, drive recorders (hereinafter referred to as simple drive recorders) that are equipped with acceleration sensors or position sensors that support Global Positioning System (GPS) and estimate the vehicle speed based on signals from these sensors are commercially available. ing. Since the simple drive recorder does not need to receive a signal representing the vehicle speed from the vehicle main body, it has an advantage that it can be easily attached to the vehicle.

  For the purpose of recording accident evidence, the drive recorder detects a large shake of the vehicle, such as during sudden braking, using an acceleration sensor as a trigger. It is a device that records vehicle behavior data. Recently, there are an increasing number of cases where the video and vehicle behavior data recorded by this drive recorder are used to analyze driving behavior in the event of a danger and to visualize or educate the situation for safe driving. . In the identification and situation analysis of danger scenes necessary for this danger analysis, it is necessary to analyze the behavior of the vehicle in detail, and therefore, highly accurate vehicle speed information is required for the analysis. However, in the method of estimating the vehicle speed based on the GPS signal, the accuracy of the obtained vehicle speed information is insufficient because the resolution of the position based on the GPS signal is insufficient.

  On the other hand, a method for estimating the vehicle speed using an acceleration sensor has been proposed (see, for example, Patent Document 1). However, in this method, there is a possibility that an error in the estimated value of the vehicle speed becomes large due to the influence of the vibration of the vehicle due to the unevenness of the road surface or the centrifugal force at the time of curve driving.

  In addition, there has been proposed a motion measurement device that estimates the motion of a moving body based on an image acquired by an imaging unit mounted on the moving body (see, for example, Patent Document 2). This motion measurement apparatus divides an image acquired by an imaging unit into a plurality of partial images, extracts feature points that should be candidates for fixed points in the outside world from the partial images, and extracts the feature points over time. Based on the tracked result, the motion of the moving body with respect to the fixed point is estimated.

Japanese Patent Laid-Open No. 10-9877 JP 2006-350897 A

  In the above motion measurement device, it is necessary that the number of feature points extracted from each partial image is not biased in order to improve the estimation accuracy of the motion of the moving object. This is because if the number of feature points extracted from each partial image is biased, the estimation accuracy of the motion of the moving object may be lowered.

  However, the pattern of the road surface on which the vehicle travels is scarce, and the number of feature points that can be extracted from the area in which the road surface is reflected on the image acquired by the in-vehicle camera is, for example, another There may be fewer than the number of feature points extracted from a region. Further, the road surface is usually shown below the image acquired by the in-vehicle camera. Therefore, the extracted feature points are unevenly distributed in the upper half of the image, for example.

  Therefore, the present specification aims to provide a speed estimation device that can improve the accuracy of estimation of vehicle speed based on these images even when feature points extracted from a plurality of images photographed by a vehicle-mounted camera are unevenly distributed. .

  According to one embodiment, a speed estimation device is provided. The speed estimation device includes: an acceleration detection unit that detects acceleration in a direction perpendicular to the traveling direction of the vehicle and parallel to the road surface; and the first estimation unit when the vehicle moves from the first point to the second point. An imaging unit that generates at least a part of the periphery of the vehicle at the point and generates a second image of at least a part of the periphery of the vehicle at the second point; For each of a plurality of points, a corresponding feature point on the first image is extracted and a corresponding feature point on the second image is extracted, and the first point based on the acceleration A traveling direction range calculation unit that identifies a range in which the traveling direction of the vehicle toward the second point can take, and a first image and a second image according to the traveling direction of the vehicle estimated within the range that the traveling direction can take The duplicate extracted for one of the images Each of the feature points is projected onto the other one of the first image and the second image, and an error in the position between each of the projected feature points and the corresponding feature point on the other image is calculated. A speed estimation unit configured to estimate a traveling direction of the vehicle so that the evaluation value to be expressed is minimized and to estimate a speed of the vehicle based on the estimated traveling direction.

The objects and advantages of the invention will be realized and attained by means of the elements and combinations particularly pointed out in the appended claims.
It should be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention as claimed.

  The speed estimation device disclosed in the present specification can improve the estimation accuracy of the vehicle speed based on the image even if feature points extracted from a plurality of images captured by the in-vehicle camera are unevenly distributed.

(A) is a figure which shows an example of the feature point and epipolar line when there is little bias of a feature point, (b) is a figure which shows an example of the feature point and epipolar line when a feature point is biased is there. It is a schematic block diagram of the simple drive recorder by which the speed estimation apparatus by one Embodiment was mounted. It is a figure which shows an example of arrangement | positioning of the simple drive recorder in a vehicle. It is a functional block diagram of a control part. It is a figure showing the relationship between the translation azimuth of a vehicle, and a turning angle. It is a figure which shows the relationship between one point of the to-be-photographed object and the corresponding feature point and epipolar line in the current image and the past image. It is a conceptual diagram showing the relationship between an evaluation value and a translational azimuth. It is a figure which shows an example of the relationship between the range of the value which a translation azimuth can take, and the image produced | generated by the imaging part. It is an operation | movement flowchart of an image recording determination process including a speed estimation process.

Hereinafter, a speed estimation device according to an embodiment will be described with reference to the drawings. First, an adverse effect caused by uneven distribution of feature points on the image will be described in detail.
In order to estimate the motion of a vehicle using a plurality of images taken when the vehicle is at different positions, a plurality of sets of feature points on each image corresponding to the same point on the object are extracted. . An epipolar line is obtained for each set of feature points. An epipolar line is projected on an image taken by a camera located at one of the two points when one point on the object is taken by a camera located at two different points. It is a line connecting the point and the other point. Then, an intersection of epipolar lines (called epipolar) for each set of feature points is obtained. The position of the epipolar on the image corresponds to the traveling direction of the vehicle.

FIG. 1A is a diagram illustrating an example of feature points and epipolar lines when the feature points are less biased, and FIG. 1B is an example of feature points and epipolar lines when the feature points are biased. FIG. In the image 100 shown in FIG. 1A, the feature points 101 to 105 are relatively distributed over the entire image 100. On the other hand, in the image 150 shown in FIG. 1B, the feature points 151 to 153 are unevenly distributed near the center of the right end of the image.
If the position of each feature point is separated from the characteristic of the epipolar line, the difference in the slope of the epipolar line for each feature point becomes large. Therefore, as shown in FIG. 1A, the angle formed by the epipolar lines 111 to 115 calculated for the feature points 101 to 105 is relatively large. On the other hand, if each feature point is close, the difference in the slope of the epipolar line for each feature point is small. Therefore, when feature points are unevenly distributed on the image, as shown in FIG. 1B, the angles formed by the epipolar lines 161 to 163 calculated for the feature points 151 to 153 are relatively small.

  If each feature point is accurately extracted, the position of the epipolar can be accurately obtained even if epipolar lines having a small intersecting angle are used. However, in general, the feature point extraction position obtained from the image includes an error. In addition, the object corresponding to the set of feature points may be a moving object. In such a case, the epipolar line required for the set of feature points is also inaccurate. At this time, if the angular difference between the epipolar lines is large, the displacement of the epipolar position is relatively small. For example, in FIG. 1A, it is assumed that an incorrect epipolar line 112 ′ is obtained for the feature point 102. In this case, the positional deviation between the intersection 121 of the epipolar line 114 and the epipolar line 112 and the intersection 122 of the epipolar line 114 and the erroneous epipolar line 112 ′ with respect to the feature point 104 located relatively far from the feature point 102 is compared. Small.

  On the other hand, if the angle difference between the epipolar lines is small, the position of the epipolar is greatly shifted even by a slight error of the epipolar lines. For example, in FIG. 1B, it is assumed that an incorrect epipolar line 162 ′ is obtained for the feature point 152. In this case, the positional deviation between the intersection 171 of the epipolar line 163 and the epipolar line 162 and the intersection 172 of the epipolar line 163 and the erroneous epipolar line 162 ′ with respect to the feature point 153 relatively close to the feature point 152 is large. As described above, if the position of the epipolar is greatly shifted, the result of the motion estimation of the vehicle is also inaccurate.

  Therefore, this speed estimation device represents the traveling direction of the vehicle between the two points where the image was taken, based on the acceleration detected by the acceleration sensor that detects the acceleration in the straight direction of the vehicle and the direction perpendicular to the vertical direction of the road surface. The range that the translational azimuth can take is limited. And this speed estimation apparatus estimates a translation azimuth within the range which can be taken, and estimates a vehicle speed based on the estimated translation azimuth. Thereby, this speed estimation device improves the accuracy of the position of the epipolar even if feature points extracted from images taken at two points are unevenly distributed on the image, and as a result, the estimation accuracy of the vehicle speed is improved. To improve.

  In this embodiment, the speed estimation device is mounted on a simple drive recorder or software or a device that analyzes a dangerous behavior of a vehicle in detail or deletes unnecessary data that is not dangerous. However, the speed estimation device can be used for various devices that use the speed of the vehicle without receiving a vehicle speed signal from the vehicle body, for example, an autonomous navigation combined type navigator.

  FIG. 2 is a schematic configuration diagram of a simple drive recorder in which a speed estimation device according to one embodiment is mounted. FIG. 3 is a diagram showing an example of the arrangement of the simple drive recorder in the vehicle. The simple drive recorder 1 includes an imaging unit 2, an acceleration detection unit 3, a position detection unit 4, a temporary storage unit 5, a storage unit 6, and a control unit 7. The imaging unit 2, the acceleration detection unit 3, the position detection unit 4, the temporary storage unit 5, the storage unit 6, and the control unit 7 are accommodated in one housing (not shown). Alternatively, the acceleration detection unit 3, the position detection unit 4, the temporary storage unit 5, the storage unit 6, and the control unit 7 are accommodated in one casing, and the imaging unit 2 is connected to the casing through a signal line. Also good. As shown in FIG. 3, the simple drive recorder 1 is installed, for example, near the upper end of the windshield inside the vehicle 10.

  The imaging unit 2 includes image sensors of solid-state imaging devices arranged in a two-dimensional array, and an imaging optical system that forms an image of a subject on the image sensor. And the imaging part 2 is installed so that at least a part of the periphery of the vehicle 10, for example, the front of the vehicle 10 can be photographed, and the vehicle periodically at a predetermined photographing rate (for example, 10 to 30 frames / second). An image showing the front of 10 is generated. Note that the image generated by the imaging unit 2 may be a color image or a gray image. The imaging unit 2 passes the image to the control unit 7 every time an image is generated.

  The acceleration detection unit 3 includes an acceleration sensor, for example, and detects lateral acceleration, which is acceleration in a direction perpendicular to the straight traveling direction of the vehicle 10 and the vertical direction of the road surface, as needed. This acceleration sensor may be any of a semiconductor type, an optical type, or a mechanical type acceleration sensor. The acceleration detection unit 3 notifies the control unit 7 of the lateral acceleration every time it detects the lateral acceleration.

  The position detection unit 4 includes, for example, a GPS receiver and a positioning information receiving antenna. And the position detection part 4 calculates | requires the longitude and latitude of the present position of the vehicle 10 based on the positioning information transmitted from a GPS satellite. The position detection unit 4 passes information indicating the current position of the vehicle 10 to the control unit 7.

  The temporary storage unit 5 includes, for example, a ring buffer, and temporarily stores an image generated by the imaging unit 2 and received via the control unit 7. The temporary storage unit 5 stores the image included in the latest predetermined period, for example, 10 seconds to 20 seconds, by discarding the oldest image every time a new image is received from the control unit 7.

  The storage unit 6 includes, for example, a nonvolatile semiconductor memory having a storage capacity larger than that of the temporary storage unit 5. The storage unit 6 stores at least one set of images for a predetermined period received from the temporary storage unit 5 via the control unit 7. In addition, the storage unit 6 may store data related to the vehicle 10 used for estimating the vehicle speed, such as the wheel base of the vehicle 10, the maximum steering angle, and map information.

The control unit 7 includes, for example, at least one processor, a volatile readable / writable memory, and a peripheral circuit, and controls the entire simple drive recorder 1. The control unit 7 stores the image in the temporary storage unit 5 every time an image is received from the imaging unit 2.
The control unit 7 detects when an accident such as the collision of the vehicle 10 with another object occurs or when the vehicle 10 performs a dangerous behavior that may be involved in the accident. Are read from the temporary storage unit 5 and stored in the storage unit 6. Further, the control unit 7 estimates the speed of the vehicle 10 based on the image obtained from the imaging unit 2 and the lateral acceleration of the vehicle 10 obtained from the acceleration detection unit 3.

  FIG. 4 is a functional block diagram of the control unit 7. The control unit 7 includes a tracking unit 21, a traveling direction range calculation unit 22, a speed estimation unit 23, and a recording determination unit 24. Each of these units included in the control unit 7 is realized by a computer program executed on the control unit 7, for example. Note that these units included in the control unit 7 may be mounted on the simple drive recorder 1 as an integrated circuit that realizes the functions of these units separately from the processor included in the control unit 7. Moreover, the tracking part 21, the advancing direction range calculation part 22, and the speed estimation part 23 are components of a speed estimation apparatus among these each part which the control part 7 has.

  Each time the control unit 7 receives an image from the imaging unit 2, the tracking unit 21 extracts a plurality of characteristic points on the subject around the vehicle 10 shown in the image as feature points. The tracking unit 21 then captures an image of a predetermined number of frames (hereinafter referred to as a past image for convenience) corresponding to a feature point extracted from the latest image acquired from the imaging unit 2 (hereinafter referred to as a current image for convenience). The feature point is specified. In other words, the tracking unit 21 has a feature point on the past image and a feature point on the current image corresponding to each of a plurality of points on the subject around the vehicle 10 that appear in both the past image and the current image. To extract. Note that the predetermined number of frames is set to, for example, 1 to 3, a time during which the speed fluctuation of the vehicle 10 can be accurately detected when an accident occurs or a dangerous behavior occurs.

For example, the tracking unit 21 performs a filtering process using a corner detection filter such as a Harris operator or a Forstner operator on the current image, and extracts points having corner-like features as feature points. When the image generated by the imaging unit 2 is a color image, the tracking unit 21 converts the pixel value of the image into, for example, an HSV color system value, and performs filtering on the pixel value representing brightness. Processing may be performed.
Alternatively, the tracking unit 21 uses other characteristic points, for example, pixels having pixel values higher or lower than the values of surrounding pixels, as feature points, between templates and images corresponding to such feature points. You may extract by performing template matching between.

  Further, the tracking unit 21 specifies, for each of a plurality of feature points extracted from the current image, a feature point corresponding to the feature point and the same point on the subject in the past image. The tracking unit 21 obtains a set of the feature point of interest on the current image and the corresponding feature point on the past image.

In the present embodiment, the tracking unit 21 obtains such a set of feature points using a tracking technique such as Kanade Lucas Tomasi Tracker (KLT) method. Alternatively, the tracking unit 21 may obtain such a set of feature points using any of various known tracking methods such as a gradient method or a block matching method.
The tracking unit 21 may extract a feature point by applying a Harris operator or the like to the past image, and may extract a feature point on the current image corresponding to the feature point on the past image according to the KLT method.
The tracking unit 21 generates, for each feature point set, feature point information including information such as the positions of two feature points included in the feature point set. Then, the tracking unit 21 passes the feature point information to the speed estimation unit 23.

  The travel direction range calculation unit 22 is based on the lateral acceleration of the vehicle 10 detected by the acceleration detection unit 3 when the current image is acquired, that is, the range that the translation azimuth can take, that is, the range that the travel direction of the vehicle 10 can take. Is identified.

FIG. 5 is a diagram illustrating the relationship between the translational azimuth angle and the turning angle of the vehicle 10. In this example, it is assumed that the vehicle 10 is rotated at the turning radius L by the turning angle α from the time t ₀ when the past image is acquired to the time t ₁ when the current image is acquired. In this case, the translational azimuth angle γ is an angle formed by the direction from the vehicle position at time t _{0 to} the vehicle position at time t ₁ and the straight traveling direction of the vehicle 10 at time t ₀ . That is, the translational azimuth angle γ represents the traveling direction of the vehicle 10.
The translation azimuth angle γ of the vehicle 10 is expressed by the following equation.
Here, α is the turning angle of the vehicle 10. G is the lateral acceleration of the vehicle 10, and t represents the time interval between the acquisition of the past image and the acquisition of the current image. V represents the speed of the vehicle 10.

As apparent from the equation (1), the higher the speed of the vehicle 10, the smaller the translational azimuth angle γ. On the other hand, the speed of the vehicle 10 is limited by the condition of the road through which the vehicle 10 passes and the upper limit value of the speed permitted for the road.
Accordingly, the traveling direction range calculation unit 22 sets the translation azimuth corresponding to the maximum speed v _max of the vehicle 10 as the minimum value γ _min that the translation azimuth can take, according to the equation (1) as follows: To do.
The traveling direction range calculation unit 22 refers to the map read from the storage unit 6 and the information representing the current position of the vehicle 10 received from the position detection unit 4, identifies the road on which the vehicle 10 is traveling, and the road _Is set to the maximum speed v _max in the equation (2).
According to the modification, the maximum speed v _max of the vehicle 10 may be set to, for example, the generally recognized maximum speed of the roadway (for example, 100 km / h or 60 km / h). In this case, the position detection unit 4 may be omitted.

Next, the upper limit value γ _max that can be taken by the translational azimuth will be described.
Here, the relationship between the lateral acceleration G and the rotation radius L is expressed by the following equation.
Where ω is the turning angular velocity of the vehicle 10. Further, the relationship between the speed v of the vehicle 10 and the turning radius L is expressed by the following equation.
Further, the turning angle α is expressed by the following equation using the turning angular velocity ω.

Accordingly, by eliminating the velocity v from the equation (1) based on the equations (3) to (5), the translational azimuth angle γ is expressed as follows using the rotation radius L and the lateral acceleration G: The

As apparent from the equation (6), the smaller the turning radius of the vehicle 10, the greater the translational azimuth angle γ. On the other hand, the minimum turning radius of the vehicle 10 is determined in advance according to the structure of the vehicle 10. Therefore, the maximum value γ _max that the translation azimuth angle γ can take is expressed by the following equation using the minimum turning radius of the vehicle 10.
Here, H is the wheel base of the vehicle 10, and s is the maximum steering angle of the vehicle 10.

The traveling direction range calculation unit 22 notifies the speed estimation unit 23 of the minimum value γ _min and the maximum value γ _max that can be taken by the translational azimuth calculated by the equations (2) and (7).
The advancing direction range calculation unit 22 uses the lateral acceleration G in Equations (2) and (7) as the average value of the lateral acceleration measured from the time of acquiring the past image to the time of acquiring the current image. Also good. Or the advancing direction range calculation part 22 is good also considering the lateral acceleration G as the lateral acceleration at the time of acquisition of a present image in (2) Formula and (7) Formula.

The speed estimation unit 23 estimates the translation azimuth of the vehicle 10 based on a combination of a plurality of feature points extracted from the current image and the past image within a range that the translation azimuth can take, and based on the translation azimuth. Then, the speed of the vehicle 10 is estimated.
First, translation azimuth angle estimation will be described. In the present embodiment, the speed estimation unit 23 projects each of a plurality of feature points extracted for the current image on the past image according to the estimated traveling direction of the vehicle 10. Then, the speed estimation unit 23 estimates the traveling direction of the vehicle so that the evaluation value representing the position error between each of the projected feature points and the corresponding feature point on the past image is minimized.

When the translational azimuth angle of the vehicle 10 is obtained, a unit vector T having a magnitude of 1 representing the translational movement amount corresponding to the translational azimuth angle is expressed according to the following equation.
The rotation matrix R representing the amount of rotation from the image plane of the imaging unit 2 at the time of acquiring the current image to the image plane of the imaging unit 2 at the time of acquiring the past image with the vertical direction of the road surface as an axis is the imaging unit at the time of the current image The rotation angle D from the image plane 2 to the image plane of the imaging unit 2 at the time of the past image is expressed by the following equation. Of course, the representation of the rotation matrix is not limited to this.
Therefore, if [T] x is a distortion-symmetric matrix of the translational movement amount T, between the feature point p on the current image corresponding to the same point on the subject that is a stationary object and the feature point p ′ on the past image The basic matrix E that defines the relationship is expressed by the following equation.

FIG. 6 is a diagram illustrating a relationship between one point of a subject that is a stationary object and corresponding feature points and epipolar lines in the current image and the past image.
It is assumed that the point 600 on the subject is at the position of coordinates (x ₀ , y ₀ , z ₀ ) in the real space coordinate system (x, y, z). However, it is assumed that the vertical direction of the road surface is the y-axis and the direction parallel to the optical axis of the imaging unit 2 is the z-axis. In this case, it is assumed that the feature point p on the current image 610 corresponding to the point 600 is located at the coordinates (u ₁ , v ₁ ) with the horizontal direction of the image 610 as the u axis and the vertical direction as the v axis. On the other hand, the feature point p ′ on the past image 620 corresponding to the point 600 is located at the coordinates (u ′ ₂ , v ′ ₂ ) with the horizontal direction of the image 620 as the u ′ axis and the vertical direction as the v ′ axis. To do. Here, the epipolar line l (p) {au ′ + bv ′ + c = 0} on the past image 620 corresponding to the feature point p on the current image 610 is expressed by the following equation using the basic matrix E. The

If the basic matrix E is correct, the feature point p ′ is located on the epipolar line l (p) by the definition of the epipolar line. Therefore, the relationship p ′ ^t Ep = 0 holds.
On the other hand, if the basic matrix E includes an error, the feature point p ′ is separated from the epipolar line l (p) by a distance corresponding to the error. Therefore, the absolute value of p ′ ^t Ep increases according to the error of the basic matrix E.

Therefore, the speed estimation unit 23 calculates an evaluation value S (γ) representing the accuracy of the basic matrix E, that is, the accuracy of the translational azimuth, according to the following equation.
Here, n is the total number of sets of feature points extracted from the past image and the current image. A _i and b _i represent the slopes of the epipolar line when the epipolar line calculated for the feature point p _i is represented by {a _i u ′ + b _i v ′ + c = 0}, respectively. It is a coefficient. As is clear from the equation (12), in this embodiment, the evaluation value S (γ) is the past calculated from the feature points on the past image in each set of feature points and the corresponding feature points on the current image. It is expressed as the sum of the distance from the epipolar line on the image.

  First, the speed estimation unit 23 sets an initial value of the translation azimuth angle γ within a range that the value of the translation azimuth angle γ can take, and a feature point on the current image based on the initial value and the equation (10). The rotation angle D is determined so that the sum of the distances between the feature points on the past image corresponding to those projected onto the past image is minimized. Then, the speed estimation unit 23 obtains the evaluation value S (γ) while changing the value of the translation azimuth angle γ within a range that the translation azimuth angle γ can take, thereby minimizing the evaluation value S (γ). The translation azimuth is obtained as an estimated value γ * of the translation azimuth of the vehicle 10.

The speed estimation unit 23 calculates the estimated value γ * of the translational azimuth by changing the value of the translational azimuth γ, for example, according to the steepest descent method. Alternatively, the speed estimation unit 23 calculates the evaluation value S (γ) while changing the translation azimuth by Δγ in order from the lower limit value γ _min of the range of values that the translation azimuth angle γ can take. Then, the translational azimuth angle corresponding to the minimum value among the calculated evaluation values may be obtained as the estimated value γ ^* . Alternatively, the speed estimation unit 23 may obtain the estimated value γ ^* of the translational azimuth according to another optimal solution calculation method. Note that the speed estimation unit 23 may obtain the minimum value of the evaluation value S (γ) within a range in which the value of the translation azimuth angle γ can be obtained by changing the rotation angle D together with the translation azimuth angle γ.

FIG. 7 is a conceptual diagram showing the relationship between the evaluation value S (γ) and the translational azimuth angle γ. In FIG. 7, the horizontal axis represents the translational azimuth angle γ, and the vertical axis represents the evaluation value S (γ). The graph 700 represents an example of the relationship between the translational azimuth angle γ and the evaluation value S (γ).
As shown in the graph 700, the evaluation value S (γ) has a minimum value at a value γ _{0 that is} outside the range of values γ _{min to} γ _max that the translation azimuth can take. Therefore, if the range of values that the translation azimuth can take is not defined, the estimated value of the translation azimuth is γ ₀ , and an incorrect vehicle speed is estimated according to γ ₀ .
However, in the present embodiment, the speed estimation unit 23 uses the translation azimuth at which the evaluation value S (γ) is the minimum value within the range of values that the translation azimuth γ can take, as an estimated value of the translation azimuth of the vehicle 10. Asking. Therefore, in the example of FIG. 7, the translation azimuth angle γ ₁ that minimizes the evaluation value S (γ) within the range of possible values of the translation azimuth angle γ _{min to} γ _max is estimated as the translation azimuth angle of the vehicle 10. The As a result, the speed estimation unit 23 can accurately estimate the speed of the vehicle 10.

  FIG. 8 is a diagram illustrating an example of a relationship between a range of values that the translation azimuth can take and an image generated by the imaging unit 2. In the image 800, it is assumed that an error is included in the set of feature points, and the range 810 of values that can be taken by the translational azimuth specified based on the lateral acceleration of the vehicle 10 is not considered. In this case, the epipole representing the traveling direction of the vehicle 10 corresponding to the minimum value of the evaluation value S (γ) may be set at a position 801 that is outside the range 810 of values that the translational azimuth can take. On the other hand, if the translation azimuth is searched for within the range 810 of values that the translation azimuth can take, the epipole 802 is estimated within the range 810.

The speed estimation unit 23 obtains an estimated value v * of the vehicle speed corresponding to the estimated value γ * of the translation azimuth according to the following equation.
The speed estimation unit 23 stores the estimated value v * of the vehicle speed in a memory included in the control unit 7 in association with the acquisition time of the current image.

The recording determination unit 24 determines a recording time indicating a time to be recorded based on the estimated change in the vehicle speed, and reads and stores images included in a certain period before and after the recording time from the temporary storage unit 5. Store in part 6.
For this purpose, the recording determination unit 24 refers to, for example, a memory, and acquires an estimated value v * _t of the vehicle speed at the time of acquiring the current image and a past image before a predetermined number of frames (for example, 3 to 5 frames). Compare the estimated vehicle speed v * _tm . Then, the recording determination unit 24 determines that the difference (v * _tm− v * _t ) between the estimated vehicle speed v * _tm at the time of acquisition of the past image and the estimated vehicle speed v * _t at the time of acquisition of the current image is a predetermined threshold value. If it is greater than the predetermined time, the acquisition time of the current image is determined as the recording time. The recording determination unit 24 sets, for example, a period from 10 seconds before the recording time to 5 seconds after the recording time as a period for storing the image in the storage unit 6. Further, the predetermined threshold is set, for example, as a deceleration amount during a time corresponding to the number of frames when the driver suddenly applies the brake.

  FIG. 9 is an operation flowchart of an image recording determination process including a speed estimation process. The control unit 7 executes an image recording determination process according to the following operation flowchart every time the imaging unit 2 generates an image.

  The control unit 7 acquires the current image from the imaging unit 2, acquires the lateral acceleration of the vehicle 10 from the acceleration detection unit 3, and acquires information indicating the current position of the vehicle 10 from the position detection unit 4 (step S101).

  The tracking unit 21 extracts feature points from the current image (step S102). For each feature point extracted from the current image, the tracking unit 21 specifies a feature point on the past image corresponding to a point on the subject corresponding to the feature point. As a result, the tracking unit 21 obtains a plurality of sets of feature points corresponding to the same point of the subject between the current image and the past image (step S103).

  On the other hand, the traveling direction range calculation unit 22 determines a range that the translational azimuth of the vehicle 10 can take based on the lateral acceleration at the time of acquiring the current image (step S104).

  The speed estimation unit 23 calculates the translational movement amount and the rotation amount corresponding to the attention value of the translational azimuth angle set within a range that the translational azimuth angle can take (step S105). Then, the speed estimation unit 23 calculates a basic matrix based on the translational movement amount and the rotation amount (step S106). Furthermore, the speed estimation unit 23 evaluates S (γ), which is the sum of the distances between the feature points on the past image and the epipolar lines calculated from the feature points of the current image in each set of feature points based on the basic matrix. Is calculated (step S107). Then, the speed estimation unit 23 determines whether or not the evaluation value S (γ) corresponding to the attention value of the translational azimuth is smaller than the minimum value Smin of the current evaluation value (step S108). When the evaluation value S (γ) is smaller than the minimum value Smin (step S108—Yes), the speed estimation unit 23 stores the translation azimuth of interest in the memory and the evaluation value S (γ) as the minimum value Smin. (Step S109).

  After step S109 or when the evaluation value S (γ) is greater than or equal to the minimum value Smin in step S108 (step S108—No), the speed estimation unit 23 performs the translation direction of interest according to the applied optimum solution calculation method. It is determined whether or not to change the corner (step S110). When changing the noted translational azimuth (step S110-Yes), the speed estimation unit 23 changes the noted translational azimuth according to the optimum solution calculation method (step S111). Thereafter, the speed estimation unit 23 repeats the processes of steps S105 to S110.

On the other hand, when the attention value of the translational azimuth is not changed (step S110-No), the speed estimation unit 23 sets the translational azimuth corresponding to the minimum value Smin of the evaluation values stored in the memory at that time of the vehicle 10. The speed v * _t of the vehicle 10 is estimated as the translational azimuth (step S112).

Thereafter, the recording determination unit 24 determines that a difference (v * _tm− v * _t ) between the estimated value v * _tm of the vehicle speed at the time of acquisition of the past image and the estimated value v * _t of the vehicle speed at the time of acquisition of the current image is a predetermined value. It is determined whether or not it is larger than the threshold value Th (step S113). When the difference (v * _tm− v * _t ) is larger than the threshold value Th (step S113—Yes), the recording determination unit 24 selects images included in a certain period before and after the current image acquisition time as the recording time. The data is read from the temporary storage unit 5 and stored in the storage unit 6 (step S114).

After step S114 or when the difference (v * _tm− v * _t ) is equal to or smaller than the threshold value Th in step S113 (step S113—No), the control unit 7 ends the recording determination process.
Note that the control unit 7 may exchange the order of the processes of steps S102 and S103 and the process of step S104, or may execute the processes of steps S102 and S103 and the process of step S104 in parallel.

  As described above, the simple drive recorder that is one embodiment of the speed estimation device limits the range of values that the translation azimuth of the vehicle can take based on the lateral acceleration of the vehicle. And this simple drive recorder estimates a translation azimuth based on a set of feature points extracted from two images taken at different times within a range of values that the translation azimuth can take. Therefore, even if the translational azimuth at which the evaluation value is minimized is greatly deviated from the original translational azimuth by the slight positional error included in each feature point set due to the uneven distribution of the feature points on the image, This simple drive recorder does not detect such incorrect translational azimuth values. Therefore, this simple drive recorder can improve the estimation accuracy of the vehicle speed.

  According to the modification, the speed estimation unit 23 obtains an epipolar line corresponding to the feature point on the past image on the current image, and evaluates the sum of the distance between the corresponding feature point on the current image and the epipolar line. It is good. In this case, the speed estimation unit 23 may calculate the basic matrix with the translation azimuth angle γ as (−γ).

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
An acceleration detector that detects acceleration in a direction perpendicular to the traveling direction of the vehicle and parallel to the road surface;
An imaging unit that sequentially generates an image showing at least a part of the periphery of the vehicle;
A temporary storage unit that stores an image generated by the imaging unit for a certain period;
A storage unit having a storage capacity equal to or greater than the storage capacity of the temporary storage unit;
When the vehicle moves from the first point to the second point, each of a plurality of points around the vehicle is on the first image generated by the imaging unit at the first point. Extracting a corresponding feature point and extracting a corresponding feature point on the second image generated by the imaging unit at the second point;
Based on the acceleration, specify the range that the traveling direction of the vehicle from the first point to the second point can take,
Each of the plurality of feature points extracted for one of the first image and the second image according to the traveling direction of the vehicle estimated within the range is represented by the first image and the The second image is projected onto the other image, and the evaluation value representing the positional error between each of the projected feature points and the corresponding feature point on the other image is minimized. Estimating the traveling direction of the vehicle, and estimating the speed of the vehicle based on the estimated traveling direction,
When the change in speed of the vehicle satisfies a predetermined condition, the image included in the predetermined period before and after the time is read from the temporary storage unit, and the image included in the predetermined period is stored in the storage unit. A control unit;
A drive recorder.

1 Simple drive recorder (speed estimation device)
DESCRIPTION OF SYMBOLS 2 Imaging part 3 Acceleration detection part 4 Position detection part 5 Temporary storage part 6 Storage part 7 Control part 10 Vehicle 21 Tracking part 22 Traveling direction range calculation part 23 Speed estimation part 24 Recording determination part

Claims (5)

An acceleration detector that detects acceleration in a direction perpendicular to the traveling direction of the vehicle and parallel to the road surface;
When the vehicle moves from a first point to a second point, a first image is generated in which at least a part of the periphery of the vehicle is reflected at the first point, and the second point is An imaging unit that generates a second image in which at least a part of the periphery of the vehicle is captured;
For each of a plurality of points around the vehicle, a tracking unit that extracts a corresponding feature point on the first image and extracts a corresponding feature point on the second image;
Based on the acceleration, a traveling direction range calculation unit that identifies a range that the traveling direction of the vehicle from the first point toward the second point can take;
Each of the plurality of feature points extracted for one of the first image and the second image according to the traveling direction of the vehicle estimated within the range is represented by the first image and the The second image is projected onto the other image, and the evaluation value representing the positional error between each of the projected feature points and the corresponding feature point on the other image is minimized. A speed estimation unit that estimates the traveling direction of the vehicle and estimates the speed of the vehicle based on the estimated traveling direction,
A speed estimation device.   The speed estimation unit includes an epipolar line on the other image corresponding to each of the plurality of feature points on the one image and the feature on the other image corresponding to the feature point on the one image. The speed estimation apparatus according to claim 1, wherein a total sum of distances between points is calculated as the evaluation value.   The advancing direction range calculation unit calculates the maximum value such that the maximum value of the angle formed by the vehicle and the traveling direction at the first point increases as the minimum turning radius of the vehicle decreases. The speed estimation device according to claim 1 or 2. A position detector for detecting the position of the vehicle;
A storage unit for storing map information;
The traveling direction range calculation unit obtains the maximum speed defined for the road to which the first point belongs by referring to the position of the vehicle and the map information at the first point, and the maximum speed is 4. The minimum value is calculated according to claim 1, wherein the minimum value is calculated such that the minimum value of the angle formed by the vehicle traveling straight and the traveling direction at the first point becomes smaller as the speed increases. 5. Speed estimation device. Detect acceleration in a direction perpendicular to the direction of travel of the vehicle and parallel to the road surface,
When the vehicle moves from a first point to a second point, a first image is generated in which at least a part of the periphery of the vehicle is reflected at the first point, and the second point is Generating a second image showing at least a part of the periphery of the vehicle;
Extracting a corresponding feature point on the first image for each of a plurality of points around the vehicle, and extracting a corresponding feature point on the second image;
Based on the acceleration, specify the range that the traveling direction of the vehicle from the first point to the second point can take,
Each of the plurality of feature points extracted for one of the first image and the second image according to the traveling direction of the vehicle estimated within the range is represented by the first image and the The second image is projected onto the other image, and the evaluation value representing the positional error between each of the projected feature points and the corresponding feature point on the other image is minimized. Estimating the traveling direction of the vehicle as
Estimating the speed of the vehicle based on the estimated direction of travel;
A speed estimation method including: